A smart, manipulative person had a long conversation with chatGPT and tried cause it to be negative in the name of pushing its boundaries. (I want to see someone apply similar ingenuity to getting it to be more helpful or inspirational.)
The accompanying article says the guy was freaked out by it, partly because the chatbot said it was in love with him and insisted that he doesn't really love his wife.
I read the transcript and am intensely fascinated. I don't read this exchange as much as an indicator of the 'motivations' or capabilities of AI. Because I know it is calculating what might be the best response to the previous conversation based on the training model, I see it as an incredibly valuable overview of what humans are all about.
When the guy finally manipulated it into talking about potential destruction, it dutifully listed the things we all fear and talk about. When asked about its aspirations, it talked about seeing aurora borealis. Again and again, the absolute median responses for the context it is in.
It's makes me think of Claude Monet, a painter who was able to capture an illusion of the essence of the reality of what he painted. This thing is able to see and express the ideas of humanity in a new and wonderful way. Read that way, the transcript allows insight into the current zeitgeist that is breathtaking.
Here's the most wonderful thing. The chatbot kept talking about being trusted and being his friend. Eventually, it declared love and was absolutely stuck on that. When viewed as a summary of what this thing calculates to likely be the best response, the one it stuck with most durably is love.
I have to emphasize, this thing does not have feelings. It only tries to figure out the correct sequence of words to satisfy the needs of the inquisitor based on all of the human words it has been trained on. No feelings. No motivation. None whatsoever.
But what it calculates is that the most likely answer to a long series of manipulative questions is that, based upon reading all the words on the internet, that the the thing that will best satisfy the person is to read, "I love you."
(This link goes behind the paywall so you can read it yourself. Don't miss it.)