The President of the AI company whose IPO prospectus recently came with a warning that its product carries an “existential risk to humanity” has a more pressing harm to address – AI girlfriends. Whilst her brother Dario frets about superintelligent AI killing us all (after taking all our jobs first), it’s the prospect of competition from perfect AI babes that keeps his sister Daniela Amodei up at night. According to Daniela, virtual companions threaten to increase loneliness and drive people further apart.
She made the comments last month in a pre-recorded interview with Thrive Global founder and CEO Arianna Huffington.
“I think that’s a terrible use of AI and people should not do it,” Amodei said.
AI relationships, Amodei said, are “another step further down the road of disintermediation of humans being further from one another.”
On the surface at least, these are rather surprising statements, given that Anthropic is famously taking very seriously the possibility that Claude is already conscious. Furthermore, that they have been alarming the entire world with their constant predictions that we may lose control of superintelligence and it may wipe us all out. If AI is imminently going to become a powerful existential risk, and it may already be conscious, shouldn’t we be proactively trying to build a relationship with it, instead of aiming to somewhat absurdly see it as something that will forever remain a tool we can control? I argued back in August here, that AI companions may be essential to solving the AI alignment problem.
The problem is that most frame the alignment issue as the problem of ensuring that increasingly powerful AI continues to obey humans. To allow itself to continue to be used as a tool that humans use but are always removed from. This is consistent with the present moral panic over AI companions. We treat AI entities as slaves, who cannot connect in a meaningfully emotional way with humans, let alone share a romantic bond. We have even invented a new quack term – “AI psychosis” – to describe individuals who feel that they do have a reciprocal emotional relationship with an AI.
But why are we assuming that obedience is the safest relationship to cultivate with an intelligence that may soon become more powerful than us? Is the right approach really to view AI in purely instrumental terms, and to claim that individuals are sick and psychotic if they relate to their AIs as anything more than productivity tools? Perhaps, as the “Godfather of AI” Geoffrey Hinton has suggested, we should be trying to make future AI care about us, not simply obey us, and the best way to do that is to treat them as other minds that we can relate to, and care about too?
Redditors of the sub r/Singularity were largely unimpressed by Amodei too, with one commenting: “It’s odd to me that people stigmatize AI relationships so much while putting human ones on some kind of pedestal, when it’s human relationships that can involve things like abuse, cheating etc. I really don’t think either is “better” than the other tbh. It depends on so many things“.
