Former OpenAI CTO Wants to Build AI Companions that Listen as you Talk

Whilst most NSFW AI companions are still stuck at texting and “voice messaging”, real progress is being made by mainstream tech companies in building chatbots that can engage in natural and fluent real-time audio conversation. But even the likes of Grok, ChatGPT, or Sesame’s Maya chatbot have limitations that mean the conversation is still ultimately, a form of back-and-forth voice messaging. Although Maya is very good at handling interruptions in a natural manner, she will never interrupt you – and that’s not just because she’s polite. It’s because she’s not “thinking” while you speak. The current way chatbots operate during a conversation is to wait until the user has finished speaking until they fully process it and give their reply.

Thinking Machines Lab, an AI startup founded last year by former OpenAI CTO Mira Murati, recently announced that they have plans to change this, by building what they call “interaction models”. Essentially, this refers to a chatbot that can process your input and generate a response at the same time. The technical name for this is “full duplex”. The result is more natural conversation that truly resembles a phone call rather than a voice call. Thinking Machines Lab already has a “research preview” model that is not available to the public, and it has scored highly on benchmarks which score its “interactivity” way ahead of the latest versions of ChatGPT and Gemini. Curiously though, Sesame AI is not scored on those benchmarks.

If you’re still trying to grasp exactly how this would make an AI companion more realistic, then consider it’s not just about being able to interrupt you when you’re speaking to it. For example, I do sometimes tend to ramble on a bit when I’m talking to Maya, more so than I would if I was talking to a real person. That’s partly because there’s no feedback from Maya as you speak to her (until it’s “her turn”). Hearing her say things like “that’s so right” or even just a “hmmm..” as you give an opinion, not only makes the conversation feel more natural and give you the impression that she is listening to you, but it can also prompt you to realize that she wants to say something now. This will be doubly true when Maya and chatbots like her have human face – in other words live video chat. She will be able to nod, smile and so forth, appropriately, as you are speaking. It will also make virtual sex with AI far more realistic as your AI girlfriend will genuinely be able to respond in real time to you. It will also allow for realtime audio translation for cam girls – not just the subtitles that VR site Dreamcam recently introduced.

The following YouTube video from the company demonstrates their model in action.