FreshSynapse

AI trends & daily updates, in plain English

AI Tools

ElevenLabs Unveils Its Interaction Engine and 13 Upgrades — Voice AI's Move From Models to Memory

By FreshSynapse Editorial · · 4 min read

Two women recording a podcast in a bright studio, each speaking into a microphone on a boom arm

Photo: Hippopx (public domain). Illustrative image.

What happened. ElevenLabs, the AI voice company, announced 13 new products and upgrades at its New York Summit on Wednesday 11 November 2026, in a package aimed squarely at enterprises rather than hobbyists. The announcements cover four areas: custom models tuned on live data, smarter agents that remember customers and automate follow-up work, a workplace app for meetings, and a partner channel backed by Nvidia infrastructure. Most of the new items are previews, early access or phased rollouts rather than fully available products — so the immediate effect is a roadmap for enterprise buyers, not a finished catalogue.

Three new voice models. Over the last month, ElevenLabs researchers have shipped three new voice models, all of which the company says lead independent benchmarks — a company claim, not an independent finding. Scribe v3 is a speech-to-text transcription model covering 102 languages, which ElevenLabs says produces 27% fewer word errors than Scribe v2 on its own evaluation. Eleven v4 is a text-to-speech model that reads the tone, pacing and context of a script for more expressive speech. Eleven Realtime is a new class of speech-to-speech models built for fast, accurate voice agents. The company also launched Scribe v3 Medical, a transcription model tuned for healthcare terms, and said it plans to build more models for industries such as financial services.

The Interaction Engine. The centrepiece is a new system to orchestrate voice models and language models across a conversation. In a real conversation, different moments need different models: a cheap one can handle triage and routine answers, while a frontier model takes the complex parts. The Interaction Engine manages turn-taking and interruptions and assigns the right model to each part, balancing performance and reliability against cost. “Interaction Engine orchestrates all this under the hood, so you can be confident you’re delivering the right outcome for the customer, without wasting money on tokens,” said Mati Staniszewski, ElevenLabs’ co-founder and chief executive. What the company has not disclosed is any independent benchmark showing how much businesses could save, or whether the system improves customer resolution rates compared with existing deployments.

Agents that remember, and a workplace app. Alongside the models, ElevenLabs announced upgrades to ElevenAgents, its platform for voice and chat agents. Memory now gives agents the full context of every returning customer, recalling what they have said before — from a new shirt size to a pending move — so nobody has to repeat themselves. Routines add multi-step automation, and enterprise customers can now fine-tune models on their own live data. The company also opened early access for ElevenAI, a new app that acts as an AI teammate for enterprise teams, starting with meetings. A new Partner Program counts more than 550 partners in 44 countries, and ElevenLabs announced an alliance with McKinsey to help organisations build and scale agents, plus an expanded Nvidia partnership: Nemotron 3.5 Super VL is now available in ElevenAgents, and ElevenLabs will add 15 MW of new Vera Rubin compute capacity.

Why it matters. Voice is where AI agents meet customers first — on the phone, in support chats, in meetings — and it is also where their costs are most visible, because every second of conversation burns tokens twice: once to hear, once to think. ElevenLabs is betting that the winner in voice will not be the company with the single best model, but the one that orchestrates many models well and remembers who it is talking to. That is a reasonable bet, and the Interaction Engine is the clearest version of it yet. The test, as always, is the one the site recommends before trusting any new tool: try it on your own real tasks, measure what it actually saves, and judge the claims against your own results before you pay or switch.

Sources

  • ElevenLabs
  • voice AI
  • AI agents
  • AI tools
  • speech

Related articles

AI Tools

How AI Image and Video Generation Works

Type a sentence, get a picture — or a film clip. Here is the idea underneath modern AI image and video generators: diffusion, noise, and why video is the harder problem.

· image generation, video generation, diffusion, explainer