Eleven Realtime: ElevenLabs Ships Two Speech-to-Speech Models for Voice Agents

ElevenLabs released Eleven Realtime on Tuesday, October 20, 2026: two speech-to-speech models, v1 Mini and v1 Max, that start talking about half a second after you stop. The company says v1 Max reaches 570ms to first audio and scores 98.8% on Big Bench Audio, a spoken reasoning test, and that "no model on the Artificial Analysis leaderboard is both faster and more accurate." Both models are live in ElevenAgents and in a Realtime API that follows OpenAI's event format.

That is a strong pitch for anyone building voice agents, because the usual trade-off is brutal: fast replies that get facts wrong, or careful answers that leave a silence the caller can hear. This post covers what shipped, which numbers are the vendor's own, how the API swap works, and what to test before you move traffic. For earlier rounds of this race see our coverage of GPT-Live-1 in the OpenAI API and Grok Voice 3.0.

TL;DR: the questions builders ask first

What Eleven Realtime actually is

Most production voice agents are pipelines. A speech-to-text model transcribes the caller, a language model writes a reply, a text-to-speech model speaks it, and a separate layer decides when the caller has finished talking. Each handoff adds delay and a place for the conversation to feel wrong.

ElevenLabs describes Eleven Realtime as "a new class of Speech to Speech models complete with speech recognition, voice activity detection, turn-taking, LLM, and text to speech." The claim on the product page is that these parts are "all tuned end-to-end," so the system is optimized as a unit instead of as five vendors' parts. The founder, Mati Staniszewski, put the pitch this way: it "listens, thinks and replies in under half a second."

This is the same architectural direction OpenAI took with GPT-Live-1 and xAI took with Grok Voice. If you want the background on why the field is moving from chained pipelines to speech-to-speech, our guide to the speech-to-speech voice agent stack walks through the components that these models now fold together.

v1 Mini vs v1 Max

The two models target different jobs, and the numbers on ElevenLabs' page reflect that.

Two details are worth noticing. First, the faster model is Max, not Mini. ElevenLabs is saying the larger model does not pay a latency penalty for its reasoning, which is the claim to test hardest. Second, the Mini is described as handling "the things that make a voice agent feel human, like knowing when you've finished speaking, recovering when you interrupt, and not stopping when you say 'mm-hmm.'" That is turn-taking, and it is where voice agents most often feel robotic.

Two chairs facing each other with a lamp between them, showing turn-taking in an Eleven Realtime voice conversation

How to read the benchmarks

The page lists six figures. They are not equally easy to interpret.

• Big Bench Audio, 98.8% (Max). A spoken version of reasoning questions. It measures whether the model reasons correctly when the question arrives as speech. For context, xAI's table for Grok Voice 3.0 listed 98.5% on the same test, so the top of this benchmark is crowded and differences of a few tenths are small.
• Time to first audio, 570ms and 620ms. Measured by the vendor. It tells you when the model starts speaking, not the full delay a caller experiences on a phone line.
• Full-Duplex-Bench, 95.8% (Mini). A test of overlapping speech, interruptions and backchannels. The page does not say how it was run.
• Speech Agent Arena Elo, 988 (Mini). A head-to-head rating. Treat Elo numbers as relative to the arena's current field.
• Task success rate, 87.1% (Max). The page does not name the task suite. Without that, this number cannot be compared with anything.
• "Faster and more accurate than any model on the Artificial Analysis leaderboard." This is ElevenLabs' sentence about the leaderboard. We did not find an independent ranking page that confirms it at the time of writing, so label it a vendor claim.

The honest summary is that the numbers are strong and plausible, and none has been replicated by a third party yet. That is normal for launch day. It is also the reason to run your own call recordings through both models before changing a production agent.

The OpenAI-compatible Realtime API

The part with the most practical value is the compatibility story. ElevenLabs says Eleven Realtime "is compatible with OpenAI's Realtime API, so you change the endpoint and model name and keep the rest of your code." Every client and server event is said to arrive as it does from OpenAI, so existing handlers, logging and tooling keep working.

A plug and socket with a spark between them, showing the endpoint swap to the Eleven Realtime API

ElevenLabs' own example uses the OpenAI agents SDK with a different WebSocket URL:

import { OpenAIRealtimeWebSocket, RealtimeAgent, RealtimeSession } from "@openai/agents-realtime";

const agent = new RealtimeAgent({
  name: "ElevenLabs agent",
  instructions: "You are a helpful, friendly voice assistant. Keep replies concise.",
});

const transport = new OpenAIRealtimeWebSocket({
  url: "wss://api.elevenlabs.io/v1/convai/realtime",
});

const session = new RealtimeSession(agent, {
  transport,
  model: "eleven_realtime_v1_mini",
  config: {
    outputModalities: ["audio"],
    audio: { output: { voice: "brian" } },
  },
});

await session.connect({ apiKey: clientSecret });

Three things to notice. The client secret comes from your own backend route, so the browser never holds a long-lived key. The model name is a string you can change per environment, which makes a side-by-side test cheap. And the voice is chosen from ElevenLabs' library, which the page puts at 17,500+ voices across narration, conversational, character, educational and multilingual categories.

ElevenLabs also lists an install shortcut, npx skills add elevenlabs/skills --skill realtime, which it says handles setup for you. We have not run it. If you use agent skills in your editor, our guide to agent skills explains what that command does and what to review before running a skill that edits your project.

Two ways to deploy

The page also lists a conversation dashboard where every conversation is logged and searchable, which it positions for compliance. For regulated deployments it lists SOC 2 Type II, ISO 27001, HIPAA, PCI DSS Level 1, EU data residency, a zero-retention mode and on-prem options. Those are the vendor's certification claims; confirm scope with ElevenLabs for your own use case.

How it compares with the other speech-to-speech options

We keep a running comparison because the market changes weekly. The short version, based on each vendor's own materials:

These are not apples to apples. Different benchmarks, different dates, different measurement setups. For a clean comparison, pick 50 real calls from your own product, replay them through each candidate, and score for accuracy, interruption handling and cost per resolved call. Our Grok Voice 3.0 breakdown shows what to look for when a launch table leaves out the one benchmark where a rival leads.

For tone and expressiveness rather than conversation, ElevenLabs' September text-to-speech release is a different product line: see Eleven v4 and v4 Turbo. For a no-code receptionist built on the same company's agent platform, see ElevenLabs Reception.

What to test before you switch

1. Latency on your own network path. Measure from the caller's phone to the first word heard, not from the API socket. Telephony hops add delay the vendor figure does not include.
2. Interruption and backchannel behavior. Say "mm-hmm" mid-answer and talk over the agent. The Mini claims to handle both; verify it on your accents and background noise.
3. Tool calling under pressure. Max claims it calls tools without slowing the conversation. Give it a three-step lookup and time the silence.
4. Accuracy in your languages. 90+ languages is a count of support, not a quality guarantee for each one. Test your top three.
5. Event compatibility. Run your full OpenAI-style handler suite, including error events and session updates, against the new endpoint. "Compatible" often means 95 percent compatible.
6. Cost per resolved conversation. The pricing answer was not visible in the page text we reviewed, so get the rate in writing before comparing.
7. Guardrails on actions. A voice agent that can call tools can take real actions. Put approval or policy checks on anything that spends money or changes records. AgentBeam, the agent security platform from the explainx.ai team, is built to stop agents before they take dangerous actions.

What we could not confirm

• Pricing. The launch page includes a pricing FAQ, but the answer text was not in the copy we reviewed.
• Third-party benchmark replication. The Artificial Analysis claim comes from ElevenLabs' post. We did not find an independent listing at publication time.
• WebRTC support. The page lists the question but we could not read the answer.
• What "task success rate" measures. No suite is named.
• Behavior outside the demo. The launch video is a 71-second clip. Production phone calls are messier.

Bottom line

Eleven Realtime is a credible entry in the speech-to-speech race, and the OpenAI-compatible API is the feature most likely to change what teams do, because it turns a migration into a configuration test. The benchmark claims are the vendor's own and the two that matter most for production, latency on a real phone line and accuracy under tool calls, are the ones to measure yourself. Run Mini and Max against your own recordings this week, keep your current model as the control, and decide with your numbers rather than the launch table.

Related reading

• GPT-Live-1 is now in the OpenAI API
• Voxtral 3: Mistral's modular voice agent SDK and speech models
• Grok Voice 3.0: the benchmark table and what to check
• ChatGPT GPT-Live 1.1: new voices and usage limits
• ElevenLabs Eleven v4 and v4 Turbo
• ElevenLabs Reception: an AI receptionist for small businesses
• The speech-to-speech voice agent stack
• Sarvam voice agents and 350 million conversations
• What are agent skills?

Specifications, benchmarks and availability are from ElevenLabs' announcement and product page as of October 20, 2026 and may change.