ElevenLabs Launches Eleven Realtime: A Voice Agent in a Single Model, 98.8% on Big Bench Audio

ElevenLabs has released Eleven Realtime, a new class of speech-to-speech models that package an entire voice agent into a single model. Previously, a developer had to pick speech recognition, a voice activity detector, turn-taking logic, an LLM and speech synthesis separately, then spend months tuning them to work together. Now there are two options: Eleven Realtime v1 Mini and v1 Max. In independent tests by Artificial Analysis, Max answers 98.8% of Big Bench Audio voice questions correctly and starts speaking after 0.57 seconds on average.

One Module Instead of Five

ElevenLabs' main argument is that a natural-sounding voice agent is built from many small decisions: how long to wait after a pause before replying, how to react when the user interrupts, how the voice, prompt and model fit together. In the classic architecture, each of these decisions lives in a separate component with its own settings, and the seams between them become the main source of unnatural behavior. Realtime models take this work on themselves. On the ElevenAgents platform, the same parameters can still be configured manually.

Both models support more than 90 languages and work with any of the 17,000+ voices in the ElevenLabs library.

• Eleven Realtime v1 Mini: natural conversations at a low price. It suits AI assistants, tutors and interactive characters. The company believes it is enough for most scenarios.
• Eleven Realtime v1 Max: complex tasks such as customer support, lead qualification and multi-step processes, where the agent must follow a procedure and reason step by step.

What the Benchmarks Show

The models were run independently through three public tests: Big Bench Audio (reasoning in voice mode), Full-Duplex-Bench (conversation dynamics) and Speech Agent Arena (human preference and task success).

• Big Bench Audio, 1000 spoken questions: v1 Max scored 98.8% correct answers, v1 Mini 75.9%.
• Time to first response: v1 Max 0.57 s, v1 Mini 0.62 s.
• Full-Duplex-Bench (pauses, turn-taking, interruptions, backchanneling): v1 Mini 95.8%, v1 Max 94.1%.
• Speech Agent Arena (blind comparison in live conversations): v1 Max 998, v1 Mini 988.
• Tasks involving tool calls: v1 Max completed 87.1%, v1 Mini 60.3%.

ElevenLabs separately highlights the balance between accuracy and speed. Typically, the models with the best Big Bench Audio results are the slowest to respond, while the fastest lose accuracy. According to the test data, Max sits on the Pareto frontier on both axes: it is among both the most accurate and the fastest. The gap on tool use is the most noticeable. If an agent has to query a CRM, book slots or check an order status, Mini at 60.3% will be a weak choice, which is why the company recommends Max for scenarios with heavy function calling.

Migrating from the OpenAI Realtime API

ElevenLabs' most practical move is compatibility with the OpenAI Realtime API. The endpoint works over WebSocket and follows the OpenAI specification, so the official OpenAI SDK works as is. The example in the documentation uses the address wss://api.elevenlabs.io/v1/convai/realtime and the model eleven_realtime_v1_mini, with the voice selected by a parameter such as brian. For anyone who has already built an integration, trying the new model comes down to changing the address and the model name.

There are limitations. The Realtime API works only over WebSocket, so integrations built on WebRTC will first have to be moved to WebSocket. In addition, the API runs separately from ElevenAgents: it has no channels, knowledge base or other platform features.

Where to Start

For production use, the company recommends going through ElevenAgents: create an agent or open an existing one and pick a Realtime model from the dropdown. Everything needed for operation comes with it: telephony, web, WhatsApp and CCaaS channels, workflows, a knowledge base, guardrails, testing and monitoring.

In practice, the choice looks like this. For a voice tutor, a game character or a simple assistant, start with Mini: it scores higher on conversational naturalness on Full-Duplex-Bench and costs less. For a contact center, lead qualification and scenarios with several steps and tool calls, Max is the better pick. If you already have code built for OpenAI Realtime, it makes sense to run an A/B test on your own conversations: swap the endpoint and model, then compare latency and the share of completed tasks. If your architecture depends on WebRTC, plan time for the move to WebSocket in advance.