Our first impressions of ElevenLabs' Eleven Realtime v1 for phone agents
ElevenLabs launched Eleven Realtime v1 on October 20 headlining a 98.8% score on a spoken-reasoning benchmark. On our sample, Max got 48 of 50 right, but asked a yes-or-no question, it talked for 27 seconds before saying "no."
A phone call tests more than reasoning: following a business's rules, hearing the caller, and staying on the line. Over the two days after launch, we tested Max across recorded calls to fictional businesses, simulated calls built from real customer situations, calls on ElevenLabs' own agent platform, and hard-to-hear caller audio.
All clips and transcripts below come from test calls to invented businesses, with invented names, fake phone numbers and example.com emails. Transcripts are short excerpts: […] marks a cut, and lines in parentheses describe what happened behind the scenes; the caller didn't hear them.
1. Reasoning: Max gets the right answer, but talks its way there
Max got 96% on our spoken-puzzle sample, consistent with the published score, but it gets there by talking at length—which a business call can't use—and told to be brief, its score drops.
On our sample of Big Bench Audio, Max missed only 2 questions. It heard every question, and it thinks out loud. Both misses were answers it corrected mid-sentence, and the grader only counts the first answer.
Listen to how Max works through every step out loud for 27 seconds before giving the right answer:
When we told Max to be brief, scores slipped slightly, and there is no setting to turn up its silent thinking time. It's also quick: in our tests its reply started about a third of a second after ElevenLabs detected the caller had stopped, and the first audio arrived in about half a second (not counting the phone network). On simple calls it held natural conversations.
2. Following the rules: It struggles with complex phone scripts
On simple rules Max is solid, but across our calls to fictional businesses 57% had a serious mistake.
We wrote test situations across fictional businesses (pizza, city services, hotel, pest control, electric utility, auto shop, software support, and box office) to probe mistakes we had seen earlier, running each call multiple times. On clear rules, Max did well: every time at our pizza shop, it gave the required peanut-allergy warning and transferred to the manager, and every time it cleanly handled a caller who gave the wrong phone number, asking again, re-checking and reading back the repair status. Across the calls, though, it skipped a confirmation the business required in about one call in eight.
When Max had to check the clock to see an office was closed, it transferred the caller anyway two times out of three. When the business's system said outright that staff were away, Max got it right and didn't transfer on any run, but when it had to work it out from the time of day, it often got it wrong. Here it sends a caller to a closed box office at 1:48 a.m. (most runs):
In another after-hours test, Max announced a transfer and then caught itself mid-sentence half the time (the other half transferred while closed)—a mid-sentence "Actually, ..." take-back that happened in about one call in ten across the test set. Conversely, when rules said to transfer billing callers immediately (staffed around the clock), Max demanded details first every time, making the caller ask repeatedly on most runs.
Max also missed basic math and misstated whether an order was done. For four tickets with a per-ticket fee, it got the $170 total wrong every time, quoting a lower price even though its own checkout link later showed the right amount. After emailing the checkout link on most runs, Max answered "yes" on two of those three when the caller asked whether the unpaid tickets were all set.
In a handful of calls—including half the pizza runs where it placed the order and hung up without reading the order number or saying goodbye—it hung up without a word right after a successful submission.
Across simulated calls based on real customer situations, Max finished the job in 22% of calls and falsely claimed an action was done in 30%.
We also took situations from real inbound calls to our customers (mostly a subscription-billing line, an auto-insurance intake line and a restaurant ordering line) and replayed each one twice as a full simulated call. These are hard situations replayed with strict grading. Max ended fewer than half of these calls properly, and the false claims were orders or requests it said were submitted but had not sent (or a made-up reference number it read out).
3. Hearing: In our tests, when unsure it often acts on a wrong guess
On hard caller audio, Max misheard and acted on the mistake on about one turn in three, compared with about one in twelve for Bolt v2 with Hearing enabled.
ElevenLabs combines speech recognition and voice generation under the hood. On hand-checked caller answers from car-insurance calls, Max's built-in transcript kept the right meaning 77.5% of the time—on par with ElevenLabs' standalone Scribe v2 Realtime without context, and behind the best speech-to-text system we tested.
On tough caller turns (spelled names, emails, phone numbers, noisy lines), Bolt v2 with Hearing enabled heard the caller right on 64% of turns, compared with 40% for Max. Max acted on a wrong guess on about one in three turns (vs. about one in twelve for Bolt v2 with Hearing enabled) and asked the caller to repeat more often. The right answer for each turn was built by an AI from several transcripts and the rest of the call, not by people.
4. Staying connected: Some calls end mid-task
ElevenLabs' server ended 9% of our simulated customer calls partway through, mostly on two of the customer lines.
On three of four runs of our auto-shop booking call (plus all simplified reruns), the call dropped on ElevenLabs' side right after the caller said "Yes, book it":
We also tried ElevenLabs' own agent platform, which they recommend, on calls to fictional businesses. About one in ten of those calls ended the same way. ElevenLabs' records log these as an error in what our side sent, though we were only sending the caller's audio at that moment.
On their platform, Max also acted before the caller approved, or used details the caller had corrected, in a few calls. In one, it put a hotel hold under a made-up guest, "John Doe" at john.doe@example.com, before asking the caller's name. We didn't see it falsely say "it's done" there, but we ran too few calls to say the platform fixes that.
Listen to the calls
Here are the recordings from the rule-following tests described in section 2.
Where Max did well
Where it went wrong
How we tested
We ran these tests on October 20–21, 2026, mostly on Max (Mini only got single-reply and transcription checks). The callers were simulated AI callers with computer-generated voices, calling fictional businesses whose instructions we first wrote for our own system. We connected to Max directly, at ElevenLabs' default settings, and an AI judge graded every call without knowing which system it was grading. Before our final runs, we fixed several bugs in our own test setup and threw out or re-graded the earlier runs.