Our first impressions of the GPT-Live-1 API for voice agents
OpenAI released GPT-Live-1, its new full-duplex voice model, to the API two days ago. We build and run AI phone agents in production, so we put it on a real phone number and called it. A lot.
We tested it with a ~13,000-token warm-lead qualification script from one of our insurance customers, with mandated verbatim lines, strict read-back rules, and branching follow-ups. Over one long day, we ran a dozen real phone calls, about 25 simulated interviews, and a batch of unstructured tests.
These are our first impressions, with transcripts and audio. A deeper write-up is coming shortly.
Transcripts are from our test calls, as transcribed by GPT-Live. We've changed names and identifying wording, trimmed exchanges, and merged split fragments. The timestamps, in seconds from the start of the call, are from the originals.
The naturalness is exceptional
GPT-Live-1 is the most natural-sounding conversationalist we have put on a phone line. It is a real step forward.
It listens and speaks on one continuous audio stream, without separate transcription and synthesis stages. Most of the conversational mechanics just work. Callers heard its first audio about 1.3 seconds after they stopped talking (median), or around 0.7 seconds without the phone leg. We added no voice-activity detection or turn-taking logic. It handles interruptions gracefully, produces natural backchannels ("Mm-hmm"), and speaks at a brisk, human-like pace.
Given goals rather than lines to read, it fits questions to the conversation and takes corrections in stride:
It never lost composure across corrections, interruptions, or callers changing their answers mid-call. If naturalness were the whole job, this post would end here. Unfortunately, there's more to it than that.
It doesn't reliably follow instructions
GPT-Live-1 does not reliably follow instructions on the behaviors that matter for a scripted phone call.
More than once, it responded to a clear "yes" as though the answer had been "no," re-asking the question or moving on as if the caller had declined.
The most consistent failure was taking instructions too literally. Our prompt said: if the caller says the housing status on file is wrong, ask whether they own, rent, or live with their parents. A caller said "no, I own it now," which already answers that question. The model read out the menu anyway:
Some early weirdness was our prompt's fault, but the literalism held across every prompt variant we tried. Any answer we hadn't spelled out was handled mechanically instead of sensibly.
It also sometimes thinks out loud under pressure:
Numbers and alphanumerics are risky
Spellings, addresses, dates of birth, and ID numbers have to be exact. This is where GPT-Live-1's speech has the most issues.
We heard it drop and substitute characters in alphanumeric strings. Here it reads a made-up claim number; the clips are untouched apart from trimming silence. In English, its transcript was correct but the audio added a "Y":
In Russian it was worse: the "Q" became an "X" in both the audio and its transcript.
Dates caused the same problem. A caller gave a birth date as digits, which the model later read back as a number:
It also occasionally speaks a line that belongs to the caller:
Accents in other languages
We test new voice models across the 47 languages our product supports. GPT-Live-1's Russian was fluent but had a thick American accent (whereas the TTS voices we run in production typically sound native):
We also tried Luganda. On the first call, it answered in Swahili instead. On the second, it spoke Luganda, more naturally than we expected from a general model, but again with a thick American accent. We haven't tested every language, but we expect the accent pattern probably generalizes. That doesn't matter for an English-only agent, but is a pretty significant caveat for use cases in other languages.
A few API constraints worth knowing
Based on what we found this week, a few API constraints matter for a real agent:
- Instructions are immutable after the session starts, and capped at 16,384 tokens.
- The output audio never stops streaming — silence is streamed too — so "audio stopped" can't be used as a turn-end signal for use cases that need to segment turns.
- It's reachable only through the new
v1/live/sessionsendpoint, and it wasn't yet available on Azure when we checked.
Our impression so far
Production phone agents have to sound natural and consistently follow scripts. GPT-Live-1 is amazing at the former, but has some serious issues with the latter. We'll be running more tests as time goes on and will continue to report our findings.