Introducing ThunderPhone v3, featuring Zeus

Phone calls influence over a trillion dollars of consumer spending in the US alone every year, but the existing ways businesses handle phone calls have serious limitations.

Human-powered call centers are very expensive, making many use cases unaffordable and costing businesses billions in operations costs to handle routine matters over the phone.

Human-powered call centers also have significant other problems, including:

• Headcount is fixed at any given time, causing excruciating hold times for customers when call volumes go up.
• Inconsistent quality and adherence to rules, meaning two calls to the same business may get very different experiences.
• High friction and inertia: hiring and training people takes a long time, so changing what a call center does for a given business takes huge amounts of cost and effort.

These advantages have driven adoption of AI-powered calls for years, but the results have typically been unsatisfying for customers. Phone AI systems often mishear callers, answer in unhelpful ways, and conduct conversations in unnatural and unpleasant ways.

These problems were even worse a few years ago, and have been improved upon with the latest generation of generative-AI-powered systems, but there is still a lot of room for improvement because of the limitations of current voice AI stacks.

A large part of the problem has been the convergence of the industry on the “three-step-pipeline”: phone AI that handles calls by turning user audio into text with a transcription model, generating a response to that text with an LLM, and finally turning the LLM’s text response into audio with a TTS (text-to-speech) model.

Unfortunately, this approach has some very big problems, including the two biggest:

1. Transcription models inevitably make some mistakes. Because they take a rich audio signal and compress it into a single best-guess transcript in text form, they strip out all information other than the single best-guess transcript. If the transcript is wrong, typically there’s not enough information for the LLM to guess what the right transcript might have been.
2. LLMs currently suffer from a high-stakes tradeoff between speed and accuracy. Fast LLMs can handle voice conversations snappily, but often make mistakes. Slow LLMs make fewer mistakes, but take many seconds to respond, which is unacceptable for phone calls.

Because of these tradeoffs, it’s a very tricky task to pick the right transcription model and LLM for any given type of call powered by a 3-step pipeline. Some transcription models are faster but make more mistakes, and some struggle with particular languages or short utterances; each flaw matters more on some calls than others. And any 3-step pipeline based stack inevitably has some issues with audio understanding mistakes from the transcription model and instruction-following mistakes and other odd behavior from the LLM - even when the best possible transcription model and LLM for that task are selected.

The three-step pipeline, because of its turn-based nature, also makes it quite hard to make conversations feel natural for the user.

The newer architecture of speech-to-speech models is increasingly popular, but presents different tradeoffs. Speech-to-speech models typically feel much more natural in conversations, and sometimes (though not always) are more accurate at audio understanding than a three-step pipeline. However, speech-to-speech models still suffer tremendously from unreliable performance on instruction following and tool calls, and erratic behavior generally, making them risky for business phone calls.

At ThunderPhone, we’ve taken a different approach entirely, one that offers superior performance to either 3-step pipelines or to speech-to-speech models.

We do this by doing 2 things: running a complex, configurable architecture that uses significantly more than 3 models at once to power any given call, and by optimizing that system dynamically with AI to squeeze the best possible performance out of every call.

First, the architecture.

To begin with, we’ve found that getting AI phone calls to behave reliably takes a lot more models than a typical 3-step pipeline.

To ensure that callers are heard correctly, we typically run multiple, or even many, transcription models at once, and often also pipe audio directly to one or more LLMs - sending it as audio, and not as text, has its own disadvantages but can be a powerful complement to text-based transcripts.

On the LLM side, we also combine multiple models, doing things like running a faster and a slower LLM in the same conversation to combine speed and accuracy, as well as switching which model handles each conversation turn dynamically based on the state of the conversation at that time.

And finally, we also combine multiple models to optimize conversation handling performance for things like noise cancellation and identification, speaker separation, start and end of turn detection, and more - far exceeding the performance of any single model on those tasks as well.

We’re also increasingly fusing speech-to-speech models with other models on the side to combine the naturalness of the speech-to-speech models with the reliability of text-based models.

We also offer an extensive voice library covering 62 languages, and agents switch languages mid-call when the caller does. Platform voices are powered by Inworld and are the least expensive option; premium voices from Cartesia and ElevenLabs v4 Turbo cost a few cents a minute more. Voices trade naturalness against stability: the more natural-sounding ones are more prone to mispronunciations and hallucinations, especially on numbers and letters. Zeus helps you pick the right tradeoff for your calls.

We combine open-source and commercial models and fine-tune them where it helps.

With these approaches, we’ve gotten AI voice agents to make significantly fewer mistakes than any three-step or speech-to-speech system we’ve seen to date.

ThunderPhone already holds the top score on Big Bench Audio, 99.4%, and handles thousands of real customer calls for businesses. More benchmarks are coming, including public versions of our own internal evals.

Moreover, with this complex architecture, we’re able to intelligently trade off between quality and reliability, latency (response speed), and cost of calls, optimizing for the needs of every business use case.

The downside of all of this, of course, is complexity. Optimizing all of these pieces is a lot of work! Until Zeus, we were doing this in two ways:

• Offering 3 pre-baked stacks for self-serve customers - Spark optimized for cost, Bolt for speed, and Storm for intelligence.
• Custom-rolling stacks for enterprise customers.

However, we realized that the pre-baked stacks were rarely the exact right fit for any given use case - just like 3 sizes of off-the-rack suit wouldn’t fit most customers perfectly.

The enterprise stacks we were hand-tuning, on the other hand, were incredibly high-performance for their use cases. The downside was that they took a lot of effort from us to get right, but we eventually realized that we were doing similar optimization processes for every enterprise stack.

We then realized that we could train an AI to do that job for us. Thus, Zeus was born.

With v3, Spark, Bolt and Storm are retired. Every agent now runs on one unified v3 architecture, and Zeus tunes it to your use case.

Zeus is the fully agentic harness for voice AI: an AI agent that builds, tests and tunes your voice agents for you - think of it as Claude Code for your phone calls.

Zeus is able to configure the modular ThunderPhone architecture to your exact use case, maximizing performance of the voice agent for your exact business needs and priorities, whether that’s maximizing conversions, minimizing hangups, reducing the cost of each call, or some combination of all of the above and more.

Zeus does this with a “full closed loop” approach, where it helps you with the full life cycle of a voice agent:

1. Creating it from scratch, by talking to you about your business needs and letting you edit the prompt, and/or by importing call-center recordings, scripts, and SOPs, or by importing existing voice agents from Vapi, Retell, Bland or ElevenLabs in one click, with our AI handling the import end to end.
2. Connecting to VoIP and software integrations through powerful AI that makes it simple to set up even the most complex integrations.
3. Testing the voice agent both manually and through text and audio based simulations that help uncover problems before they crop up on live calls.
4. Automatically identifying problems on live calls and automatically proposing and offering A/B tests of solutions, improving the performance of the calls over time.

You decide how much autonomy Zeus gets: have it ask before every change, or let it fire away.

We’ve been heavily inspired in how we’ve created Zeus by working closely with Claude Code and Codex as software engineers. Unfortunately, tools like Claude Code and Codex are significantly worse at configuring and optimizing voice agents than they are at building standard software, for a few reasons:

1. They were trained heavily on text data, particularly code, but far less on audio data. The audio data they WERE trained on tends to be from places like YouTube and less from live call-center conversations, causing performance gaps when they’re given real call data.
2. The knowledge of how to get AI to perform well on phone calls is only available in piecemeal ways on the Internet. Moreover, there are unfortunately also a lot of false or misleading marketing claims about the performance of voice AI systems on the Internet, and the AI tends to blindly believe those claims and follow that “knowledge” when you ask it to build voice agents - leading to poor performance.
3. Unlike code, which typically has an inherent closed loop because it works or doesn’t work in ways agents can usually identify right on the computer, coding agents do NOT currently have a closed loop for voice agents, because they aren’t able to run real (or even simulated) calls through it out of the box, and they are very bad at spotting problems in real calls. So Claude Code, Codex etc. tend to thrash around and be very bad at resolving problems with AI voice agents and improving their performance.

With Zeus, we’ve overcome these problems to achieve a Claude Code-like experience by giving Zeus access to several important sources of signal:

With this approach, we’ve found Zeus to already be a much easier and more flexible way to get outstanding performance out of phone calls for a variety of use cases.

Among other advantages, allowing Zeus to optimize phone calls also leads to cost efficiency. Zeus takes into account both the least expensive stack necessary to get the performance needed on each use case, and the unit economics and business needs of each use case, meaning it’s no longer necessary to overpay for an overly powerful stack to get simple calls to work well.

Zeus automatically optimizes calls for cost using one of three strategies:

• “Spend whatever it takes to make the calls go well” - whether that’s 2c/min, 8c/min or even 15c/min for complex and difficult use cases. Zeus first estimates your cost per minute and per call from simulated calls and works through it with you until the config makes sense, then dials it in on your first few real calls. You can add a hard cap at any time.
• Hard cap on price per minute, per call, etc.
• Unit-economic optimization, e.g. optimize for maximizing conversion value minus call cost.

ThunderPhone uses cost-plus pricing: you pay our vendors’ prices for what each call uses, plus our hosting cost, plus our margin. Our margin is 17% of what you pay on calls that cost us 1¢ a minute or less, rising gradually to 25% on calls that cost us 15¢ a minute or more. We’re bootstrapped and profitable, not venture funded, so unlike many of our competitors, thin margins are a choice we can make without a VC breathing down our necks.

Our per-minute prices don’t include telephony. We encourage you to bring your own carrier, which adds no ThunderPhone charge. We can also supply US phone numbers for $1/month each, with carrier minutes from Telnyx passed through at list rate (0.35¢ a minute inbound, 0.5¢ outbound) under the same cost-plus terms.

Zeus itself is free for the time being, but will eventually cost money. We intend to eventually release a lightweight Zeus that stays free, usage-based and/or subscription pricing for the full-fledged Zeus, and the option to bring your own OpenAI or Anthropic key. Simulated calls Zeus runs are billed like any other test call.

With the optimizations our system is able to make, as well as vendor price cuts that reach your calls automatically, we believe you’d be hard-pressed to save money on your calls compared to ThunderPhone pricing even if you roll your own stack from scratch.

We’re also able to offer even lower pricing for enterprise use-cases with more extensive optimization, including fine-tuning models for specific high-volume calls - prices can be 1c/min or even lower. We also offer other options to enterprise customers, including custom implementations and integrations, custom compliance and data residency needs, and more. Reach out to alex@thunderphone.com to learn more about enterprise offerings.

ThunderPhone is now usable through a variety of interfaces:

ThunderPhone supports both phone and web/app calls.

On the phone, ThunderPhone handles the telephony details real deployments need: warm, cold and whisper transfers; voicemail and call-screener detection; DTMF; outbound campaigns with deliverability tools like caller ID registration and spam-label monitoring; and much more.

And for web/app use, you can use ThunderPhone’s built-in widget functionality to integrate your voice agents wherever you’d like to use them.

ThunderPhone supports HIPAA and GDPR compliance, and Zeus can help you get started with both.

We believe the future of voice AI looks a lot like Zeus - give it a try and let us know what you think. Zeus is still an early product; if you run into any issues or experiences you’d like to work differently, let us know and we’ll fix them ASAP.