Experiments (A/B testing)
Run two agent configurations side by side on real calls and let the success rates pick the winner.
A/B testing answers "is the new prompt actually better?" with data instead of vibes: incoming calls are split between agent variants, each call is graded as usual, and you compare success rates per variant.
There are two surfaces:
- Per agent — tests are configured and tuned in each agent's
builder, on its A/B tab (
/dashboard/agents/<id>?tab=variants). That's where variants are created and the test is switched on. - Cross-agent overview — the Experiments page
(
/dashboard/experiments, under Quality & Testing) aggregates every agent's test into one table.
Set up a test
From the Experiments page, click Set up an A/B test and pick an agent — you land on that agent's A/B tab, where you create the variant configurations (one is marked control) and enable split testing. See Build an agent for the builder itself.
Read the overview
For each agent with a test, the Experiments table shows:
- Status — Active (split testing on) or Paused.
- Variants — every variant as a chip, with the control labeled.
- Leading — the variant with the best success rate so far, with its percentage. Until variants have graded calls, this reads Not enough data yet — don't crown a winner off a handful of calls.
- Calls — total calls across the test's variants.
Agents without a test are listed underneath as one-click nudges to start one. Clicking any row jumps to that agent's A/B tab to tune or conclude the test.