Split Testing
A/B test agent variants on live traffic and compare per-variant outcomes.
Split testing routes an agent's live calls across variants —
weighted configurations that override specific fields (prompt, voice,
product, …) while inheriting everything else from the agent's deployed
config. Each call records which variant served it
(agent_variant_id / agent_variant_label on the
Call object), and the stats
endpoint compares outcomes per variant.
The control variant (is_control: true) represents the agent's
deployed config unchanged. Enabling split testing auto-creates a
control if none exists.
Endpoints
| Method | Path | Description |
|---|---|---|
GET | /v1/agents/{agent_id}/variants | List variants |
POST | /v1/agents/{agent_id}/variants | Create a variant |
PATCH | /v1/agents/{agent_id}/variants/{variant_id} | Update a variant |
DELETE | /v1/agents/{agent_id}/variants/{variant_id} | Delete a variant |
POST | /v1/agents/{agent_id}/split-testing | Enable / disable split testing (staged as a draft) |
GET | /v1/agents/{agent_id}/variant-stats | Per-variant call stats |
Variant object
{
"id": 41,
"label": "Warmer greeting",
"weight": 50,
"is_control": false,
"prompt": "You are a warm, upbeat support agent…",
"voice": null,
"product": null,
"background_track": null,
"thinking_level": null,
"acknowledgement_prompt_mode": null,
"acknowledgement_prompt": null,
"inbound_speak_order": null,
"outbound_speak_order": null,
"silence_interval_seconds": null,
"silence_max_checkins": null,
"created_at": "2026-04-20T18:24:10.113Z",
"updated_at": "2026-04-20T18:24:10.113Z"
}| Field | Type | Description |
|---|---|---|
id | integer | Variant id |
label | string | ≤ 100 chars |
weight | integer | 0–100, default 50. Traffic is distributed proportionally to weights |
is_control | boolean | At most one control per agent |
prompt, voice, product, background_track, thinking_level, acknowledgement_prompt_mode, acknowledgement_prompt, inbound_speak_order, outbound_speak_order, silence_interval_seconds, silence_max_checkins | field type | null | Overrides. null = inherit the agent's deployed value; set a value to change just that field for this variant |
POST requires label (everything else optional) and returns 201.
PATCH accepts any subset; DELETE returns 204.
Enable / disable split testing
curl -X POST https://api.thunderphone.com/v1/agents/12/split-testing \
-H "Authorization: Bearer sk_live_YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"enabled": true}'| Field | Type | Default | Description |
|---|---|---|---|
enabled | boolean | true |
{
"split_testing_enabled": false,
"draft_split_testing_enabled": true
}The toggle is staged as a draft on the agent — live traffic starts
(or stops) splitting only after you
deploy. Enabling with no
control variant auto-creates one (label: "Control", weight: 100).
Variant stats
curl https://api.thunderphone.com/v1/agents/12/variant-stats \
-H "Authorization: Bearer sk_live_YOUR_API_KEY"[
{
"variant_id": 40,
"label": "Control",
"is_control": true,
"calls": 132,
"success_rate": 84.1,
"avg_duration_seconds": 163
},
{
"variant_id": 41,
"label": "Warmer greeting",
"is_control": false,
"calls": 129,
"success_rate": 88.4,
"avg_duration_seconds": 171
}
]| Field | Type | Description |
|---|---|---|
calls | integer | Total calls served by the variant |
success_rate | number | null | % of graded, completed calls with call_outcome="success" (ungraded calls excluded); null when nothing graded yet |
avg_duration_seconds | integer | null | Mean duration of completed calls |