---
title: "Realtime WebSocket"
description: "Exchange live call audio, transcripts, and call-control events over one WebSocket."
---

The Realtime WebSocket API provides bidirectional audio over a single
connection. It uses a focused realtime event subset for connecting
ThunderPhone agents to your telephony stack.

For a setup walkthrough and test checklist, see [Use ThunderPhone with OpenAI
Realtime clients](/guides/use-with-openai-realtime).

One WebSocket connection represents one call. Realtime calls appear in
[call history](/api-reference/calls) and are billed like any other call at
your product's per-minute rate. A session whose client never joins is not billed.

## Create a managed realtime session

`POST /v1/realtime/sessions` creates a LiveKit room and returns a scoped
participant token. Authenticate with a secret API key and provide either a
saved `agent_id` or an inline `config`, but not both.

```bash
curl -X POST https://api.thunderphone.com/v1/realtime/sessions \
  -H "Authorization: Bearer sk_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"agent_id": 12}'
```

Inline configuration accepts only `prompt`, `voice`, `product`, `tools`,
`primary_language`, `additional_languages`, `background_track`, `telephony`,
`placeholders`, `outbound`, `ivr_mode`, `call_events`, `transfer_destinations` (see
[Call capabilities](#call-capabilities)), and `max_hold_seconds` (optional integer,
10–900 seconds; `null` uses the default). `prompt` is required.
Unknown keys are rejected with a validation error rather than silently
ignored.

```json
{
  "config": {
    "prompt": "Listen carefully and help complete the call.",
    "voice": "olivia",
    "product": "spark"
  }
}
```

The operation returns `201 Created` with the room credentials:

```json
{
  "call_id": 123456789,
  "room_name": "realtime-123456789",
  "livekit_url": "wss://your-livekit-host.example.com",
  "token": "<scoped-participant-token>"
}
```

A realtime session is a call and counts toward your organization's
[concurrent-call limit](/api-reference/introduction#call-limits). At the
limit, this request returns `429 Too Many Requests` with code
`concurrent_call_limit_reached` and a `Retry-After` header. Over the
WebSocket, the same refusal arrives as an `error` event.

## Endpoint

| Protocol | URL | Description |
|----------|-----|-------------|
| WebSocket | `wss://api.thunderphone.com/v1/realtime` | Start one realtime call |

<Note>
  The realtime endpoint shares its host with the REST API — the same
  `api.thunderphone.com` base URL and API key work for both.
</Note>

## Connect and authenticate

Authenticate the WebSocket upgrade with your secret API key:

```http
Authorization: Bearer sk_live_YOUR_API_KEY
```

Browser clients that cannot set an upgrade header can pass the key in the
query string instead:

```text
wss://api.thunderphone.com/v1/realtime?api_key=sk_live_YOUR_API_KEY
```

<Warning>
  A query-string key can appear in browser history and network logs. Use the
  `Authorization` header whenever your WebSocket client supports it, and
  never expose a secret API key in public client code.
</Warning>

After the connection opens, the server sends `session.created`. Configure
the call with `session.update`; the server confirms the accepted settings
with `session.updated`.

## Configure the session

Send the GA nested session shape before streaming audio. The first
`session.update` starts the call. When `instructions` is omitted or empty,
ThunderPhone uses `You are a helpful voice assistant.`

```json
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "instructions": "Listen to the caller and help them complete the call.",
    "audio": {
      "input": {
        "format": { "type": "audio/pcm", "rate": 16000 }
      },
      "output": {
        "format": { "type": "audio/pcm", "rate": 16000 },
        "voice": "olivia"
      }
    }
  }
}
```

### Session fields

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `session.type` | string | no | Use `"realtime"` |
| `session.instructions` | string | yes | Instructions that define the agent's behavior for this call |
| `session.audio.input.format.type` | string | no | `"audio/pcm"` (default), `"audio/pcmu"`, or `"audio/pcma"`; beta names `"pcm16"`, `"g711_ulaw"`, `"g711_alaw"` are also accepted |
| `session.audio.input.format.rate` | integer | no | PCM input rate: `24000` (default) or `16000` Hz; G.711 is always `8000`. Send the rate explicitly if your client is not at 24 kHz — a mismatch plays audio at the wrong speed rather than erroring |
| `session.audio.input.noise_reduction` | object | no | `{"type":"near_field"}` selects noise cancellation tuned for wideband or browser-style audio; `{"type":"far_field"}` or omission uses the default telephony tuning |
| `session.audio.output.format.type` | string | no | `"audio/pcm"` (default), `"audio/pcmu"`, or `"audio/pcma"`; beta names `"pcm16"`, `"g711_ulaw"`, `"g711_alaw"` are also accepted |
| `session.audio.output.format.rate` | integer | no | PCM output rate: `24000` (default) or `16000` Hz; G.711 is always `8000`. Send the rate explicitly if your client is not at 24 kHz — a mismatch plays audio at the wrong speed rather than erroring |
| `session.audio.output.voice` | string | no | ThunderPhone voice name from the [voice gallery](/api-reference/voices); omitted (or an OpenAI voice name) falls back to the default voice. Entitled organizations can pass `custom:<public_id>` for a ready [custom voice](/api-reference/custom-voices#use-a-custom-voice), or `cartesia:<voice_id>` for a voice in their own Cartesia account once ThunderPhone has [configured it](/api-reference/custom-voices#bring-your-own-cartesia-voice) |
| `session.binary_output` | boolean | no | When `true`, return audio deltas as raw binary WebSocket frames instead of base64 JSON events |
| `session.live_transcripts` | boolean | no | When `true`, stream caller transcript deltas while the caller is still speaking, ahead of the settled transcript — see [Live transcripts](#live-transcripts). Also accepted as `session.config.live_transcripts`. Can be sent in any `session.update`, including after the session starts. **{/* rate:live-table */}+1.5¢/min{/* /rate */}** on the whole session once enabled |
| `session.config.tools` | array | no | Inline function definitions delivered back to this client when the agent calls them |
| `session.config.product` | string | no | Product tier for the call, such as `storm-base` (the default) |
| `session.config.additional_audio_context` | boolean \| null | no | Include the last few turns of caller audio rather than only the most recent turn, improving corrections and spelling/number-heavy data collection at a small latency/cost overhead. Defaults on for browser/WebSocket sessions; `null` keeps the default |

`session.config` accepts only `voice`, `product`, `tools`,
`primary_language`, `additional_languages`, `background_track`, `telephony`,
`placeholders`, `outbound`, `ivr_mode`, `call_events`, `additional_audio_context`,
`transfer_destinations`, and `max_hold_seconds` (optional integer, 10–900 seconds; `null` uses the default). The call prompt comes from
`session.instructions`; there is no separate `prompt` key over the WebSocket. Unknown keys are rejected with a
validation error rather than silently ignored. Set `telephony` to `false` for
the same wideband or browser-style input tuning selected by
`session.audio.input.noise_reduction.type` set to `near_field`.

All audio is mono. PCM audio is signed 16-bit little-endian at 16000 or
24000 Hz; G.711 (`audio/pcmu` μ-law, `audio/pcma` A-law) is 8000 Hz, matching
telephony trunks — no client-side transcoding needed. Input and output formats
are configured independently, but neither can change after the call starts.
After the call starts, agent configuration is immutable: `instructions`,
`voice`, `tools`, `product`, and other `session.config` fields cannot change.
Bridge-local transport options such as `binary_output` remain changeable.

For compatibility, flat `instructions`, `voice`, `input_audio_format`, and
`output_audio_format` fields are also accepted on `session.update`. New
integrations should use the nested shape above.

Inline sessions can define custom functions under `session.config.tools`. The
function schema is available to the agent; your WebSocket client remains
responsible for executing the function and returning its output.

```json
{
  "type": "session.update",
  "session": {
    "instructions": "Help callers check an order.",
    "config": {
      "tools": [
        {
          "type": "function",
          "function": {
            "name": "lookup_order",
            "description": "Look up an order by id.",
            "parameters": {
              "type": "object",
              "properties": { "order_id": { "type": "string" } },
              "required": ["order_id"]
            }
          }
        }
      ]
    }
  }
}
```

Custom tools on saved-agent sessions continue to use the routing configured on
the saved agent; they are not delivered to the WebSocket client.

### Inline sessions are client-steered

With inline configuration, ThunderPhone adds no automatic greeting or spoken
silence check-ins, and by default nothing is spoken while a response is being prepared.
If you want progress speech during slow responses — for example while the
agent waits on one of your tool results — set `placeholders` in the session
config to a list of short phrases in your own words (e.g.
`{"placeholders": ["One moment.", "Still checking."]}`). They are spoken in
the session voice, cycling only when a response takes long enough to need
them.
If the caller goes quiet, the platform's silent-call limit ends the session
after 600 seconds by default. Silence-behavior config keys are not accepted and
are rejected with a validation error.

To have the agent open the call, configure the session, append a system or user
message, and explicitly request one response:

```json
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "instructions": "Help the caller."
  }
}
```

```json
{
  "type": "conversation.item.create",
  "item": {
    "type": "message",
    "role": "system",
    "content": [
      { "type": "input_text", "text": "Greet the caller and offer help." }
    ]
  }
}
```

```json
{ "type": "response.create" }
```

`session.created` is sent as soon as the WebSocket connects. `session.updated`
is sent only after the configuration has been accepted and the session fully
provisioned, so it is the signal that the call is live; a rejected
configuration produces an `error` event instead, and the session stays usable
for a corrected retry. You do not need to wait, though: message items and
`response.create` sent early are buffered and delivered in order once the
agent is ready, so the opening sequence above is safe to send immediately
after `session.update`.

Sessions created with `agent_id` use the deployed prompt, voice, product,
languages, knowledge, and compatible server-run tools. The greeting remains
available, but Realtime disables saved-agent silence check-ins and does not
offer `transfer_call` or `send_keypad_input`.

### Use a saved agent

To use an existing [agent](/api-reference/agents), connect with its id:

```text
wss://api.thunderphone.com/v1/realtime?agent_id=12
```

The saved agent starts immediately, so a `session.update` with instructions
is not required. Set its wire audio before minting with the optional
`input_audio_format`, `output_audio_format`, `input_rate`, and `output_rate`
query parameters. Formats accept `audio/pcmu`, `audio/pcma`, `g711_ulaw`,
`g711_alaw`, or `pcm16`; PCM rates accept `16000` or `24000` (G.711 is always
8000 Hz). For example:

```text
wss://api.thunderphone.com/v1/realtime?agent_id=12&input_audio_format=g711_ulaw&output_audio_format=pcm16&output_rate=24000
```

Add `call_events=1` to the query to receive [`call.*` events](#call-events)
for a saved agent. Inline sessions opt in through `session.config.call_events`
instead, though the query flag works for them too.

Prompt, voice, product, languages, knowledge, and compatible tools come from
the deployed saved agent. Realtime omits `transfer_call` and
`send_keypad_input` and disables spoken silence check-ins.

## Client events

Send client events as JSON text frames unless otherwise noted.

| Event | Fields | Behavior |
|-------|--------|----------|
| `session.update` | `session` | Configures the session using the nested shape above; flat compatibility fields are also accepted |
| `input_audio_buffer.append` | `audio` | Appends a base64-encoded mono pcm16 audio chunk |
| Raw binary frame | binary pcm16 | Appends audio without base64 encoding; this is a ThunderPhone extension |
| `input_audio_buffer.commit` | — | Acknowledged with `input_audio_buffer.committed`; turn detection remains automatic |
| `input_audio_buffer.clear` | — | Acknowledged with `input_audio_buffer.cleared`; turn detection remains automatic |
| `conversation.item.create` | `item` | Sends a caller utterance (`user`) or silent guidance (`system`/`developer`), or acknowledges a function call; message items are acknowledged with `conversation.item.added` then `conversation.item.done` using your item id |
| `conversation.item.truncate` | `item_id`, `content_index`, `audio_end_ms` | Acknowledged with `conversation.item.truncated` |
| `conversation.item.delete` | `item_id` | Acknowledged with `conversation.item.deleted` |
| `response.cancel` | `response_id` optional | Stops the in-progress response's remaining audio and closes it with status `cancelled`; errors with `response_cancel_not_active` when nothing is streaming |
| `response.create` | — | Triggers exactly one model response using the conversation so far; if a response is active, emits an `error` with code `conversation_already_has_active_response` and does not start another |

### Append audio

```json
{
  "type": "input_audio_buffer.append",
  "audio": "<base64-pcm16>"
}
```

Send audio at its natural pace rather than uploading an entire recording at
once. You can send each chunk as a raw binary frame instead; no session flag
is required for binary input.

### Send text guidance

Message items are appended to the conversation and never trigger a response by
themselves. Use `user` for a completed caller utterance and `system`, with
`developer` accepted as an alias, for silent guidance. A response begins either
from server turn detection on audio or from an explicit `response.create`.
Message items are acknowledged with `conversation.item.added` followed by
`conversation.item.done`; both echo the client-provided `item.id`, or a
generated id when it is absent. Text is not converted into caller audio.

```json
{
  "type": "conversation.item.create",
  "item": {
    "type": "message",
    "role": "user",
    "content": [
      { "type": "input_text", "text": "Ask whether they need anything else." }
    ]
  }
}
```

For silent guidance, set `"role": "system"` in the same message shape.
Assistant-role message injection is unsupported.

### Request a response

`response.create` may carry `response.metadata` (an object of your own keys);
the server echoes it on the `response.created` that answers the request, as
OpenAI does, so SDKs can match the reply they asked for.

Send `response.create` to trigger exactly one response using the conversation
so far. If a response is already in progress, the server emits an `error` event
with code `conversation_already_has_active_response` and does not start another.

After handling a custom function call, the client must send a
`function_call_output` item with the received `call_id`. The `output` can be a
JSON string or object. The default deadline is 30 seconds. If the deadline
expires, the conversation continues with an error result so the agent can
recover. Every function call names one of your declared tools and expects a
`function_call_output`; platform actions arrive as `call.*` events instead
(see below) and take no output. A `call_id` is at most 32 characters, OpenAI's
limit, so SDKs that enforce it pass it through unchanged. An accepted output is
acknowledged with
`conversation.item.added` and `conversation.item.done` carrying the item (and
your `id`, if you set one), as OpenAI does.

```json
{
  "type": "conversation.item.create",
  "item": {
    "type": "function_call_output",
    "call_id": "call_abc123",
    "output": "{\"status\":\"ok\"}"
  }
}
```

## Server events

Server events are JSON text frames. Each JSON event includes a unique
`event_id`.

| Event | Important fields | Description |
|-------|------------------|-------------|
| `session.created` | `session` | Sent when the WebSocket opens with the session id and defaults |
| `session.updated` | `session` | Confirms the current session configuration |
| `conversation.item.added` | `item` | A conversation item (user or assistant message, or function call) was added |
| `conversation.item.done` | `item` | A conversation item reached its final state |
| `conversation.item.input_audio_transcription.delta` | `item_id`, `delta` | Caller transcript text. By default sent once per turn with `delta` carrying the whole transcript; with `live_transcripts` on, a stream of word fragments while the caller speaks — see [Caller transcripts](#caller-transcripts) |
| `conversation.item.input_audio_transcription.completed` | `item_id`, `transcript`, `audio` | The settled caller transcript, plus a signed link to the caller's audio |
| `conversation.item.truncated` | `item_id`, `audio_end_ms` | Acknowledges `conversation.item.truncate` |
| `conversation.item.deleted` | `item_id` | Acknowledges `conversation.item.delete` |
| `response.created` | `response` | Starts an agent response |
| `rate_limits.updated` | `rate_limits` | Emitted after each `response.created` |
| `response.output_item.added` | `response_id`, `item` | Adds an audio message or function call to the response |
| `response.content_part.added` | `response_id`, `item_id`, `part` | Opens the audio content part of a message item |
| `response.output_audio.delta` | `response_id`, `item_id`, `delta` | Base64-encoded output audio; replaced by raw binary frames when binary output is enabled |
| `response.output_audio_transcript.delta` | `response_id`, `item_id`, `delta` | Agent transcript segment |
| `response.output_audio.done` | `response_id`, `item_id` | Output audio for the item is complete |
| `response.output_audio_transcript.done` | `response_id`, `item_id`, `transcript` | Final agent transcript for the response |
| `response.content_part.done` | `response_id`, `item_id`, `part` | Closes the audio content part with the final transcript |
| `response.function_call_arguments.delta` | `call_id`, `delta` | Incremental JSON arguments for a function call |
| `response.function_call_arguments.done` | `call_id`, `arguments` | Complete JSON arguments for a function call |
| `response.output_item.done` | `response_id`, `item` | Completes an audio message or function-call item |
| `response.done` | `response` | Completes the response |
| `input_audio_buffer.speech_started` | `audio_start_ms`, `item_id` | The server detected the start of caller speech; `item_id` is a stable advisory id for the speech interval and may not match the later history item id |
| `input_audio_buffer.speech_stopped` | `audio_end_ms`, `item_id` | The server detected the end of caller speech; `item_id` is a stable advisory id for the speech interval and may not match the later history item id |
| `input_audio_buffer.committed` | `item_id`, `previous_item_id` | The caller utterance was committed (also acknowledges a manual `commit`) |
| `input_audio_buffer.cleared` | — | Acknowledges `input_audio_buffer.clear` |
| `error` | `error.type`, `error.code`, `error.message` | Reports an authentication, validation, or server error |

## Caller transcripts

A caller's transcript improves as the turn proceeds — a later pass can
rewrite words an earlier one produced. ThunderPhone therefore does not
stream partial caller text by default (opt in with
[`live_transcripts`](#live-transcripts)). Each caller turn is announced
**once**, when its transcript has settled:

```text
input_audio_buffer.committed
conversation.item.added
conversation.item.input_audio_transcription.delta
conversation.item.input_audio_transcription.completed
conversation.item.done
```

Two consequences worth designing around:

- **The single `delta` carries the entire transcript**, identical to the
  `completed` transcript. Concatenating every delta for an item yields
  exactly the final text.
- **The turn is announced once the agent has acted on it** — begun replying,
  pressed keypad digits, or called a tool — because that is the earliest
  point the text is settled. Use `previous_item_id` to reconstruct
  conversation order rather than arrival order.

### Transcript finality

The text in `completed` matches the durable per-call record (the same text
the dashboard and `GET /v1/calls/{call_id}/history` show) for every caller
turn that finished before the call ended. When the call ends normally — the
agent hangs up, or the far end does — the session reconciles against the
final record before `call.ended`, so a turn is never left unannounced.

Two cases fall outside that. A turn still in progress when the call ends has
no settled text yet: its words arrive as live deltas (when enabled) and its
final text exists only in the durable record. And if the call is torn down
abnormally — the media session drops rather than the agent or the far end
hanging up — there is no final record to reconcile against, so the last
turns are released with the most recent text the session had, which may be
superseded in the durable record.

In both cases the durable record is authoritative and reachable with the
`call_id` on `session.updated`. A client that must not miss a correction
should reconcile against `GET /v1/calls/{call_id}/history` after the call;
one that treats the WebSocket as the record of truth will be right on every
normally-ended call.

The platform `call_id` for fetching that record appears on
`session.updated` (`session.call_id`) and on `call.ended`.

### Live transcripts

Set `session.live_transcripts: true` to also receive caller words **while the
caller is speaking**. A separate streaming transcription runs on the caller
audio and its output is put on the wire immediately; the settled transcript
described above still arrives and is still the text of record. Per caller
turn the sequence becomes:

```text
input_audio_buffer.committed
conversation.item.added                                  (status: in_progress)
conversation.item.input_audio_transcription.delta        (repeated, word fragments)
conversation.item.input_audio_transcription.completed    (the settled transcript)
conversation.item.done
```

Rules for a client:

- **Deltas are genuine fragments.** Concatenate the deltas of an item to
  render the live caption. They begin roughly a second behind the caller's
  voice, before `input_audio_buffer.speech_started` in some cases, and
  before the agent has replied.
- **`completed` replaces the concatenated deltas, and it is terminal.** It
  carries the settled transcript, which is generally better than the live
  text and can differ from it. Show that text and discard the fragments. An
  item receives exactly one `completed`, and never a `delta` after it;
  `conversation.item.done` closes the item.
- **The item is opened before its place in the conversation is certain.**
  Words are streamed as soon as they are heard, so at a turn boundary (a
  keypad press, the caller resuming right as the agent replies) the last
  words of one turn can briefly show on the previous item until that item's
  `completed` replaces its text. Anchor each item to its first delta's
  timestamp; the settled text always lands on the right item.
- Items are always closed. If a call ends before a live item's transcript
  settles, its `completed` carries the live text.

Live transcripts are a WebSocket-only addition billed at **{/* rate:live-prose */}+1.5¢{/* /rate */} per
minute** on top of the session's tier rate (see [Pricing](/guides/pricing)).
They are off by default. Turn them on with `session.live_transcripts: true`
in any `session.update` — the first one for inline sessions, or a later one
for saved-agent sessions, which start at connect:

```json
{ "type": "session.update", "session": { "live_transcripts": true } }
```

Once enabled, the surcharge applies to the whole session; a later
`session.update` with `live_transcripts: false` stops the stream but does not
remove the charge. If the server cannot provide live transcripts at any
point, the session continues with settled transcripts only and a non-fatal
`error` event says so.

### One `completed` per item

Each caller item receives exactly one
`conversation.item.input_audio_transcription.completed`, and never a
`delta` after it. Persist by `item_id` and treat `completed` as the final
text. In the rare case a better transcript is produced after an item was
announced, it is kept in the call's history and dashboard but is not sent
again over the WebSocket.

### Caller audio

`conversation.item.input_audio_transcription.completed` carries an `audio`
object with a pre-signed link to what the caller said for that turn:

```json
{
  "url": "https://...",
  "expires_at": "2026-08-12T19:04:00Z",
  "format": "audio/wav"
}
```

The turn's duration is already on the same event as `usage.seconds`; it is
not repeated here.

The recording is WAV, 16-bit PCM, mono, 16 kHz, and covers the whole turn.
It is the audio as received on the call, not a noise-suppressed version.
Issue a plain `GET` — the URL is already signed, so do not send an
`Authorization` header. Links expire 24 hours after they are issued and are
not reissued, so download and store the bytes rather than the URL.

The `audio` object arrives once, on the item's single `completed`, so store
it on sight.

The `audio` field is **omitted** — not null — whenever no recording of the
turn exists, including when call recording is disabled for your
organization. Treat it as optional on every item. Whether a turn has a
recording is decided once, so an item that arrives without `audio` will not
gain one later.

It rides the transcription event rather than the item's content part
because the GA schema types a content part's `audio` as a base64 string, so
an object there would fail strict validation in the official SDK. It is an
extra top-level key, which GA models accept because they permit unknown
fields.

Note that ThunderPhone's [call events](#call-events) are a different kind of
addition: they are new event *types*, not extra keys, so a validator that
rejects unknown event types will not recognise them. Route on `type` and
ignore what you do not handle.

### Conformance notes

Item ids are opaque and carry no structure — do not parse them or derive
ordering from them. Use `previous_item_id` for conversation order and
`item_id` for identity.

Caller items are announced once their transcript settles, which is after the
agent has begun replying, so arrival order is not conversation order (with
`live_transcripts` on they open as soon as words are heard instead).
`previous_item_id` always reflects conversation order: an agent item names
the caller turn it answers, even though that turn is announced moments
later.

One consequence to build for: **`previous_item_id` can name an item you have
not received yet.** It arrives within the next few events. Accumulate items
and resolve the order afterwards rather than assuming a predecessor is
already present when an item arrives.

Usage counters on `response.done` are always zero. Calls are metered per
minute rather than per token, so ThunderPhone reports honest zeros instead
of fabricated token counts. The `usage` on transcription events is real and
reports the caller audio duration.

ThunderPhone supports PCM at 16000 Hz as an extension. The OpenAI GA schema
pins `audio/pcm` to 24000 Hz, so strict validators of `session.created` and
`session.updated` should also allow rate 16000.

The normal audio response sequence is:

```text
response.created
rate_limits.updated
response.output_item.added
conversation.item.added
response.content_part.added
response.output_audio.delta (repeated)
response.output_audio_transcript.delta (repeated)
response.output_audio.done
response.output_audio_transcript.done
response.content_part.done
response.output_item.done
conversation.item.done
response.done
```

Audio and transcript deltas can be interleaved within the response.

## Function calls

Function calls arrive as `function_call` output items and always name one of
the tools your client declared. Read the function name and `call_id` from
`response.output_item.added`, accumulate argument deltas, and act after
`response.function_call_arguments.done`. Platform actions are never function
calls — they arrive as [call events](#call-events).

### Call capabilities

Every session's agent can end the call and can deliberately stay silent for a
turn while the far side is still talking. Inline sessions with `outbound: true`
can also wait out hold queues. Inline outbound sessions with `call_events`
enabled can press keypad digits, and inline sessions with `call_events` can
request a cold transfer for your client to execute from the `call.transfer`
instruction. Saved-agent sessions omit transfer and keypad input. Capabilities
that instruct your client are granted only when the event channel exists to
carry the instruction.

To limit where an inline session can transfer, send `transfer_destinations`
in the config: up to 50 `{name, number}` entries, validated like an agent's
[transfer numbers](/api-reference/agents#transfer-numbers). The agent then
transfers by name, sees only the names, and cannot transfer anywhere else; a
`call.transfer` event carries both the entry's number and its `destination`
name. A transfer to anything else is refused and the conversation continues.
An empty list removes the transfer capability. Without `transfer_destinations`,
the agent chooses the number itself and `call.transfer` carries only
`phone_number`.

A set of function names is reserved for the platform and rejected in your
`tools` with a validation error: `end_call`, `transfer_call`,
`send_keypad_input`, `no_response`, `wait_on_hold`, `search_knowledge_base`,
`play_sound`, `send_email`, `send_sms`, `calculate`, `release_probe_echo`,
`speak_uninterruptible`.

### Call events

Platform call actions and state changes are delivered as dedicated events in
the `call.*` namespace — a ThunderPhone extension to the realtime event set.
They are an explicit opt-in: set `call_events: true` in the session config to
receive them. They are notifications: nothing is sent in reply, and there is
no `call_id` or output. Handle every type below in your event loop — clients
ported from other realtime APIs often switch on known event types and would
otherwise drop them.

| Event | Payload | Meaning |
|-------|---------|---------|
| `call.keypad` | `{"digits":"2#"}` | Send these keypad digits out-of-band through your telephony stack now |
| `call.transfer` | `{"phone_number":"+15551234567", "destination":"Billing"?}` | Transfer your telephony leg to this number now; a `call.ended` with reason `transfer` follows. `destination` is the entry name, present only when the session has `transfer_destinations` |
| `call.ended` | `{"reason":"agent_hangup" \| "remote_hangup" \| "transfer" \| "error", "call_id":"…"}` | The call is over; the server closes the WebSocket shortly after this event, so drive your teardown from it. `call_id` addresses the durable per-call record (`GET /v1/calls/{call_id}/history`) |
| `call.hold.started` / `call.hold.ended` | — | The agent is waiting out a hold queue / the hold ended and normal conversation resumed |
| `call.speech_ignored` | `{"reason":"no_response" \| "hold" \| "superseded" \| "response_create_dropped", "detail":"…"?}` | The far side said something and the agent deliberately produced no response — it judged the audio as an automated system still talking (`no_response`), hold content while waiting for a human (`hold`), or newer audio made its drafted response stale and it was dropped (`superseded`). `response_create_dropped` means a `response.create` you sent was discarded because the agent was busy; `detail` says why (`assistant_speaking`, `caller_speaking`, `turn_active`, `hold`, `not_ready`, or `queued_then_blocked`) — re-issue the trigger when the blocking state clears (for example after `response.done`) |

```json
{ "type": "call.keypad", "event_id": "event_1a2b", "digits": "2" }
```

```json
{ "type": "call.ended", "event_id": "event_3c4d", "reason": "agent_hangup" }
```

### Outbound calls

Set `outbound: true` in the session config when the session is an outbound
call — the agent dialed out, so the far side may be an automated menu, a hold
queue, voicemail, or a screener rather than a person. This enables the
engine's outbound call handling: the agent recognizes and navigates automated
systems reliably, responds immediately when a human picks up so they are not
met with dead air, and applies cost-saving measures that keep the price down
while the call is working through menus, waiting on hold, or handling
voicemail. It is off by default; sessions without it treat the far side as a
human caller. Menu navigation and hold handling need no further
configuration — steer them through your instructions:

```json
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "instructions": "Call the pharmacy, navigate the phone menu, and ask whether the prescription is ready.",
    "audio": {
      "output": { "voice": "olivia" }
    },
    "config": {
      "outbound": true,
      "call_events": true,
      "tools": [
        {
          "type": "function",
          "function": {
            "name": "report_status",
            "description": "Report the prescription status back to the app.",
            "parameters": {
              "type": "object",
              "properties": { "status": { "type": "string" } },
              "required": ["status"]
            }
          }
        }
      ]
    }
  }
}
```

`ivr_mode` sets how an outbound session handles automated phone menus (IVRs):

| Value | Behavior |
|-------|----------|
| `auto` (default) | The agent paces itself for a phone menu until it hears a person, then talks to that person normally for the rest of the call. The short pickup greeting ("Hello. Can you hear me?") plays only when a person picks up after menu or hold audio, and only if the agent has not spoken since. |
| `on` | The whole call is handled as a phone menu: the agent waits for complete menu prompts before answering, and it plays the pickup greeting as soon as it detects a person. This was the behavior of every outbound session before `ivr_mode` existed. |
| `off` | No phone-menu mode: the agent replies to whoever answers as it would to a person, and the pickup greeting never plays. Hold waiting and, with `call_events`, keypad input stay available. |

`ivr_mode` needs `outbound: true`; setting it to `auto` or `on` without that is
rejected with a validation error.

A standalone function call follows this sequence:

```text
response.created
response.output_item.added
response.function_call_arguments.delta
response.function_call_arguments.done
response.output_item.done
response.done
```

## Using the official OpenAI SDK

Existing OpenAI Realtime integrations work by changing only the WebSocket
base URL and the API key. Pass the base ending at `/v1` — the SDK appends
`/realtime` itself, mirroring `wss://api.openai.com/v1`:

For prerequisites, configuration choices, and an end-to-end test checklist,
see [Use ThunderPhone with OpenAI Realtime
clients](/guides/use-with-openai-realtime).

```python
import os

from openai import AsyncOpenAI

client = AsyncOpenAI(
    api_key=os.environ["THUNDERPHONE_API_KEY"],
    websocket_base_url="wss://api.thunderphone.com/v1",
)

async with client.realtime.connect(model="thunderphone-realtime") as connection:
    await connection.session.update(session={
        "type": "realtime",
        "instructions": "Listen to the caller and help them complete the call.",
    })
    async for event in connection:
        ...
```

The `model` argument is echoed back but does not change behavior; the agent
configuration comes from the session instructions or a saved agent.

## Minimal Python client

Install `websockets`, set `THUNDERPHONE_API_KEY`, and provide an uncompressed
mono pcm16 WAV file sampled at 16000 Hz:

```bash
pip install websockets
export THUNDERPHONE_API_KEY=sk_live_YOUR_API_KEY
python realtime_client.py caller.wav
```

```python
import asyncio
import base64
import json
import os
import sys
import wave

import websockets

URL = "wss://api.thunderphone.com/v1/realtime"
RATE = 16_000
CHUNK_MS = 100


def read_wav(path):
    with wave.open(path, "rb") as wav:
        if (wav.getnchannels(), wav.getsampwidth(), wav.getframerate()) != (1, 2, RATE):
            raise ValueError("WAV must be mono pcm16 at 16000 Hz")
        return wav.readframes(wav.getnframes())


async def send_audio(ws, pcm):
    chunk_size = RATE * 2 * CHUNK_MS // 1000
    loop = asyncio.get_running_loop()
    deadline = loop.time()
    for offset in range(0, len(pcm), chunk_size):
        chunk = pcm[offset : offset + chunk_size]
        await ws.send(json.dumps({
            "type": "input_audio_buffer.append",
            "audio": base64.b64encode(chunk).decode("ascii"),
        }))
        deadline += CHUNK_MS / 1000
        await asyncio.sleep(max(0, deadline - loop.time()))


async def receive(ws):
    async for message in ws:
        if isinstance(message, bytes):
            continue  # output audio, when binary output is enabled
        event = json.loads(message)
        event_type = event["type"]
        if event_type == "conversation.item.input_audio_transcription.completed":
            print("caller:", event.get("transcript", ""))
        elif event_type == "response.output_audio_transcript.delta":
            print(event.get("delta", ""), end="", flush=True)
        elif event_type == "response.output_audio_transcript.done":
            print()
        elif event_type == "response.output_item.done":
            item = event.get("item", {})
            if item.get("type") == "function_call":
                arguments = json.loads(item.get("arguments") or "{}")
                print("function:", item["name"], arguments)
                # Run your declared tool here and return its output.
        elif event_type == "error":
            print("error:", event["error"], file=sys.stderr)


async def main(path):
    headers = {"Authorization": f"Bearer {os.environ['THUNDERPHONE_API_KEY']}"}
    async with websockets.connect(URL, additional_headers=headers) as ws:
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "type": "realtime",
                "instructions": "Listen carefully and help complete the call.",
                "audio": {
                    "input": {"format": {"type": "audio/pcm", "rate": RATE}},
                    "output": {
                        "format": {"type": "audio/pcm", "rate": RATE},
                        "voice": "olivia",
                    },
                },

            },
        }))
        while json.loads(await ws.recv())["type"] != "session.updated":
            pass
        await asyncio.gather(send_audio(ws, read_wav(path)), receive(ws))


asyncio.run(main(sys.argv[1]))
```

## Protocol notes

- Turn detection uses server-side voice activity detection and is always on.
  Clients do not need to commit or clear the input audio buffer.
- Transcript events are emitted as per-utterance segments, not token-by-token
  deltas. Use the `completed` and `done` events as final text.
- Unknown or unsupported client event types produce an `error` event without
  closing the WebSocket. The client can correct the request and continue.