ThunderPhone 2.0 is live.Self-serve, from 2¢/min.Read the announcement
Get started

Facts verified against published sources on .

OpenAI Realtime API clients

ThunderPhone's realtime WebSocket speaks the OpenAI Realtime API protocol. A client written for that protocol, whether an official OpenAI Realtime SDK, a framework adapter or a hand-rolled WebSocket loop, can connect to ThunderPhone by changing the endpoint and the key. The Pipecat, LiveKit Agents and TEN Framework plugins are all built this way, as thin layers over each framework's existing Realtime client.

What is on the other end of the socket is a ThunderPhone call: speech recognition, the language model, the voice, turn-taking, 47 languages and tool execution run on ThunderPhone, the session appears in call history with a recording, transcript and grade, and it is billed at the product's per-minute rate. This page describes what carries over from the protocol unchanged, what ThunderPhone adds, and where its semantics differ.

Connecting

The endpoint is wss://api.thunderphone.com/v1/realtime. Authenticate with a secret API key (sk_live_...) in an Authorization: Bearer header, or as an api_key query parameter for clients that cannot set headers. Wire audio is chosen before the session starts with query parameters: input_audio_format and output_audio_format accept PCM (pcm16, 16 or 24 kHz via input_rate and output_rate) or G.711 μ-law and A-law at 8 kHz, which is what a telephony bridge usually wants. Audio is mono in both directions.

Two session types exist:

  • Inline session. Send session.update with instructions, optional tools, a voice and the engine. The first update starts the call, so send everything at once; the same fields are then fixed for the rest of the call.
  • Saved agent. Add agent_id to the query. The call starts as soon as the socket opens, with the prompt, voice, engine, languages, tools and greeting configured on ThunderPhone, and no session.update is required.

from_number and to_number query parameters record the real parties on the call when the client fronts a phone line.

What carries over from the protocol

The client events a Realtime client already sends work as expected: session.update, input_audio_buffer.append (base64, or raw binary frames as a ThunderPhone extension), conversation.item.create for caller text, silent guidance and function_call_output, response.create, response.cancel, and item truncate and delete. The server side emits the events those clients expect: session created and updated, speech started and stopped, caller transcripts (delta and completed), assistant audio and transcript deltas, output items, function-call arguments, response.done and rate limits. Client-supplied item ids are echoed back, and a response.create carrying metadata gets that metadata back on the response.created it opens, so SDKs that match replies to requests keep working.

Function calling follows the standard shape: the agent emits a function call with a call_id (at most 32 characters, within the protocol's limit), the client answers with a function_call_output item, and the agent continues the turn with the result.

What ThunderPhone adds

With call_events enabled (a session config flag, or call_events=1 in the query for a saved agent), the server also sends call.* events: call.ended with the hang-up reason, transfer, keypad and ignored-speech events. The session object carries a call_id, which the REST API accepts for fetching the recording and transcript afterwards. Live caller transcript fragments can be turned on for clients that want words as they are spoken.

Where the semantics differ

  • A socket is a call. When the agent hangs up, or the caller does, the server sends call.ended (if enabled) and closes the socket normally. A client that treats every close as a connection failure will try to reconnect, and a reconnect starts a new, separately billed call. The plugins handle this; a custom client should too.
  • Turn detection is server-side and always on. Client-side voice activity detection and manual commits are accepted but do not change who takes the turn.
  • Instructions, tools and voice are frozen once the call starts. Later session.update events that try to change them are answered with a non-fatal error event, and the call continues.
  • Injected assistant messages are not accepted. Put the greeting in the prompt or on the saved agent and trigger it with response.create.
  • Audio deltas use the current event names (response.output_audio.delta and friends). Clients still on the earlier beta names need the mapping the plugins already include.

The complete event reference, including a minimal Python client, is in the Realtime API documentation.

Related workflows and terms