Use ThunderPhone with OpenAI Realtime clients

Point an OpenAI Realtime-compatible server client at ThunderPhone, using either a saved agent or inline session configuration.

ThunderPhone implements a focused subset of the OpenAI Realtime event model. An existing server-side client can keep its WebSocket, audio, session, and response event flow while using a ThunderPhone voice agent.

Before you connect#

You need a ThunderPhone organization secret API key and a server runtime that can open WebSockets. Do not connect from browser code or expose the key to a browser. To use a saved agent, deploy it first and copy its numeric agent ID.

Connect to:

wss://api.thunderphone.com/v1/realtime

Authenticate the WebSocket upgrade from your server:

Authorization: Bearer sk_live_YOUR_API_KEY

Query-string authentication exists for clients that cannot set handshake headers, but URLs are easier to leak into logs.

Choose who owns the configuration#

Saved agent#

Keep what every call shares in a saved agent, where it has versions and test sets, and send what changes per session in the first session.update: session.prompt.id picks the deployed agent, session.instructions carries this session's brief, and session.prompt.variables fills the agent prompt's {{name}} placeholders.

{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "prompt": {
      "id": "12",
      "variables": { "user_name": "Ada Park" }
    },
    "instructions": "Ada is a Costco member in Mountain View. Call the store and ask for today's opening hours.",
    "audio": {
      "input": { "format": { "type": "audio/pcm", "rate": 24000 } },
      "output": { "format": { "type": "audio/pcm", "rate": 24000 } }
    }
  }
}

The instructions become the call's context, added after the agent's prompt for this call only. The session's own options (telephony, outbound, ivr_mode, call_events, placeholders) and the language it starts in (primary_language) go in session.config, as for an inline session. The agent's voice, product, and tools come from its deployed configuration: a voice, another agent setting, or an OpenAI-only field such as a non-default tool_choice in the update produces an error event and is ignored, and tools your client runs are refused (client-executed tools need an inline session). prompt.version is accepted and ignored. See Use a saved agent for every rule.

Connecting with ?agent_id=12 instead starts the agent while the socket connects, so a session.update is not required, but its session config is then fixed: instructions, variables and session options (outbound, ivr_mode, telephony, placeholders, call_events) sent after that are refused with an error event. Add &wait_for_session_update=1 to the URL to start it at your first session.update instead, which then carries instructions, variables and session options the same way; an outbound session needs this. Until that update arrives, caller audio is held, other events are refused, and the socket closes if none arrives within 10 seconds.

The greeting remains available, but Realtime disables the agent's spoken silence check-ins and offers transfer_call and send_keypad_input only with call_events. Other compatible tools run on ThunderPhone; client-executed tools need an inline session. Audio formats and the session's instructions and variables cannot change after the call starts.

Inline session#

Without a saved agent, wait for session.created, then send session.update:

{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "instructions": "Answer questions clearly and keep responses brief.",
    "audio": {
      "input": {
        "format": { "type": "audio/pcm", "rate": 24000 }
      },
      "output": {
        "format": { "type": "audio/pcm", "rate": 24000 },
        "voice": "olivia"
      }
    },
    "config": {
      "product": "bolt"
    }
  }
}

The first accepted update provisions the call. session.updated means the session is live. Instructions, voice, product, tools, and audio formats cannot change after that point. To keep the instructions fixed and vary only named values, write {{name}} placeholders into them and send the values in session.config.variables.

Inline sessions have no automatic greeting or spoken silence check-ins. To make the agent speak first, append a system or user message and send response.create. A completely idle session still ends at the platform's silent-call limit, 600 seconds by default.

Use the official OpenAI SDK#

Pass a WebSocket base URL ending at /v1; the SDK appends /realtime. The model value is a compatibility name and does not select the ThunderPhone product. Choose the product in the session configuration or the saved agent.

This connection check creates an inline Bolt session, prints events until the first session.updated, and closes. Use the minimal Python client to stream audio.

import asyncio
import os
 
from openai import AsyncOpenAI
 
 
async def main():
    client = AsyncOpenAI(
        api_key=os.environ["THUNDERPHONE_API_KEY"],
        websocket_base_url="wss://api.thunderphone.com/v1",
    )
 
    async with client.realtime.connect(
        model="thunderphone-realtime"
    ) as connection:
        await connection.session.update(session={
            "type": "realtime",
            "instructions": "Listen to the caller and help them complete the call.",
            "config": {"product": "bolt"},
        })
        async for event in connection:
            print(event.type)
            if event.type == "session.updated":
                break
 
 
if __name__ == "__main__":
    asyncio.run(main())

Keep your existing handling for input-audio append, response audio deltas, interruptions, function calls, errors, and clean socket closure. Inline custom functions execute in your client; return their results through the Realtime protocol. Saved-agent tools execute on ThunderPhone.

Session lifecycle and failures#

One WebSocket represents one call. An invalid agent ID or rejected inline configuration produces an error event. Wait for session.updated before treating the session as live. Declare the actual input and output formats and sample rates: a PCM rate mismatch plays audio too fast or too slowly instead of producing a validation error.

After the session starts, close the socket cleanly when your application is done. Inline sessions can use client-executed functions. Saved-agent sessions use compatible ThunderPhone-run tools, and offer transfer_call (plus send_keypad_input on outbound sessions) only with call_events; your client carries out those call.* events.

Test the integration#

Start with the minimal Python WAV client and a mono PCM16 WAV at its declared rate. Verify:

1. The server accepts the configuration and sends session.updated.
2. Input produces transcript and response-audio events at the expected speed.
3. Interruption and response cancellation stop remaining output audio.
4. Inline function results or saved-agent tool results return to the model.
5. Invalid input produces an error event your client handles.
6. Your client closes the socket and the call appears in Call history.

The Realtime WebSocket reference lists the accepted events, audio formats, session fields, and complete examples.

Cost#

Realtime calls are priced like any other call: what their parts cost at our providers' list prices, plus an infrastructure charge and a markup. Enabling live transcript deltas adds transcription work for the whole session; see Live transcripts and Pricing.

POST /v1/realtime/sessions is a separate managed LiveKit path. It creates a room and scoped participant token; it is not required for a direct WebSocket connection.

For framework integrations, see Use ThunderPhone from Pipecat and Use ThunderPhone from LiveKit Agents.