OpenAI Realtime API Guide 2026: Build a Voice Agent That Interrupts and Listens Live

You want a voice assistant that responds in under a second without the turn-taking lag of STT-to-LLM-to-TTS pipelines. That is what the Realtime API was built for.

## OpenAI Realtime API: Speech-to-Speech Without the Pipeline Lag The OpenAI Realtime API, launched in October 2024, lets you send audio directly to the model and receive audio back - no separate speech-to-text or text-to-speech calls in between. Latency drops to roughly the time it takes the model to think, and the API supports natural turn-taking, including the model hearing you while it speaks and stopping when interrupted. ## Why Teams Choose It Over the Classic Pipeline The classic stack (Whisper to transcribe, an LLM to think, TTS to speak) has two problems: each hop adds 300-800ms, and the pipeline has no sense of conversation timing. The Realtime API keeps one WebSocket connection open and streams audio both ways. For phone agents, kiosks, and in-car assistants, that changes the feel from walkie-talkie to conversation. ## What You Get in the API - **Audio in, audio out** over a single WebSocket, using the gpt-4o-realtime or gpt-4o-mini-realtime family of models. - **Function calling** - the agent can look up order status or book a slot mid-conversation and fold the result into its answer. - **Turn detection built in** - the model detects when the user stops speaking and when it is being interrupted. - **Input and output audio samples** - you get the exact audio both ways for recording and compliance. ## Pricing Reality Check Audio is billed per minute, and audio tokens are more expensive than text tokens - roughly $0.06 per minute of audio input and $0.24 per minute of audio output on the standard realtime model tier (see OpenAI's pricing page for current rates). A 5-minute support call therefore costs on the order of a dollar and a half before any text-tool calls. That is why many production voice agents only use the Realtime API for the actual conversation and fall back to cheaper text calls for background lookups. ## A Minimal Working Example ```python from openai import OpenAI client = OpenAI() with client.beta.realtime.connect(model="gpt-4o-realtime") as conn: conn.send({"type": "session.update", "session": {"instructions": "You are a hotel concierge."}}) # stream mic audio in, receive audio deltas out ``` The pattern is the same for every app: open a socket, send session config, stream audio in, play audio deltas out. ## FAQ **Do I need a separate TTS model?** No - the model produces speech directly. **Can I use it for phone calls?** Yes, it pairs with telephony providers like Twilio; the audio bridges into the same WebSocket. **Is the Realtime API more expensive than text?** Yes, audio tokens cost more than text tokens per minute of use. **Does it support languages other than English?** Yes, the underlying models handle dozens of languages natively.