Every voice AI vendor advertises a latency number. Vapi says ~500ms. Retell says ~800ms. Synthflow points at a Deepgram blog claiming ~300ms STT. The catch: none of those numbers describe what the caller on the other end of the phone actually hears.
We spent six weeks running an open test harness against five production voice platforms — Retell, Vapi, Synthflow, Deepgram Voice Agent, and Finn — across US-East, EU-West, and India routes. This post publishes the P50/P90/P99 numbers, a methodology you can rerun yourself, and the parts of the pipeline vendors quietly leave out of their marketing graphs.
Why "latency" is the wrong number
When a vendor markets latency: 500ms, they almost always mean TTFB — time from end-of-user-utterance to the first audio byte the TTS emits. That's a useful engineering number. It is not what the user perceives.
What the user perceives is turn-taking delay: the silence between when they stop talking and when they hear the agent talking. That number includes:
- Endpointing delay — how long VAD waits before deciding you stopped (typically 200–600ms of deliberate silence).
- Network jitter buffer on the SIP/WebRTC leg (40–120ms).
- TTS first-byte to TTS-playable — the first chunk has to be large enough for a jitter-resilient playout.
- Barge-in recovery when the user interrupts mid-response.
A platform that posts 500ms TTFB but uses 600ms endpointing produces a perceived turn-taking delay north of 1.2s. Another vendor advertising 800ms but with aggressive 150ms semantic endpointing feels noticeably faster on the call. Marketing numbers and human-perceived numbers are not the same axis.
Rule of thumb from CX research: under 800ms perceived turn-taking feels conversational. 800–1200ms feels "AI but tolerable." Above 1.2s, callers start talking over the agent.
Anatomy of a voice AI request
Here is the full pipeline for a single turn, with the median budget we measured across the five platforms in 2026:
| Stage | What happens | Median (ms) | P90 (ms) |
|---|---|---|---|
| SIP/WebRTC ingress | Audio frame arrives at media server | 20 | 60 |
| VAD + endpointing | Detect end of user speech | 280 | 520 |
| STT finalization | Force-finalize partial transcript | 90 | 180 |
| LLM TTFT | First token from the model | 320 | 780 |
| LLM → TTS handoff | Sentence boundary + stream start | 40 | 120 |
| TTS first-byte | First audio chunk emitted | 180 | 410 |
| TTS → playable | Enough buffer to start safely | 60 | 140 |
| Egress jitter buffer | Carrier-side smoothing | 50 | 110 |
| Total perceived | End-of-utterance → first audio heard | ~1040 | ~2320 |
The two stages vendors compete on hardest — LLM TTFT and TTS first-byte — are only ~48% of the median budget. Endpointing and jitter compounding own the tail.
The 2026 benchmark table — measured, not marketed
All numbers below are perceived turn-taking delay, US-East caller → vendor default region, recorded over 500 turns per platform on a fixed 8-turn appointment-scheduling script. Hardware: Mac mini M4, PSTN-bridged via a Twilio Programmable Voice number, recordings analyzed with our open script (link in repo below). Full methodology in the next section.
| Platform | Marketed | P50 measured | P90 measured | P99 measured | Barge-in recovery |
|---|---|---|---|---|---|
| Retell | ~800ms | 1180 | 1740 | 2680 | 410ms |
| Vapi (default) | ~500ms | 1020 | 1620 | 2510 | 380ms |
| Vapi (custom STT/LLM) | — | 880 | 1410 | 2240 | 360ms |
| Synthflow | ~600ms | 1240 | 1980 | 3120 | 520ms |
| Deepgram Voice Agent | ~300ms STT | 940 | 1480 | 2390 | 290ms |
| Finn | — | 820 | 1290 | 1980 | 240ms |
Takeaways most vendor benchmarks bury:
- Every platform's P90 is roughly 1.5–2× its P50. If you only watch averages, your worst 10% of calls feel like a different product.
- P99 always crosses 2 seconds. Tail latency is where users hang up.
- Barge-in recovery — how fast the agent shuts up when interrupted — varies 2×. This single number drives more "bad call" complaints than TTFB.
How to reproduce these numbers
The harness is intentionally boring. No proprietary load-gen, no synthetic SIP — a real phone number calling a real agent, with a microphone-loopback timestamping rig.
Frequently asked questions
What does sub-second latency actually mean?
It depends what is being measured. Model inference time, time to first audio, and mouth-to-ear latency are three different numbers, and only the last is what a caller experiences.
Which latency number should I hold a vendor to?
Mouth-to-ear on a real call over a real carrier, measured at the tail rather than the mean. A good average with a bad p95 means one call in twenty is unusable.
Why did latency get worse in production than in testing?
Usually the network path. A demo on a local connection skips the carrier leg and the region hop that production calls pay for.
