Voice AI latency calculator

How this is calculated

The perceived gap is the sum of every stage between end of speech and first audio: the silence you wait through before deciding the caller has finished, transcription finalisation, the model's first token, the voice's first byte, two network hops, and the telephony path.

perceived gap = endpointing silence      ← your setting
              + STT finalisation
              + LLM time to first token
              + TTS time to first byte
              + network RTT × 2           ← your region choice
              + telephony / jitter buffer

Why it is usually not the model

In a typical setup the endpointing silence threshold is the single largest item — around 600ms of a roughly 1,500ms budget, or 40% of everything the caller waits through. It is not a vendor limit. It is a number in your configuration that decides how long the system listens to silence before concluding the caller has finished, and teams routinely spend weeks benchmarking models while it sits at a default nobody chose.

Shortening it is the cheapest win available, and it is a genuine trade-off rather than free: cut too far and you interrupt anyone who pauses mid-sentence. Region choice is the other lever in your hands — the network round trip is counted twice, so a cross-region deployment can add 400ms before any vendor has done anything wrong.

What good looks like

Natural human turn-taking gaps sit near 200ms. Under 500ms an agent feels immediate; under 800ms it reads as a normal pause. Past about 1.2 seconds callers start repeating themselves or talking over the agent, because the silence reads as a dropped line rather than as thinking. A reasoning model at 1.4 seconds to first token spends the entire budget on its own — which is why those belong after the call, summarising and deciding, rather than on the live turn.

The stage figures here are typical published or observed ranges, not guarantees. Real latency varies with region, load, payload and network conditions — measure your own and use this as a budget to design against.

Frequently asked questions

What is a good latency for a voice AI agent?

Under 800ms feels comfortable and under 500ms feels immediate — natural human turn-taking gaps sit around 200ms. Past about 1.2 seconds callers start repeating themselves or talking over the agent, because the silence reads as a dropped line rather than as thinking.

Why is my voice agent slow when every vendor claims low latency?

Because the vendor numbers are only part of the budget. In a typical setup the endpointing silence threshold — the time you wait to be sure the caller has stopped talking — is the single largest contributor, often 40% of the total. It is a setting on your side, not a vendor limit, and no amount of model shopping will fix it.

How do I reduce voice agent latency?

In order of return: shorten the endpointing threshold, deploy in the same region as your providers, shorten the system prompt, stream text to speech rather than waiting for a complete response, and only then consider a faster model. The first two are free and usually worth more than the rest combined.

What is endpointing?

Deciding when the caller has finished speaking. Wait too long and every reply feels sluggish; cut too early and you interrupt anyone who pauses mid-sentence. It is a genuine trade-off rather than a bug — but it is a trade-off most teams never consciously make, leaving a default in place and blaming the model.

Does a reasoning model make sense for voice?

Rarely on the critical path. A model that takes 1.4 seconds to first token blows the entire budget on its own. Reasoning models fit better after the call — summarising, extracting, deciding next actions — while the live turn runs on something built for time to first token.

Related reading