How latency is measured in voice AI

Speech-to-text vendors publish first-word latencies of 60 to 150 milliseconds. Text-to-speech vendors publish first-audio latencies of 40 to 80 milliseconds. Production voice agents take 800 milliseconds to 2 seconds per turn.

Every one of those figures is defensible, and together they explain nothing. The component numbers are generation time on warm hardware, measured where the vendor controls the conditions. The system number is what a caller sitting on a phone call experiences. The distance between them is not measurement error. It is the segments nobody counts.

A conversational turn crosses a path, and what a caller experiences is the sum of the delays still on the critical path once streaming and overlap have done their work. Audio has to leave a mouth, cross a network, get decoded, get understood, get answered, get synthesized, and cross a network again. Most published latency claims measure one segment of that path and present it as the whole. The claim is not usually false. It is answering a narrower question than the reader thinks.

The segments of a conversational turn, in order

Here is what a turn traverses, from the moment a caller stops speaking to the moment they hear a response.

Before any AI touches the audio:

That is roughly 60 to 100 milliseconds inbound on a PSTN call, and the same again outbound. A typical PSTN call spends roughly 120 to 200 milliseconds on two-way media transport before AI processing begins, and that portion sits largely outside the AI application's control. A WebRTC call from a browser skips the carrier segment and pays less, which is one reason browser demos feel faster than the phone number they ship behind.

For scale: ITU-T Recommendation G.114 puts 150 milliseconds as the one-way budget below which conversation feels transparent. A single direction of PSTN transport can consume most of it before a model has seen a word.

The processing segments, where vendor figures and measured figures diverge:

The text-to-speech gap is the instructive one. Vendor figures are generation time on warm hardware. A number measured from a client includes the network round trip, gateway processing, and queueing on shared inference capacity.

SignalWire measured it directly, across every synthesis engine the platform supports. Identical text to each engine. Requests were built from each engine's real wire format rather than from its documentation, and called round-robin, so network and backend drift hit every vendor equally.

The first measurement was time to the first byte of the response body. Then a second one, because the first byte is frequently not audio. Streaming synthesis APIs commonly open with container metadata, a RIFF or ID3 header. Worse, the audio that follows often begins with digital silence, and silence plays out at 1x realtime like any other sound. What a caller waits through is both: time to first byte plus leading silence.

Measuring the second number reorders the results.

Azure has the fastest first byte of anything measured and the most leading silence, which moves it from first place to third. ElevenLabs is the actual leader, with 20ms of padding and 203ms to audible speech. Fish gains for the same reason in reverse: 10ms of padding means its 475ms is nearly all real synthesis, and it passes Grok once the silence is counted.

The spread from fastest to slowest is 11x, against a vendor-claimed range of 40 to 80ms for the category.

Leading silence was measured by decoding each response to 16 kHz mono and scanning in 10ms windows for the first that clears 2% of that clip's own peak, so an engine that renders quietly is not mistaken for one that pads. MP3 decoder delay of roughly 25 to 30ms sits inside the figures for the MP3 engines, which means differences under about 30ms between an MP3 engine and a WAV or raw-PCM engine are not meaningful. Azure's 240ms is far too large to be codec delay. That is real padding.

The transport reorders it again

Everything above is the request-per-utterance path. Six of these engines also accept text over a persistent socket, and measuring that reorders the table a second time.

Smallest is not a slow engine. It has a slow request path. It measures 1135ms one way and 322ms the other, moving from second-worst to mid-table. A comparison that ignores transport gets that engine badly wrong, and it is the third time in this section that a better measurement has changed who wins.

Fish moves the other way, and for an unexpected reason. It is second-fastest to produce audio on a warm socket at 178ms, then pads it with 260ms of leading silence, against 10ms of padding on its request path. Same engine, same voice, same text, and the streaming path emits far more silence in front of it.

Streaming buys almost nothing on a cold socket. Connecting, negotiating, and sending the first text costs roughly what a plain request costs: 193ms against 203 for ElevenLabs, 610 against 605 for Rime, 531 against 527 for Grok. The entire benefit lands on the second utterance onward. A platform that opens the socket when the language model starts generating turns every utterance into a warm one. A platform that opens it on demand pays connect cost on the first utterance of every turn, and streaming becomes close to pointless.

That distinction is architecture, not vendor selection. The same engine, reached two ways, differs by 3.5x.

Warm sockets also cut variance rather than only the mean. Three consecutive warm measurements came in at 73, 74, and 69 milliseconds for one engine, and 367, 376, and 353 for another. Tight enough to design against, which the request path was not.

The caveats travel with all of it. Three trials per engine, on a single box in a single region, separates tiers reliably and cannot split engines a few tens of milliseconds apart. Request-path figures each paid a fresh TLS handshake, where a production integration reuses connections. The broad performance tiers are more meaningful than the precise ordering: three trials separate 203ms from 2330ms confidently and cannot rank engines 40 to 100 milliseconds apart.

So the first byte, the first audio sample, and the first sound a human perceives are three different events, routinely reported as one. The gap between them is not small, it is not evenly distributed across vendors, and it moves when the transport changes. A leaderboard built on the wrong one does not merely overstate. It recommends the wrong engine.

Time to first token has the same problem. It is the moment the first token becomes available. It is not a sentence, not an answer, and not audio. A caller cannot hear a token.

The language model has the same problem, one layer up

Time to first token is the figure the model vendors publish. A voice pipeline cannot use it directly. Most production pipelines wait for a usable clause before committing text to synthesis, because sending arbitrary token fragments damages prosody and makes corrections hard to unwind. Streaming engines will accept incremental text, so this is a policy choice rather than a hard requirement, but it is a common production policy. The number that governs perceived responsiveness is time to the first speakable chunk.

SignalWire measured both across every model the platform can reach, on 30 July 2026, round-robin, three trials each:

Sorted by the number that decides when a caller hears something:

The gap between the two columns is not uniform, which is what makes the substitution dangerous. It reorders the table exactly like the synthesis measurements did. gpt-5.4-nano has the second-fastest first token of anything here at 395ms, and gpt-4.1-nano is 36ms behind it. Measured to speakable output the order flips, and gpt-4.1-nano wins by 69ms. gpt-4.1-mini is the starker case: respectable at 628ms to first token, then another 570ms to close a sentence, which puts it 739 milliseconds behind gpt-4.1-nano on the metric a caller experiences.

The most expensive mistake available is choosing a model by its name. gpt-5-nano takes over five seconds to produce a first token, reproducible across every trial. It is a reasoning model that spends its opening budget on reasoning tokens before emitting content, so a developer picking "nano" expecting a fast small model gets the slowest thing measured. The current gpt-5.4 generation behaves nothing like it. gpt-5.4-nano reaches speakable output in 528ms against gpt-5-nano's 5163ms, a 10x difference between two models one generation apart with the same size label.

The same metric choice reorders vendors here too. Reaching gpt-4o through Azure rather than OpenAI directly costs 450ms on first token, which reads as a rout. Measured to speakable output the gap narrows, because Azure buffers and delivers a larger first chunk that already contains a complete sentence, so its two numbers are nearly identical at 914 and 914. OpenAI still wins on the metric that matters, but the size of the win depends entirely on which number is quoted.

One caveat belongs with these figures: time to a speakable chunk depends partly on how a model chooses to phrase an answer. A model that opens with a short sentence scores better than one that opens with a long clause, independent of raw speed. The prompt asked for short sentences to limit that effect without fully controlling it.

What the caller actually waits for, end to end

Combining the best of both measurements, model time to first speakable chunk plus synthesis to audible speech:

Within the inference-and-synthesis portion of a turn, model generation dominates. Moving synthesis from the worst configuration to the best saves about a second. Choosing the wrong model costs five. Both decisions are made in a config file, by developers who have mostly been reading vendor pages that report neither of these numbers.

What speech recognition latency is not

First-word latency is not a useful measure of conversational responsiveness, and the reason is worth walking through, because it is the same reasoning that invalidates most published component figures. It matters for other things, including live captions and speculative generation. It does not tell a caller when they will hear a reply.

SignalWire measured speech onset to first transcript text at roughly 861 milliseconds median across 28 segments. That number looks alarming and means almost nothing. It is dominated by the caller talking. "I am sick" takes about 800 milliseconds to say, and no system can report a phrase before it has been spoken.

The distribution says more than the median. Excluding the first turn of a call, every measurement landed between 851 and 920 milliseconds regardless of what the caller said. That tightness is not the pipeline being consistent. It is the recognizer's fixed interim cadence, emitting a first partial at a set interval after speech onset. The first turn of a call is the outlier at 1440 milliseconds, the same warm-up lag that makes the opening turn the slowest on every metric.

There is a more useful framing, which is how long after a word finishes being spoken it lands in the application. SignalWire does not publish a figure for it, because one does not exist yet. Deriving it from current timestamps means using the recognizer's own word timings as the zero point, and that is a different clock from the audio ground truth. The two disagree by roughly 380 milliseconds on the same event. One turn reads 170 milliseconds on one clock and 554 on the other, for the same moment in the same call.

That gap is a clock offset, not a delay. Publishing it as a latency figure would be a measurement error dressed as a result. The fix is a stamp at first-partial arrival measured against true audio onset. It is a small change and it has not been made.

What a turn actually waits on is not the first word at all. It is the final transcript, which arrives only after the recognizer decides the utterance ended. That decision, and what it costs, is the largest segment remaining.

These segments overlap, and that matters

A well-built pipeline does not run these stages one after another. Recognition streams while the caller is still speaking. Generation can begin before recognition finalizes. Synthesis streams while the model is still producing text. So a turn is a critical path through overlapping work rather than a straight addition, and an accounting that ignores the overlap is wrong.

Overlap is real and it is worth engineering. It also has limits that get glossed over.

Overlap compresses segments. It does not delete boundaries. Two services that stream concurrently still exchange bytes across a network, still hold independent queues, and still fail independently. And overlap cannot compress deliberate waiting, because waiting for a caller to finish is not work that can be started early. It is time spent on purpose, and it is usually the largest single segment in the turn.

How a 300 millisecond claim gets built

There are four reliable ways to produce a number far below what a caller experiences.

Start the timer late. Begin measuring after speech recognition has finalized, after the prompt is assembled, or after the request reaches a warm worker.

Stop the timer early. Stop at the first token emitted internally, the first byte flushed by the server, or the first filler syllable handed to a synthesizer.

Measure the friendly case. A small model, a short prompt, batch size one, no queueing, a warm cache, and an idle high-end GPU.

Reuse the prompt. Prefix caching makes a repeated prompt dramatically faster than a novel one, which is the opposite of what a live conversation produces.

Measure a byte instead of a sound. Report time to first byte when the first bytes are a file header, or ignore the leading silence the synthesizer puts in front of the speech. Correcting for this reorders the synthesis leaderboard above.

None of these repeal the compute cost. They relocate it outside the measurement. Component vendors publish generation time because that is the part they control, which is reasonable of them and useless to anyone assembling a pipeline. The number that matters is the one observed from where the audio has to travel.

One metric survives all five tricks, because it brackets everything: from the moment the caller stops making sound to the moment the agent starts making sound. SignalWire measures it from the recorded waveform of real calls, in stereo, with the caller on one channel and the agent on the other. The tool is open source at signalwire/latency_checker, and it works against any platform.

Every vendor boundary is a segment

The path above assumes the pieces are adjacent. In an assembled stack they are not. Audio typically travels from the carrier to a telephony provider, then over a WebSocket to the developer's server. From there it goes out to a speech-to-text service, out to a model, out to a synthesis service, and back through the chain. Every boundary introduces another network path, queue, scheduler, and failure domain. Depending on placement and load the penalty ranges from negligible to hundreds of milliseconds, and six boundaries is a common count.

Boundaries do not only add to the mean. They add variance, because each service has its own queue, its own scheduling, and its own failure behavior. Variance is what a caller experiences as an agent that is usually fine and occasionally strange.

The transport choice compounds it. Streaming audio over a WebSocket means streaming over TCP, and TCP guarantees ordered delivery. When a packet is lost, everything behind it waits for the retransmission. A 200 millisecond stall to recover a fragment of a syllable damages a conversation more than the missing fragment would have. Real-time media protocols use UDP for this reason: a dropped frame is better than a frozen one. Reliability is the wrong guarantee for audio that is only useful if it arrives on time.

There is also a structural cost to routing media through an application at all. A voice call delivers 50 audio frames per second, per call. A stack that pipes those frames into application code pays for that on every frame, on every concurrent call.

SignalWire separates signaling from media. Application code controls the call while the audio stays in the media path and never enters the application process. A developer gets the same control over the conversation without the frames passing through their runtime.

The segment nobody counts

Every number above is processing. There is one more segment, it is often the largest, and it appears in no vendor's benchmark: deciding that the caller is finished.

SignalWire's published benchmark shows the size of it. One configuration waits 1000 milliseconds of silence before treating a turn as ended and averages 1.46 seconds per conversational turn. Another waits 250 milliseconds and averages 1.10 seconds. Both ran gpt-4o-mini and the same ElevenLabs voice. The faster configuration also lowered sampling temperature, so the timeout is not the only variable, but it is by far the largest.

A third of a second of measured latency, moved by a waiting policy rather than by any optimization. No component got faster between those two runs. The system was told to be less patient.

The reduction is smaller than the change in the setting, which is itself informative. The timeout is a ceiling on how long to wait for more speech, not a fixed pause added to every turn. Callers frequently stop talking in ways that resolve before the ceiling is reached.

What human conversation actually does

Turn-taking has been measured properly. Stivers and colleagues, publishing in PNAS in 2009, examined ten languages ranging from traditional indigenous communities to major world languages. The modal transition between speakers falls between 0 and 200 milliseconds, and cross-language means sit inside a band of about 250 milliseconds.

The finding that matters most is the shape of the pressure. Every language studied showed "a general avoidance of overlapping talk and a minimization of silence between conversational turns." Two competing pressures, held in tension, universal to the species.

Note what that rules out. Humans do not reach 200 milliseconds by detecting silence and then beginning to think. There is not enough time. Humans predict the end of a turn while it is still in progress, and prepare a response before the other person finishes.

Any system that commits after end-of-speech inherits a floor above the human number. That floor is worth stating plainly rather than marketing around. The useful question is not how to beat 200 milliseconds. It is where the remaining time is spent, and whether it is buying anything.

Interrupting is worse than pausing

Ending a turn too early and ending it too late are both failures, and they are not symmetric.

Too early, the agent interrupts. The caller's sentence is cut in half, or the closing word is clipped and the agent answers a question that was never asked. The caller has to start over. The conversation breaks.

Too late, there is dead air. It is irritating, it accumulates across a call, and the conversation survives it.

One failure costs a turn. The other costs the call. A system tuned purely for a lower average will trade the expensive failure for the cheap one. It will do so precisely where it hurts most: long sentences, dictated numbers, and people who think while they talk.

Adaptive turn detection

SignalWire runs adaptive turn detection built in-house. The design follows from that asymmetry, and it weighs several independent kinds of evidence, chosen because they fail in different places.

How the caller sounded. A model judges whether the delivery of the last stretch of audio is that of someone finishing a thought or someone pausing inside one. This works on sound rather than words, so it is language-agnostic.

Whether the words form a complete thought. "What is the weather in" is unfinished no matter how confidently it was spoken.

What the caller is evidently doing. Reading a phone number, spelling a name, and listing items have recognizable shapes, and pauses inside them are structural rather than terminal.

Each is blind exactly where another sees:

One asymmetry does the safety work: the meaning and activity signals can only delay a commit, never accelerate one. A signal that can only apply the brake cannot cause an interruption. The system's ability to be wrong is bounded on the side where being wrong is cheap.

The behavior is dynamic and deterministic rather than a fixed pause. Where the evidence is unambiguous, it commits immediately. Where the evidence conflicts, it holds the floor. Holds are bounded several ways, so a wrong hold costs measurable extra time and can never strand a caller in silence.

This works with the end-of-speech timeout rather than replacing it. The timeout remains the outer bound, the answer to "how long before giving up on more speech." Adaptive detection commits earlier than that bound when the evidence is clear, and spends the wait only when the evidence says spending it prevents an interruption.

What adaptive turn detection costs, measured

Turn detection latency, measured end-of-speech to turn delivered, over 27 turns (nine turns across three runs) against a live transcriber. Ground truth for end-of-speech comes from the audio, not from the transcriber's own word timings.

Nine of fifteen ordinary turns landed under 500 milliseconds, and fourteen of fifteen under 750.

The figure for a normal turn is 496 milliseconds. The pooled mean across all three classes is 1004 milliseconds, and it averages three populations with deliberately different targets.

Multi-sentence turns are the design working. "Hello. I was thinking about talking to you" contains a complete sentence, a natural stop, and then more. Committing at that internal break would split one thought into two turns, and the agent would answer the first half. The extra 400 milliseconds buys the whole thought.

Fragmentation recovery is a different story, and it is not a decision problem. When a transcriber splits a final word, emitting "Sport" before correcting to "Sports.", the system waits for the correction instead of publishing the truncation. That cost is paid to an upstream vendor's behavior, and the lever for improving it is the vendor.

Ordinary turns face a related limit. They will not go much below 496 milliseconds without changing where detection happens. What remains is dominated by how quickly the transcriber declares a segment finished, which is upstream of every decision described here.

These figures come from a turn-detection measurement, not from the full-call benchmark quoted earlier. The published platform numbers were produced with a fixed timeout.

The turns where agents actually feel slow

Conversational turns are the ones vendors publish. Tool turns are the ones callers remember, because the agent goes quiet while an API is queried. Measured across platforms on the same mock API, tool turns run 2.0 to 3.7 seconds, against roughly 1 to 1.9 for conversational turns.

The fix is not always a faster tool. Speech fillers play immediately while the tool runs in the background, which changes what the caller experiences from silence to a response. In the same benchmark, adding fillers moved a tool turn from 3.01 seconds of dead air to 1.40 seconds before the caller heard anything. That is a 53% reduction in perceived latency, with no change to the underlying tool. That configuration completed one call, so treat it as directional rather than definitive.

When the models collapse the middle

Speech-to-speech models fold recognition, reasoning, and synthesis into a single pass, and the obvious objection to a segment-by-segment article is that they erase the segments. They erase some of them. Conversational turns land around 600 milliseconds on SignalWire with speech-to-speech voice models, against roughly 1200 typical on a cascaded pipeline.

As the models get faster, the remaining latency is architecture. A speech-to-speech model collapses the middle of the turn. The phone network, the transport, and the turn-taking do not disappear, and turn-taking becomes a larger share of what is left.

Turn-taking policy belongs in code

Turn-taking posture is a product decision. A collections agent, a medical intake line, and a trivia game want different postures, and the right answer changes by step within a single call. Reading back a credit card number needs patience. Confirming "yes" does not.

That decision belongs in code, and on SignalWire it is expressed there:

agent.set_params({
    "end_of_speech_timeout": 250,   # ms of silence before the turn can close
    "barge_confidence": 0.6,        # how sure recognition must be to interrupt the agent
    "barge_min_words": 2,           # a cough is not an interruption
})

agent.set_params({
    "end_of_speech_timeout": 250,   # ms of silence before the turn can close
    "barge_confidence": 0.6,        # how sure recognition must be to interrupt the agent
    "barge_min_words": 2,           # a cough is not an interruption
})

agent.set_params({
    "end_of_speech_timeout": 250,   # ms of silence before the turn can close
    "barge_confidence": 0.6,        # how sure recognition must be to interrupt the agent
    "barge_min_words": 2,           # a cough is not an interruption
})

agent.set_params({
    "end_of_speech_timeout": 250,   # ms of silence before the turn can close
    "barge_confidence": 0.6,        # how sure recognition must be to interrupt the agent
    "barge_min_words": 2,           # a cough is not an interruption
})

These are per-agent defaults, adjustable per step, and changeable mid-call from a tool handler. An agent can become more patient the moment it asks for an account number and less patient once it is confirming.

The alternative is to write the policy into a prompt and hope. A prompt cannot hold the floor open while someone reads out a phone number. By the time the model sees the text, the turn boundary has already been decided. Turn-taking happens below the model, in the media path, in the milliseconds before there is anything for a model to read.

Timing is the clearest case of a more general pattern. The model makes the conversation sound natural. The code decides what actually happens. Everything a caller experiences as judgment, when to speak, when to wait, when to hand off, is settled by software before the model is involved at all.

The complete picture of voice AI latency

A defensible latency claim states where the timer started, where it stopped, and what sits between them. Anything else is a component measurement wearing a system's clothes.

The accounting for a conversational turn, with the caveat that these stages overlap rather than stack:

• 120 to 200ms of transport on a typical phone call, largely outside the AI application's control

• A turn decision, from under 200ms to roughly a second, the largest controllable segment

• Recognition, inference, and synthesis, compressed by streaming but bounded by time to the first speakable output

• A penalty per vendor boundary, in both mean and variance

SignalWire's measured full round-trip runs 1.09 to 1.46 seconds across configurations, against 1.87 for LiveKit and 1.85 for Vapi. The comparison is not perfectly matched, and the published data says so. The 1.46 configuration matches LiveKit on model and voice. The 1.09 configuration uses an open-weight model and a shorter timeout, so part of that gap is a model choice rather than an architecture difference. LiveKit was reached over PSTN while SignalWire and Vapi were reached over SIP, which disadvantages LiveKit on transport. Sample sizes are three calls per configuration.

Publishing the qualifications is the point. The full-call figures come from waveform analysis of real calls, and the tool that produced them is public, so that comparison can be re-run and disputed by anyone.

The component tables are a different matter. The synthesis and model measurements were made with an internal harness, and a number nobody else can reproduce is an assertion however carefully it was taken. Publishing that harness, along with the raw timings, the exact prompt text, voices, audio formats, client region, and connection-reuse policy, is the obvious next step and has not been done yet.

What the architecture contributes is fewer segments, because the AI runs inside the media path rather than across a network from it. The second contribution is a turn decision made on evidence, rather than by a timeout set once and forgotten.

Speed is not the goal. An agent that never interrupts and answers in a second beats one that averages 700 milliseconds and talks over people reading their account numbers. The goal is a system that knows the difference, and that difference is engineering rather than a prompt.

To find out what a stack actually costs, signalwire/latency_checker measures end-of-speech to first audio from a call recording, on any platform.