Subscribe
Tags
Voice AI
AI Agents
Contact Center
Latency & Performance
Picture a caller phoning to reschedule an appointment. Your AI agent starts reading out opening hours, the cancellation policy and a parking reminder. Halfway through, the caller says, "Sorry, I just need to move Thursday." The agent keeps going. The caller repeats themselves, louder. Now both voices overlap, the transcript is a mess, and the agent answers a question nobody asked.
That moment decides whether a voice agent feels like a helpful assistant or a recorded message with extra steps. Handling it well is what voice AI barge-in is about.
Below, we'll cover what barge-in is, why agents get it wrong, and the settings that shape natural turn-taking.
What is barge-in in voice AI?
Barge-in is the ability for a caller to interrupt an automated voice system while it is speaking, and for the system to stop, listen and respond to what the caller said.
The term comes from traditional IVR. With AI voice agents, expectations are higher: callers talk to them the way they talk to people, cutting in with corrections and quick answers without waiting for a pause.
Good barge-in does three things:
Stops the agent quickly when the caller genuinely starts talking.
Ignores sounds that are not real interruptions, such as a cough, a barking dog or a short "mm-hm."
Picks up the conversation cleanly so the agent responds to what the caller said, not to the sentence it was halfway through.
Why does my AI voice agent keep talking over callers?
The instinct is to fix it in the prompt with a line like "If the user interrupts, stop talking and listen." It rarely helps, because the language model is not the part of the system that hears the caller.
By the time the model sees anything, the audio has been captured, run through voice activity detection, transcribed and turned into text. The model's reply is then converted to speech and streamed down the line. If the caller speaks while that audio is playing, something in the media path has to notice, decide it counts and cut the playback. A sentence in the prompt cannot do any of that.
The usual culprits are:
No interruption detection during playback. The system only listens between turns.
Slow cutoff. Speech is detected, but buffered audio keeps playing for another second or two.
Thresholds that are too strict. The caller has to say a lot, or speak very loudly, before the agent yields.
Echo leaking back in. The agent's own voice bleeds into the inbound audio, so the system either interrupts itself or gets tuned to ignore real speech.
Interruption handling is a media-layer problem, not a prompt problem
It helps to think of barge-in as a chain of real-time events, most of which happen before the language model is involved.
What has to happen when a caller cuts in
Speech is detected on the inbound audio. A voice activity detector (VAD) flags speech-like sound.
The system decides whether it is a real interruption. Thresholds decide how loud, how long or how many recognized words it takes.
Playback stops. Text-to-speech output halts, and audio already buffered for the caller is flushed so they do not hear the tail of the old sentence.
The conversation state is corrected. If the reply was cut off after one sentence, the history should reflect that, or the model will assume the caller heard information they never got.
The agent listens, then responds. Turn detection decides when the caller has finished, and the model replies to the new input.
Every step lives close to the audio. That is why a platform that handles call media, speech recognition and AI orchestration together is better placed to get barge-in right than a stack of separate services passing audio and text back and forth.
Why echo cancellation matters
On speakerphones and some mobile networks, the agent's voice can return on the inbound channel. Without echo cancellation, the VAD treats it as caller speech. Teams often raise thresholds until the false triggers stop, which also makes real interruptions harder. Fixing echo at the source lets you keep barge-in sensitive.
The four controls that shape natural turn-taking
Once interruption detection lives in the media layer, tuning comes down to a handful of decisions.
1. Barge-in thresholds
A threshold defines how much evidence the system needs before treating caller speech as an interruption. Common approaches:
Minimum words. One word makes the agent very responsive. Three or four filters out backchannel sounds like "uh-huh" and "okay."
Keyword triggers. Words such as "stop," "wait" or "agent" always interrupt.
There is no universal right number. A booking agent that reads back long confirmations may need a low threshold so callers can correct a date quickly. An agent delivering a required disclosure may need a higher one.
2. Noise handling
Callers phone from cars, kitchens, warehouses and busy streets. If every burst of sound stops the agent, conversations stall. Useful levers include:
Energy thresholds that set how loud audio must be before it counts as speech.
VAD sensitivity, which affects how readily background sound is classified as voice.
Whether non-speech noise can interrupt at all. For most agents, a cough or a door slam should not cut the agent off.
3. Protected greetings and must-hear messages
Some audio should play through even if the caller talks over it. The opening line is the classic example. Callers often say "hello?" as the call connects, which can cut the greeting short.
Legal disclosures and recording notices are other candidates. The key point is that protection has to be configured on the audio itself. Asking the model to "always finish the greeting" does not stop a media-layer interruption.
4. End-of-speech timeouts
Barge-in decides when the agent should stop talking. End-of-speech detection decides when the caller has stopped. Set it too short and the agent jumps in while the caller pauses to think. Set it too long and every answer is followed by awkward silence.
A fixed silence timeout is the simplest approach. Smarter turn detection also looks at punctuation in the partial transcript or at the caller's intonation, so the agent waits when someone sounds mid-sentence. Callers reading phone numbers or spelling names often pause between chunks, so test those moments carefully.
How barge-in settings look on SignalWire
On SignalWire, interruption and turn-taking for AI agents are set as parameters on the agent, not as instructions in the prompt. The public SignalWire Markup Language (SWML) AI params reference lists them under the AI agent's params, including:
enable_barge, which controls when the caller can interrupt the AI. Setting it to false turns barging off.
barge_min_words, the number of words that must be recognized before barge behavior triggers. SignalWire fixes the recognition-confidence threshold internally, so there’s no confidence setting to tune; shape interruptions with barge_min_words and barge_match_string instead.
interrupt_on_noise, which decides whether non-speech sound can interrupt the agent. It is off by default.
static_greeting, an opening line played when the call is answered, and static_greeting_no_barge, which keeps that greeting from being interrupted.
end_of_speech_timeout, the silence in milliseconds that marks the end of an utterance. The documented default is 700 ms.
energy_level and enable_turn_detection, which tune what counts as speech and how the agent decides the caller has finished.
SignalWire's own recipe on controlling caller interruptions makes a point worth repeating: if a prompt tells the model to open with a greeting, that is generated speech, and greeting protection does not apply to it. If the opening line must play in full, make it the static greeting. Check the current SWML reference before you ship, since parameter details can change between releases.
A practical way to tune turn-taking for AI phone calls
Turn-taking is something you have to hear. A simple process works well:
Start from the platform defaults. They are usually a sensible middle ground.
Record realistic test calls from quiet rooms, cars and speakerphones, with callers who pause a lot.
Script deliberate interruptions. Cut in with one word, then three, then a full sentence. Talk over the greeting. Say "mm-hm" while the agent reads a list.
Change one setting at a time so you know which change helped.
Listen to recordings, not just transcripts. Overlaps and awkward gaps are easier to hear than to read.
Once live, watch for calls where the caller repeated themselves or asked for a human right after an overlap. Those are often turn-taking problems in disguise.
Build voice agents that know when to listen
Natural turn-taking is one of the clearest signs that a voice agent was built with care. When interruption handling lives alongside call media and AI orchestration, you can tune it with configuration instead of fighting it with prompt wording.
If you are building or refining an AI voice agent, explore SignalWire's developer documentation to see how barge-in, greetings and turn detection are configured, or talk to the SignalWire team about the calling experience you want to create. Then bring your questions to our developer community on Discord.
How do I handle interruptions in a voice AI agent?
Handle them in the media layer. Detect caller speech during playback, apply a threshold so noise and short acknowledgments do not count, stop text-to-speech and flush buffered audio, then make sure the conversation history reflects what the caller actually heard.
What is barge-in in voice AI?
Barge-in is the ability for a caller to interrupt an AI voice agent mid-sentence and have the agent stop, listen and respond. It makes automated calls feel closer to a real conversation.
Why does my AI voice agent keep talking over callers?
Usually because interruption detection is missing or too strict, buffered audio keeps playing after speech is detected, or echo pushed thresholds so high that real interruptions get ignored. Prompt changes rarely fix it, since the model is not listening to the audio.
Should I ever turn barge-in off?
Sometimes, for short stretches. Required disclosures, recording notices and opening greetings are common cases where audio should play through. For the rest of the call, letting callers interrupt almost always works better.
What is a good end-of-speech timeout for phone calls?
It depends on your callers and the task. Shorter timeouts feel snappy but risk cutting people off mid-thought. Longer ones feel patient but leave gaps. Start with your platform's default, then test with callers reading numbers or spelling names.
Related Resources
