Bottom Linear Gradient  Lines image

Article

5 min

read

Put the Smart Model on Hold

The hard thinking happens off the conversation's critical path

Adam Kalsey

Adam Kalsey

EVP, Product

Using fillers in AI conversation waits

In this article

Share

Angular Gradient Image

Build it free.

Create a space and ship your first call flow in minutes.

Subscribe

Tags

Voice AI

AI Agents

Latency & Performance

Use the reasoning model as a tool call, not the conversation model

A customer calls in. "My bill went up forty dollars this month and I want to know why."

That's a hard question. Answering it means pulling three months of invoices, comparing line items, finding the new usage or the expired discount, and explaining it in a sentence a person can follow over the phone. A small, fast model won't do that well. A script can't do it at all.

In Smart Code, Dumb Models I explained why we build voice agents so that code runs the conversation and a small, fast model does the talking. Inference only gets the jobs you can't write down as steps, which keeps the agent quick enough to hold a conversation with a person. A billing question like this one looks like it breaks that approach.

So what do you do when the caller asks something that really does need a smart model?

You make the smart model a tool. The fast model never hands the conversation to the slow one. It calls a function, the same way it would look up an order status or check business hours. That function pulls the customer's billing records, sends them to whichever large reasoning model you prefer, and waits for the analysis. When the answer comes back, the fast model speaks it. The smart model never holds the microphone. It works behind the counter like any other backend system.

This is what a good support rep does. Ask a person why your bill went up and they don't answer instantly. They say "Let me pull up your account," you hear some typing, and a minute later they explain it. Nobody minds the minute. Speed matters for turn-taking, not for every answer. What callers mind is silence, and not knowing whether anyone's still on the line.

Filling the wait during a live call

The wait has to sound like a person looking something up rather than a dropped call.

Start with an acknowledgment. Before the tool runs, the agent tells the caller what's about to happen and that it'll take a moment: "Let me look at your billing history. Give me just a second."

During the lookup, use SignalWire's fillers. These are phrases the agent speaks while a function runs:

"Just a moment more."

"Thanks for your patience."

"This is taking a little longer than usual."

"Almost there."

Spaced out over the wait, they do what a human rep does without thinking about it. They tell the caller the call is alive and someone is working on the problem.

Under the fillers, play background audio instead of dead air. Keyboard clicks work well because they're what a caller would hear from a person doing the same job. An audio throbber, a soft repeating tone, works too. Either one covers the gaps between filler phrases so the line never goes quiet.

185  if BACKGROUND_AUDIO_URL:
186      params |= {"background_file": BACKGROUND_AUDIO_URL, "background_file_volume": -20}
187  self.set_params(params)
     (...)
255  self.define_tool(
256      name="explain_bill_increase",
     (...)
266      fillers={"en-US": [[
267          "Just a moment more.",
268          "Thanks for your patience.",
269          "This is taking a little longer than usual.",
270          "Almost there.",
271      ]]},
272  )
185  if BACKGROUND_AUDIO_URL:
186      params |= {"background_file": BACKGROUND_AUDIO_URL, "background_file_volume": -20}
187  self.set_params(params)
     (...)
255  self.define_tool(
256      name="explain_bill_increase",
     (...)
266      fillers={"en-US": [[
267          "Just a moment more.",
268          "Thanks for your patience.",
269          "This is taking a little longer than usual.",
270          "Almost there.",
271      ]]},
272  )
185  if BACKGROUND_AUDIO_URL:
186      params |= {"background_file": BACKGROUND_AUDIO_URL, "background_file_volume": -20}
187  self.set_params(params)
     (...)
255  self.define_tool(
256      name="explain_bill_increase",
     (...)
266      fillers={"en-US": [[
267          "Just a moment more.",
268          "Thanks for your patience.",
269          "This is taking a little longer than usual.",
270          "Almost there.",
271      ]]},
272  )
185  if BACKGROUND_AUDIO_URL:
186      params |= {"background_file": BACKGROUND_AUDIO_URL, "background_file_volume": -20}
187  self.set_params(params)
     (...)
255  self.define_tool(
256      name="explain_bill_increase",
     (...)
266      fillers={"en-US": [[
267          "Just a moment more.",
268          "Thanks for your patience.",
269          "This is taking a little longer than usual.",
270          "Almost there.",
271      ]]},
272  )

Keeping the answer around

Have the tool return two things: a short summary for the agent to speak now, and the structured data behind it. Put the structured data in global data, where the agent can reach it for the rest of the call.

366      return (
367          FunctionResult(f"Tell the caller, in your own words: {result['spoken_summary']}")
368          .update_global_data({"bill_analysis": result["analysis"],
369                               "bill_summary": result["spoken_summary"],
370                               "analysis_status": "done"})
371      )
372
373  def get_bill_detail(self, args: dict, raw_data: dict) -> FunctionResult:
374      """Answers follow-ups from global_data. No billing API, no slow model."""
375      analysis = (raw_data.get("global_data") or {}).get("bill_analysis")
366      return (
367          FunctionResult(f"Tell the caller, in your own words: {result['spoken_summary']}")
368          .update_global_data({"bill_analysis": result["analysis"],
369                               "bill_summary": result["spoken_summary"],
370                               "analysis_status": "done"})
371      )
372
373  def get_bill_detail(self, args: dict, raw_data: dict) -> FunctionResult:
374      """Answers follow-ups from global_data. No billing API, no slow model."""
375      analysis = (raw_data.get("global_data") or {}).get("bill_analysis")
366      return (
367          FunctionResult(f"Tell the caller, in your own words: {result['spoken_summary']}")
368          .update_global_data({"bill_analysis": result["analysis"],
369                               "bill_summary": result["spoken_summary"],
370                               "analysis_status": "done"})
371      )
372
373  def get_bill_detail(self, args: dict, raw_data: dict) -> FunctionResult:
374      """Answers follow-ups from global_data. No billing API, no slow model."""
375      analysis = (raw_data.get("global_data") or {}).get("bill_analysis")
366      return (
367          FunctionResult(f"Tell the caller, in your own words: {result['spoken_summary']}")
368          .update_global_data({"bill_analysis": result["analysis"],
369                               "bill_summary": result["spoken_summary"],
370                               "analysis_status": "done"})
371      )
372
373  def get_bill_detail(self, args: dict, raw_data: dict) -> FunctionResult:
374      """Answers follow-ups from global_data. No billing API, no slow model."""
375      analysis = (raw_data.get("global_data") or {}).get("bill_analysis")

The first answer is rarely the last question. Once the agent explains that call volume doubled in September and pushed the account past its included minutes, the caller wants to know which days, what the charge on the fourteenth was for, and whether August looked the same.

If the analysis lives in global data, the fast model answers those follow-ups immediately from what's already there. Without it, every follow-up is another trip to the slow model and another wait. The smart model does the hard work once, and the fast model reuses it for the rest of the conversation.

Guardrails

Give the tool a timeout and a fallback. If the analysis hasn't come back in a reasonable time, the agent should say so and offer something useful: an email with the explanation, a callback, or a transfer to a person. A caller who's heard "almost there" for three minutes has been lied to.

353  future = analysis_pool.submit(ask_reasoning_model, REASONING_PROMPT, invoices)
354  try:
355      result = future.result(timeout=ANALYSIS_TIMEOUT_SECS)
356  except AnalysisTimeout:
     (...)
360      return FunctionResult(
361          "The analysis is taking too long. Apologize, say it's still being worked on, "
362          "and offer three options: an email with the explanation, a callback, or "
363          "connecting them to a person now."
364      ).update_global_data({"analysis_status": "timed_out"})
353  future = analysis_pool.submit(ask_reasoning_model, REASONING_PROMPT, invoices)
354  try:
355      result = future.result(timeout=ANALYSIS_TIMEOUT_SECS)
356  except AnalysisTimeout:
     (...)
360      return FunctionResult(
361          "The analysis is taking too long. Apologize, say it's still being worked on, "
362          "and offer three options: an email with the explanation, a callback, or "
363          "connecting them to a person now."
364      ).update_global_data({"analysis_status": "timed_out"})
353  future = analysis_pool.submit(ask_reasoning_model, REASONING_PROMPT, invoices)
354  try:
355      result = future.result(timeout=ANALYSIS_TIMEOUT_SECS)
356  except AnalysisTimeout:
     (...)
360      return FunctionResult(
361          "The analysis is taking too long. Apologize, say it's still being worked on, "
362          "and offer three options: an email with the explanation, a callback, or "
363          "connecting them to a person now."
364      ).update_global_data({"analysis_status": "timed_out"})
353  future = analysis_pool.submit(ask_reasoning_model, REASONING_PROMPT, invoices)
354  try:
355      result = future.result(timeout=ANALYSIS_TIMEOUT_SECS)
356  except AnalysisTimeout:
     (...)
360      return FunctionResult(
361          "The analysis is taking too long. Apologize, say it's still being worked on, "
362          "and offer three options: an email with the explanation, a callback, or "
363          "connecting them to a person now."
364      ).update_global_data({"analysis_status": "timed_out"})

Ask the smart model for output shaped for speech. Hand a model three months of invoices and it'll happily write you a report with headings. Tell it you want two or three sentences a person can understand on the phone, plus the data that supports them.

Then look at the cost. You pay for the expensive model only on calls that need it, and only for the one question that needs it. Everything else on the call runs on the fast, cheap model.

Smart where it counts

Smart code, dumb models isn't a rule against smart models. It's a rule about where they go. Keep them out of the turn-by-turn conversation, where every hundred milliseconds counts. Put them behind tool calls, where a few seconds of thinking is the same pause a person would take. Code still runs the call and the fast model still does the talking. When a caller asks a question that needs real thought, the agent can say "let me look at that" and mean it.

If you're building the future of voice AI, build it with us. Sign up for a free account and join the discussion on our community Discord.

Bottom Linear Gradient  Lines image

The Communications Stack for What's Next

APIs built for speed. Infrastructure built for scale. AI built in from day one.

The Communications Stack for What's Next

APIs built for speed. Infrastructure built for scale. AI built in from day one.

The Communications Stack for What's Next

APIs built for speed. Infrastructure built for scale. AI built in from day one.

The Communications Stack for What's Next

APIs built for speed. Infrastructure built for scale. AI built in from day one.