Cover AI agent tool latency with fillers

AI AgentsAI agent tool latency fillers

Play localized filler phrases or a hosted wait file while a slow AI agent tool finishes.

tool-calling

The claim

While a tool runs, the model is not running, so nothing in your prompt can cover the gap. Three fields on the document do. fillers on the function is a phrase for the moment this tool is called. wait_file is audio the platform loops while the tool is still running, and this recipe renders it only when you host a file. function_fillers on each language entry is the pool the platform draws from for any tool, in that language.

Why it holds

Docs: SWAIG functions and multilingual.

How it works

The function carries its own filler and, when WAIT_FILE_URL is set, its wait file.

@AgentBase.tool(
    name="check_stock",
    description="Look up how many of a part are in stock by its SKU.",
    parameters={...},
    fillers={"en-US": ["Checking the warehouse now."]},
    wait_file="https://cdn.example.com/audio/hold.mp3",
    wait_file_loops=3,
)

The function-level fillers dict takes one language key. The 3.0.1 schema defines it as one-of-one-language, and the schema rejects a dict with two keys. Your per-language pools go on the languages.

self.add_language("Spanish", "es-ES", "rime.spore",
                  speech_fillers=["Vale."],
                  function_fillers=["Un momento, estoy comprobando."])
self.set_params({"languages_enabled": True})

Pass both speech_fillers and function_fillers. With only one, the SDK writes the deprecated single fillers key on the language, and the verifier shows it. Set languages_enabled, or the platform ignores the list.

The app does not host audio. You attach wait_file by setting WAIT_FILE_URL to a file the platform can fetch; with it unset, the function renders without either wait key.

Limitations

Fillers cover latency; they do not shorten it. Keep wait_file_loops low so the returning silence tells you something is wrong.

The verifier proves what the document says, not what a caller hears. The linked platform docs describe which phrase plays and when.

What to change first

Empty WAIT_FILE_URL and run the verifier. The wait_file keys disappear from the rendered function, which is the difference between promising audio and playing it.

Where this sits

Before this

Give a voice AI agent a tool

Switch language mid-call

Seen in a build

One of 10 recipes composed by Embed a state-aware voice agent in your web page.

python/app.py

"""Cover tool latency with fillers.

A tool that takes two seconds leaves two seconds of silence unless the
document says otherwise. Three fields on the document fill it:

  fillers on the function        a phrase for the moment this tool is called
  wait_file on the function      audio looped while the tool is still running
  function_fillers per language  the phrase pool for any tool, in that language

The function-level `fillers` dict takes one language key: the bundled 3.0.1
schema defines it as one-of-one-language. Per-language coverage therefore
belongs on the `languages` entries, which the platform reads once
`languages_enabled` is on.

Written against signalwire-sdk 3.0.1.
"""
import os
import time

from dotenv import load_dotenv
from signalwire import AgentBase, FunctionResult

# the SDK does not read .env for you
load_dotenv()

# A hosted audio file the platform can fetch. This app does not serve one, so
# the wait file is only attached when you point at a real URL.
WAIT_FILE_URL = os.getenv("WAIT_FILE_URL", "").strip()

# Stand-in for a slow inventory system.
STOCK = {"SK-2210": 14, "SK-2211": 0, "SK-4400": 3}

# The phrase pools, in one place, so the languages below and the function
# share nothing by accident.
EN_FUNCTION_FILLERS = ["One moment while I check the shelf.",
                       "Let me look that up for you."]
ES_FUNCTION_FILLERS = ["Un momento, estoy comprobando.", "Deme un segundo."]
CHECK_STOCK_FILLER = ["Checking the warehouse now."]


def wait_file_fields():
    """`wait_file` and its loop count, only when there is a file to play."""
    if not WAIT_FILE_URL:
        return {}
    return {"wait_file": WAIT_FILE_URL, "wait_file_loops": 3}


class StockAgent(AgentBase):
    def __init__(self):
        super().__init__(name="stock", route="/stock")
        self.prompt_add_section(
            "Role",
            "You check parts inventory for a bicycle shop. Look the part up "
            "before you say anything about availability.",
        )
        # One pool of function fillers per language. Pass both kinds: with
        # only one, the SDK writes the deprecated single `fillers` key.
        self.add_language(
            "English", "en-US", "rime.spore",
            speech_fillers=["Right.", "Okay."],
            function_fillers=EN_FUNCTION_FILLERS,
        )
        self.add_language(
            "Spanish", "es-ES", "rime.spore",
            speech_fillers=["Vale.", "Claro."],
            function_fillers=ES_FUNCTION_FILLERS,
        )
        # Without this the platform ignores the languages list.
        self.set_params({"languages_enabled": True})

    @AgentBase.tool(
        name="check_stock",
        description=(
            "Look up how many of a part are in stock by its SKU. Use this "
            "before answering any question about availability."
        ),
        parameters={
            "type": "object",
            "properties": {
                "sku": {
                    "type": "string",
                    "description": ("The part's SKU: two letters, a dash, "
                                    "four digits, like SK-2210."),
                }
            },
            "required": ["sku"],
        },
        # One language key. This dict is one-of-one-language in the 3.0.1
        # schema; the per-language pools live on `languages` above.
        fillers={"en-US": CHECK_STOCK_FILLER},
        **wait_file_fields(),
    )
    def check_stock(self, args, raw_data):
        sku = (args.get("sku") or "").strip().upper()
        # The slow part. Set LOOKUP_DELAY_SECONDS on a real call to hear the
        # filler and the wait file; the platform covers this gap, not the model.
        time.sleep(float(os.getenv("LOOKUP_DELAY_SECONDS", "0")))
        if sku not in STOCK:
            return FunctionResult(
                f"NOT_FOUND: no part with SKU {sku}. Ask the caller to read "
                "the SKU again, letters then numbers."
            )
        count = STOCK[sku]
        if count == 0:
            return FunctionResult(f"Part {sku} is out of stock.")
        return FunctionResult(f"Part {sku}: {count} in stock.")


agent = StockAgent()

if __name__ == "__main__":
    agent.serve(host="0.0.0.0", port=int(os.getenv("PORT", "3000")))

Run it

cd python
pip install -r requirements.txt
cp ../.env.example .env          # set SWML_BASIC_AUTH_PASSWORD and WAIT_FILE_URL
LOOKUP_DELAY_SECONDS=3 python app.py

The webhook needs a public HTTPS URL. For a local run, expose port 3000 with a tunnel such as ngrok and use that hostname. Point a number’s SWML webhook at https://<user>:<password>@<your-host>/stock/ and ask for part SK-2210. The delay lets you hear the gap being covered. Set it back to 0 afterwards.

Verify it

No network, no account.

python verify.py          # from the recipe folder, not python/

It renders the SWML with and without a wait file URL, validates both, and asserts the following.

check_stock carries the exact fillers phrase, the wait file URL unchanged, and wait_file_loops
with no WAIT_FILE_URL, neither wait file key is present
a language given only speech_fillers renders the deprecated fillers key instead
both languages carry the exact speech_fillers and function_fillers arrays, and no deprecated fillers key
languages_enabled is on
the schema rejects a two-language fillers dict on the function

Report an issue