Give a voice AI agent a tool

AI Agentstool calling for voice AI

Let the model call your function, and decide what it gets back.

Also called function calling, SWAIG function, AI agent webhook tool

tool-calling

The claim

A decorated Python function, or a defineTool call in TypeScript, becomes something the model can call mid-call. You declare the arguments as JSON Schema. The SDK renders the function into the tool schema the model already understands, and your handler runs when it is called. The model asks; your code answers.

Why it holds

A SWAIG function is not a different thing from an LLM tool. The document holds a SWAIG definition, and the platform renders it into the tool schema the model reads on every turn.

How it works

@AgentBase.tool takes the name the model emits, the description it reads, and parameters as a JSON Schema dict. There is no AgentBase.parameter() helper.

@AgentBase.tool(
    name="get_order_status",
    description="Look up the delivery status of a customer's order by its "
                "order number. Use this BEFORE stating any status or date.",
    parameters={"type": "object",
                "properties": {"order_id": {"type": "string",
                                            "description": "Exactly five digits."}},
                "required": ["order_id"]},
    fillers={"en-US": ["Let me pull that order up."]},
)
def get_order_status(self, args, raw_data):
    ...

Both description fields are prompt engineering, not developer documentation. The model reads them to decide when to call the tool and how to fill the argument. A vague description is the usual reason a model has the right tool and never calls it.

The handler returns a FunctionResult, never a raw string. On a miss it returns a typed state, NOT_FOUND, with an instruction for what to do next. Without that, an unanswered lookup invites the model to fill the silence with a delivery date it invented.

The spoken sentence is built in the handler. Formatting for voice belongs in the return value, not in a prompt bullet asking the model to say it nicely.

Limitations

The SDK puts tool-selection accuracy as degrading past roughly seven or eight simultaneously active tools (core/mixins/tool_mixin.py). When an agent grows past that, scope them per step rather than adding another (scope-tools-per-step).

secure=True is the default, so the callback requires a SWAIG token. Leave it on for anything touching a real customer record.

The platform posts a tool’s arguments as argument: {parsed: [args], raw}, the documented SWAIG tool webhook. The TypeScript SDK’s /swaig route in 2.0.5 hands argument to the handler as posted, so a handler reading order_id would find parsed and raw instead. The surface overrides onFunctionCall to unwrap the documented shape before dispatch, and the verifier posts that shape. Remove the override and the known order comes back NOT_FOUND.

What to change first

Replace the description with “Lookup function” and call the agent. The tool is still there and the model stops choosing it.

Where this sits

python/app.py

"""Give an agent a tool.

A SWAIG function is not a separate concept from an LLM tool. It is rendered into
the same OpenAI-format tool schema the model sees every turn, so `name`,
`description` and the per-parameter descriptions are what decide whether the
model calls it and how it fills the arguments.

The handler returns a FunctionResult, never a raw string, and returns a typed
state when the lookup fails so the model is told what to do instead of guessing.

Written against signalwire-sdk 3.0.1.
"""
import os

from dotenv import load_dotenv
from signalwire import AgentBase, FunctionResult

# the SDK does not read .env for you
load_dotenv()

# Your system of record. A real one is a database call.
ORDERS = {
    "48815": {"status": "out for delivery", "eta": "today before 8pm",
              "carrier": "Britewave"},
    "48816": {"status": "packed", "eta": "ships tomorrow",
              "carrier": "Britewave"},
}


class OrderAgent(AgentBase):
    def __init__(self):
        super().__init__(name="orders", route="/orders")
        self.prompt_add_section(
            "Role",
            "You answer questions about orders for a home goods retailer. "
            "Look the order up before you say anything about its status. "
            "If the tool says the order was not found, ask the caller to "
            "read the number back to you.",
        )

    # The decorator registers the function. `parameters` is a JSON Schema
    # dict; there is no AgentBase.parameter() helper.
    @AgentBase.tool(
        name="get_order_status",
        description=(
            "Look up the delivery status of a customer's order by its order "
            "number. Use this BEFORE stating any status, date or carrier. "
            "Do not use it for returns or refunds."
        ),
        parameters={
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": (
                        "The order number, exactly five digits, no letters "
                        "or dashes. Ask the caller to read it out if they "
                        "have not given it."
                    ),
                }
            },
            "required": ["order_id"],
        },
        # the caller hears one of these while the lookup runs
        fillers={"en-US": ["Let me pull that order up.",
                           "Checking on that order now."]},
    )
    def get_order_status(self, args, raw_data):
        order_id = (args.get("order_id") or "").strip()
        order = ORDERS.get(order_id)
        if not order:
            # A typed failure. The model is told what to do next, so it does
            # not invent a delivery date to fill the silence.
            return FunctionResult(
                f"NOT_FOUND: no order {order_id}. Ask the caller to read the "
                "five digit order number back, then try again."
            )
        # Format for speech here, not in the prompt.
        return FunctionResult(
            f"Order {order_id} is {order['status']}, arriving {order['eta']} "
            f"with {order['carrier']}."
        )


agent = OrderAgent()

if __name__ == "__main__":
    agent.serve(host="0.0.0.0", port=int(os.getenv("PORT", "3000")))

/**
 * Give an agent a tool.
 *
 * A SWAIG function is not a separate concept from an LLM tool. It is rendered
 * into the same OpenAI-format tool schema the model sees every turn, so `name`,
 * `description` and the per-parameter descriptions are what decide whether the
 * model calls it and how it fills the arguments.
 *
 * The handler returns a FunctionResult, never a raw string, and returns a typed
 * state when the lookup fails so the model is told what to do instead of
 * guessing.
 *
 * Written against @signalwire/sdk 2.0.5.
 *
 *     npm start            # serves /orders/ and /orders/swaig
 */
import "dotenv/config";
import { AgentBase, FunctionResult } from "@signalwire/sdk";

// Your system of record. A real one is a database call.
export const ORDERS: Record<string, { status: string; eta: string; carrier: string }> = {
  "48815": { status: "out for delivery", eta: "today before 8pm", carrier: "Britewave" },
  "48816": { status: "packed", eta: "ships tomorrow", carrier: "Britewave" },
};

type Json = Record<string, unknown>;
type Hook = Promise<Json> | undefined;

export class OrderAgent extends AgentBase {
  constructor() {
    super({ name: "orders", route: "/orders" });
    this.promptAddSection("Role", {
      body: "You answer questions about orders for a home goods retailer. "
          + "Look the order up before you say anything about its status. "
          + "If the tool says the order was not found, ask the caller to "
          + "read the number back to you.",
    });
    // `parameters` is a JSON Schema object, the same shape the model sees.
    this.defineTool({
      name: "get_order_status",
      description: "Look up the delivery status of a customer's order by its order "
                 + "number. Use this BEFORE stating any status, date or carrier. "
                 + "Do not use it for returns or refunds.",
      parameters: {
        type: "object",
        properties: {
          order_id: {
            type: "string",
            description: "The order number, exactly five digits, no letters or "
                       + "dashes. Ask the caller to read it out if they have not "
                       + "given it.",
          },
        },
        required: ["order_id"],
      },
      // the caller hears one of these while the lookup runs
      fillers: { "en-US": ["Let me pull that order up.", "Checking on that order now."] },
      handler: (args) => {
        const orderId = String(args["order_id"] ?? "").trim();
        const order = ORDERS[orderId];
        if (!order) {
          // A typed failure. The model is told what to do next, so it does
          // not invent a delivery date to fill the silence.
          return new FunctionResult(
            `NOT_FOUND: no order ${orderId}. Ask the caller to read the `
            + "five digit order number back, then try again.");
        }
        // Format for speech here, not in the prompt.
        return new FunctionResult(
          `Order ${orderId} is ${order.status}, arriving ${order.eta} `
          + `with ${order.carrier}.`);
      },
    });
  }

  /**
   * The platform posts a tool's arguments as `argument: {parsed: [args], raw}`
   * (the documented SWAIG tool webhook). The 2.0.5 `/swaig` route hands
   * `argument` to the handler as posted, so a handler reading
   * `args["order_id"]` would find `parsed` and `raw` instead. This hook
   * unwraps the documented shape before dispatch and steps aside otherwise.
   */
  override onFunctionCall(name: string, args: Json, raw: Json): Hook {
    const parsed = args["parsed"];
    if (!Array.isArray(parsed)) return undefined;
    const tool = this.getTools().find((t) => t.name === name);
    return tool ? tool.execute((parsed[0] ?? {}) as Json, raw) : undefined;
  }
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].replace(/\\/g, "/"))) {
  for (const name of ["SWML_BASIC_AUTH_USER", "SWML_BASIC_AUTH_PASSWORD"]) {
    if (!process.env[name]) throw new Error(`${name} is required; see .env.example`);
  }
  new OrderAgent().serve({ port: Number(process.env["PORT"] ?? 3000) });
}

Run it

cd python
pip install -r requirements.txt
cp ../.env.example .env          # set SWML_BASIC_AUTH_PASSWORD
python app.py

Point a phone number’s SWML webhook at https://<user>:<password>@<your-host>/orders/, using the credentials you set. Without them the request is refused.

The TypeScript surface is the same agent with defineTool on @signalwire/sdk, on Node 20.18.1 or newer:

cd typescript
npm ci
cp ../.env.example .env
npm start            # serves /orders/ and /orders/swaig

Verify it

No network, no account:

python verify.py          # from the recipe folder, not python/

It renders the SWML and asserts:

get_order_status appears in ai.SWAIG.functions with order_id required
the function and its parameter both carry a description the model can act on
the fillers reach the platform
a known order returns the status, date and carrier as one spoken sentence
an unknown order returns NOT_FOUND and no date from any real order
the TypeScript surface renders the same function definition, and its own /swaig route, posted the documented argument shape, returns the same two sentences and refuses a request without credentials

Report an issue

Give a voice AI agent a tool

The claim

Why it holds

How it works

Limitations

What to change first

Where this sits

Seen in a build

Seen in a build

Seen in a build