ai_sidecar

Attaches a real-time AI observer to a live call. The sidecar listens as a third party and never speaks on the call — after each customer turn, it sends agent-facing advice to your application as webhook callbacks that an agent’s UI (or any consumer) can render. Think of it as a coach watching over the agent’s shoulder: it runs alongside the call rather than driving it.

Use it for live sales coaching, real-time compliance flagging, intent-based UI navigation, voice-of-customer signal extraction, and supervisor-on-shoulder workflows. A common pattern is to attach ai_sidecar to coach a human agent on a connect-bridged call: the customer talks to the human, and the sidecar coaches the human’s screen.

How it works

The sidecar listens to the call but never speaks on it. A call can run either the sidecar or plain live transcription, but not both at once.

Each time the customer finishes speaking, the sidecar evaluates the conversation. This evaluation is called a tick, and every callback the sidecar produces carries a tick_id. On each tick:

1. The sidecar detects the end of the customer’s turn — a final transcription result followed by a brief silence (idle_timeout_ms), or the agent starting to speak.
2. It sends the running transcript to the model, along with your operator prompt and your SWAIG tools.
3. The model returns a single line of agent-facing advice (an insight), or calls the built-in sidecar_skip tool to stay silent when no advice is needed.
4. Any tools the model calls (lookups, alerts, intent triggers) run through your SWAIG functions and MCP servers.
5. Every step is reported as a structured callback your application can consume. See Webhook callbacks below for the full catalog.

Properties

Webhook callbacks

As it observes the call, the sidecar reports each step of its activity as a structured callback on two paths:

• Relay topic calling.ai.sidecar — always fires. Subscribe with the SignalWire RELAY or browser SDK to consume callbacks in real time.
• Webhook — fires only when url is set, as an HTTP POST to that URL with the callback body wrapped under sidecar_event.

Both paths carry the same payload, and each callback fires exactly once on the relay topic regardless of whether a webhook is configured — the relay always fires, the webhook is opt-in.

Callback body shape

Each callback is POSTed to your url with the event wrapped under sidecar_event, alongside a call_info envelope describing the call. project_id and space_id are included when available:

1
{
2
  "call_info": {
3
    "project_id": "...",
4
    "space_id": "...",
5
    "call_id": "...",
6
    "content_type": "text/json",
7
    "content_disposition": "post_data",
8
    "conversation_type": "voice"
9
  },
10
  "sidecar_event": {
11
    "type": "insight",
12
    "ts": 1745870400123456,
13
    "tick_id": 7,
14
    "channel_data": { },
15
    "raw": "Confirm the customer's address.",
16
    "iter": 0,
17
    "total_iters": 1
18
  }
19
}

The actual event is under sidecar_event — unwrap that in your code before reading type and the event’s fields. Every callback also carries type, ts (microsecond timestamp), tick_id, and a channel_data object (call_id, plus caller_id_name / caller_id_number / destination_number when available).

1
{
2
  "call_info": {
3
    "call_id": "2e1e66e5-5d07-413d-9668-55542992eec0",
4
    "content_type": "text/json",
5
    "content_disposition": "post_data",
6
    "conversation_type": "voice"
7
  },
8
  "sidecar_event": {
9
    "type": "insight",
10
    "ts": 1745870400123456,
11
    "tick_id": 7,
12
    "channel_data": {}
13
  }
14
}

See the AI sidecar callback webhook page for the full payload reference. The per-type fields are listed below.

Callback types

Each callback carries a type field identifying what occurred, along with type-specific fields.

On the callbacks that come from a single evaluation (request, thought, tool_call, tool_result, insight), iter counts the steps within that evaluation, starting at 0, and total_iters on the insight is how many steps it took in total. Any callback produced while answering an ai_sidecar.ask also carries that ask’s ask_id and triggered_by: "ask", so you can match it to the question you sent.

Stop reasons

The final callback carries a stop_reason indicating why the sidecar stopped.

Error reasons

The error callback carries an error_reason describing the failure.

Tool webhook contract

When the model calls one of your SWAIG functions, the platform sends an HTTP POST to that function’s web_hook_url. Your webhook runs the function and returns the result. Both the request and the response are plain JSON.

Request

The platform sends the POST body shown below. The sidecar sends a smaller set of fields than the regular ai method’s SWAIG webhook, and it groups the caller details under channel_data.

1
{
2
  "function": "lookup_competitor",
3
  "argument": {
4
    "parsed": [
5
      {
6
        "competitor": "ACME"
7
      }
8
    ],
9
    "raw": "{\"competitor\":\"ACME\"}",
10
    "substituted": "{\"competitor\":\"ACME\"}"
11
  },
12
  "call_id": "2e1e66e5-5d07-413d-9668-55542992eec0",
13
  "channel_data": {
14
    "call_id": "2e1e66e5-5d07-413d-9668-55542992eec0"
15
  }
16
}

See the AI sidecar SWAIG tool webhook webhook page for the full field reference.

Response

Your webhook must return a JSON object with the result and any actions to take:

1
{
2
  "response": "ACME charges $99/seat. We're $79.",
3
  "action": [
4
    { "user_event": { "topic": "sidecar.alert", "level": "info" } },
5
    { "set_global_data": { "last_lookup": "ACME" } }
6
  ]
7
}

Supported SWAIG actions

Each entry in the action array is an object keyed by the action name. The sidecar supports the following:

Built-in sidecar_skip tool

The sidecar provides one built-in tool, sidecar_skip, that you do not declare. The model calls it when the latest customer turn needs no advice. The tick then ends silently — no insight callback fires — and a skip callback is emitted with the model’s reason.

Instruct the model in your prompt to call sidecar_skip when there is nothing useful to say — otherwise it will tend to fill silence with low-quality advice. The name sidecar_skip is reserved; do not define a function with that name.

Limitations

• One transcriber per call. ai_sidecar and plain live_transcribe cannot run on the same call at the same time — starting one while the other is active is rejected.
• Both legs required. A single-leg direction (for example, only remote-caller) is rejected.
• Turn detection is tuned for Deepgram. deepgram is the default speech engine, and turn-end detection is calibrated for it.
• say doesn’t speak. The sidecar has no voice on the call, so a say action is reported as a callback instead of being spoken aloud.
• The sidecar stops with the call. When the call hangs up or transcription stops, the sidecar’s final callback fires and it stops.
• sidecar_skip is reserved. Do not define a function with that name.
• Tool loops are capped. If the model calls tools in a runaway loop — too many calls in one tick, the same call repeated, or a tool that isn’t registered — the sidecar stops calling tools for that tick and returns an error callback with error_reason: tool_loop. Make sure your tools return clear, useful results and your prompt tells the model what to do after a tool returns.

Example

Answer the call, attach the sidecar with a prompt, language, webhook URL, hints, and a single SWAIG function, then bridge the call with connect and hang up.

1
version: 1.0.0
2
sections:
3
  main:
4
    - answer: {}
5
    - ai_sidecar:
6
        prompt: "You are a real-time sales copilot. After each customer turn, give the agent one concise piece of advice or call sidecar_skip if no advice is needed."
7
        lang: "en-US"
8
        url: "https://your-app.example.com/sidecar/events"
9
        hints: ["ACME", "Globex", "FedRAMP", "SOC 2"]
10
        SWAIG:
11
          defaults:
12
            web_hook_url: "https://your-app.example.com/sidecar/swaig"
13
          functions:
14
            - function: lookup_account
15
              description: "Look up an account record."
16
              parameters:
17
                type: object
18
                properties:
19
                  customer_id:
20
                    type: string
21
                required:
22
                  - customer_id
23
    - connect:
24
        from: "+15555550100"
25
        to: "+15555550199"
26
        answer_on_bridge: true
27
    - hangup: {}