Conversation analytics

Conversation analytics shows what happened during an AI conversation, how well the experience performed, and whether the user accomplished their goal. Use it to replace impressions such as “the agent felt slow” or “the call went poorly” with specific evidence you can act on.

You can identify where a response was delayed, find words the agent repeatedly misheard, understand why callers interrupted, inspect function behavior, measure outcomes, and connect resource usage to successful conversations. Comparing those signals over time helps you improve prompts, functions, voices, and recognition settings without guessing.

What you can learn

Choose the right source

SignalWire provides two independent ways to collect this information. Configure either one or both, depending on when you need the data and what you want to do with it:

Use the post-prompt report as your durable analytics record. Debug events are asynchronous, high-volume diagnostics rather than a guaranteed event store. They are not a replacement for the final report.

Post-prompt reports

A post-prompt has two parts:

• post_prompt tells the model what to extract or summarize after the conversation.
• post_prompt_url receives the complete report, including the model’s post-prompt response and the conversation telemetry assembled by the platform.

The post-prompt runs after the conversation, so it cannot change what the agent already said or did. Ask for structured JSON when you need outcomes that can be counted across conversations.

Configure a final report

1
from signalwire import AgentBase
2
3
4
class AnalyticsAgent(AgentBase):
5
    def __init__(self):
6
        super().__init__(name="bayview-analytics")
7
        self.prompt_add_section(
8
            "Role",
9
            "You are Ada, the dispatcher for Bayview Taxi. Help callers request fare quotes.",
10
        )
11
        self.set_post_prompt(
12
            "Return JSON only with these keys: intent, outcome, escalated, and sentiment. "
13
            "Set outcome to resolved, unresolved, or abandoned."
14
        )
15
        self.set_params({"enable_accounting": True})
16
17
    def on_summary(self, summary, raw_data=None):
18
        print("Outcome:", summary)
19
        print("Complete report:", raw_data)
20
21
22
if __name__ == "__main__":
23
    AnalyticsAgent().run()

The generated post-prompt callback documentation provides request examples and documents the callback’s core fields. The conversation_type field identifies voice or chat reports.

Understand outcomes

post_prompt_data contains the model’s answer to your post-prompt. Its shape depends on the output you requested and on the conversation engine, so treat the schema in your prompt as an application contract.

For aggregate reporting, ask for named fields with limited values. Useful fields include:

• intent or request category
• outcome such as resolved, unresolved, or abandoned
• whether the conversation escalated to a person
• caller sentiment
• follow-up work that remains

Voice and chat can deliver post_prompt_data in different wrappers. The Server SDK normalizes these forms before passing the result to on_summary(). If you process callbacks yourself, preserve the raw payload and handle raw, flat objects, and parsed wrappers described in the post-prompt callback reference.

The wording of the post-prompt determines whether the result is useful at scale. A request such as “summarize the conversation” produces prose for reviewing one conversation. Named fields with a small set of allowed values produce records you can group, filter, and compare. For example, once intent, outcome, and escalated are consistent, you can calculate the resolution rate for each intent without rereading transcripts.

Do not rely only on the model’s assessment for facts your application already knows. Values written by your functions appear in global_data; use those values for verified facts such as an order ID, appointment status, or transfer result.

Reconstruct the conversation

The final report provides three views of the conversation:

Dialogue entries use user and assistant roles. Other roles describe prompts, tool results, model thinking, manual speech, and platform lifecycle events. Filter by role when you need a caller-facing transcript, but retain the full logs for diagnosis.

Measure responsiveness

Voice reports expose two related views of responsiveness:

• Caller-perceived latency runs from the end of the caller’s last word to the first audio from the agent. This is the silence the caller experiences.
• Pipeline latency starts after the caller’s speech has been finalized and separates model, utterance, and audio processing.

Do not use audio_latency alone as mouth-to-ear latency. It begins after speech recognition has produced its final result, so it excludes the time spent detecting the end of the caller’s turn.

Use one value per caller turn when aggregating mouth-to-ear and turn-detection latency. A single caller turn can trigger filler speech, a function, and a final response that share the same last_word_end; only the first agent audio represents that turn’s response latency. The Post Prompt Viewer deduplicates on this anchor and can compare the reported timing with the call recording.

Track the median to understand a typical turn and the 95th percentile to expose the pauses users remember. Compare the same intent and conversation type before and after a change; one overall average can hide a slow function or a particularly verbose workflow. If the report does not contain the required timing anchor, omit that turn rather than inventing one.

Diagnose recognition

Recognition metrics apply to voice calls, not AI Chat. Each user turn can include:

• confidence, from 0 to 1, estimates how confidently speech was recognized
• content_type distinguishes detected speech from inputs such as keypad digits
• merge_count shows how many recognition segments were combined into the utterance

Confidence is most useful as a way to find patterns, not as a score to average across every call. Review the transcript around low-confidence turns and group them by repeated terms. Product names, street names, abbreviations, and account identifiers often emerge as the real problem. Add speech hints for vocabulary that is repeatedly misrecognized.

Use merge_count or timing.segments as context when a turn spans several recognition segments; those turns are not directly comparable to a simple, single-segment utterance.

Evaluate turn-taking

Recognition asks whether the platform heard the correct words. Turn-taking asks whether the agent listened and spoke at the right moments.

The explicit barge fields are carried on interrupted responses in raw_call_log and corresponding call_timeline events; they can be absent from the consolidated call_log. Use the raw entry to separate the portion the caller likely heard from the portion cut off by the interruption.

Look at the interruption rate together with response length. Frequent barges often mean the useful answer arrives too late, the response includes unnecessary detail, or the agent continues after the caller is ready to move on. Put the answer first and tighten the response guidelines in the prompt. Use short filler language only when a function genuinely takes time; AI best practices explains that pattern.

Inspect function activity

swaig_log records each SignalWire AI Gateway (SWAIG) function call, including its name, arguments, endpoint, request body, and response. Use it to answer:

• Did the agent choose the correct function?
• Were the extracted arguments correct?
• Did the endpoint return the expected response and actions?
• Did the agent repeat the same function unnecessarily?

Use swaig_log for the authoritative function name, arguments, request, and response. For timing, pair it with the corresponding role: "tool" entry and prefer execution_latency or function_latency. Do not treat the tool entry’s generic latency value as model response latency.

Aggregate runtime by function name rather than across all functions. This exposes one slow backend behind an otherwise healthy average. Also track calls per conversation: repeated calls to the same function often mean its description is too broad, its result did not give the model enough context, or its response instructed the agent to try again. See tool calling for designing clear function contracts.

Examine conversation shape

Conversation shape connects several user-experience problems. Per-response fields such as response_word_count, tokens, answer_time, and tps show how much the agent produced and how quickly. Entries with an empty response usually dispatched a function instead of speaking, so omit them from response-length averages.

These fields are stored in times[], which can also contain generations that produced a function call instead of speech. Match an entry to the spoken assistant turn by response text rather than by array position.

Long replies take longer to generate and synthesize, consume more tokens and text-to-speech characters, and create more opportunities for interruption. If response length rises, inspect the affected intents and tighten the prompt’s response guidelines instead of applying an arbitrary global word limit.

Connect usage to value

When enable_accounting is true, the final report includes totals for:

• conversation minutes
• input and output tokens
• billable wire tokens and per-minute rates
• text-to-speech characters
• speech-recognition minutes and cost factor

Compare usage with outcomes rather than optimizing it alone. A shorter prompt reduces input tokens on every turn, while shorter replies reduce output tokens, speech synthesis, latency, and interruption opportunities.

Input tokens often exceed output tokens because the prompt and accumulated conversation are sent again on later turns. Compare cost per resolved conversation, not simply cost per minute: a cheaper call that fails and requires another contact is not necessarily an improvement.

Live debug events

The debug webhook sends event objects while a voice session is running. Each request includes call_info and one or more properties named for the events in that request.

Use level 1 for routine diagnostics such as session lifecycle, errors, function activity, barges, and step changes. Level 2 also emits high-volume details such as each conversation_add and model request or response. Enable level 2 for a focused investigation rather than as a default analytics feed.

Configure debug events

1
from signalwire import AgentBase
2
3
4
agent = AgentBase(name="bayview-debug")
5
agent.prompt_add_section(
6
    "Role",
7
    "You are Ada, the dispatcher for Bayview Taxi. Help callers request fare quotes.",
8
)
9
agent.enable_debug_events(level=1)
10
11
12
@agent.on_debug_event
13
def handle_debug_event(event_type, data):
14
    print(event_type, data)
15
16
17
if __name__ == "__main__":
18
    agent.run()

The generated debug webhook documentation provides request examples and documents the event types and their fields.

Handle the stream safely

Debug events are posted asynchronously so the collector does not block the conversation. Design the collector for diagnostics:

• return a successful response quickly and process events asynchronously
• identify sessions with call_info.call_id
• handle recognized event properties and ignore unknown ones
• expect multiple event properties in one request
• avoid putting business-critical state changes behind debug delivery
• protect and redact stored event bodies when they contain conversation content

Debug events help explain a call while it is happening. The final report remains the source for a complete transcript and cross-conversation metrics.

Store and review reports

Store each final report verbatim in a JSON column and extract only the fields you query frequently. This preserves new fields without requiring an immediate database migration. Send malformed bodies to a dead-letter store instead of discarding them, and authenticate both final-report and debug collectors.

The open-source Post Prompt Viewer demonstrates this pattern. It stores the complete payload while extracting a small set of fields for search and the call index. Its views combine the transcript, event timeline, per-turn pipeline, functions, usage data, and raw report. When a recording is available, it can compare report timings with the silence measured in the audio. Point post_prompt_url at its /collect endpoint to inspect reports without building a viewer first.

Conversation reports and debug events can contain caller speech, addresses, tool arguments, and other sensitive values. Apply content redaction, restrict access, and set a retention policy appropriate for your application.

Improve the agent

Use a repeatable loop:

1. Define the outcome and performance measurements that matter.
2. Collect final reports for a representative set of conversations.
3. Segment by intent, outcome, function, or failure type.
4. Inspect the transcript and timeline behind an outlier.
5. Use debug level 2 to reproduce a live problem when the final report is not enough.
6. Change one prompt, function, or voice setting and compare the same measurements again.