Start live transcription and consume the webhook

Voicereal-time call transcription

Turn on transcription for a call and receive the text as it is spoken.

transcription

The claim

live_transcribe is a call-level verb, independent of any AI agent. Started before connect, it transcribes both legs of the bridged conversation. It POSTs partial events while live_events is on, and a final event per utterance carrying the leg it came from. When the session ends, ai_summary adds an AI-written summary. Stopping (action: stop) and summarising on demand (action: summarize) are the same verb.

How it works

- answer: {}
- live_transcribe:
    action:
      start:
        webhook: "https://<your-host>/transcript"
        lang: en
        live_events: true
        ai_summary: true
        direction: [remote-caller, local-caller]
        speech_engine: deepgram
- connect: { to: "+1555..." }

The Flask route at /transcript keeps finals per call (with the leg), keeps the summary, and drops partials, where a caption display would render them instead. The same session can be started on a live call over REST (calling.live_transcribe) or from RELAY.

A call has one transcriber, so live_transcribe cannot run alongside ai_sidecar. For a whole-call transcript delivered once at the end rather than a stream, use transcribe-a-call-in-the-background.

What to change first

Push the finals to a browser over WebSocket for live captions. Alternatively, feed them to classify-sentiment-style logic to raise a supervisor alert mid-call.

Where this sits

Before this

Answer an inbound call

Record a call Listen to a live call

Extract structured data after a call

python/app.py

"""Start live transcription.

`live_transcribe` starts a transcription session on the call and POSTs events to
a webhook: partial results as they occur (live_events), final utterances per
leg, and an AI summary when the session ends (ai_summary). The Flask route below
receives them and keeps finals and the summary; partials are printed and
dropped.

Written against signalwire-sdk 3.0.1 (SWMLService) and Flask.
"""
import os

from dotenv import load_dotenv
from flask import Flask, jsonify, request
from signalwire import SWMLService

# the SDK does not read .env for you
load_dotenv()

PUBLIC_URL = os.getenv("PUBLIC_URL", "https://your-host.example.com")
AGENT = os.getenv("AGENT_NUMBER", "+15550100001")
LANG = os.getenv("TRANSCRIBE_LANG", "en")

app = Flask(__name__)


def build(service=None):
    service = service or SWMLService(name="transcribe", route="/transcribe")
    service.reset_document()
    service.add_verb("answer", {})
    service.add_verb("live_transcribe", {"action": {"start": {
        "webhook": f"{PUBLIC_URL}/transcript",
        "lang": LANG,
        "live_events": True,       # partials as they occur
        "ai_summary": True,        # a summary when the session ends
        "direction": ["remote-caller", "local-caller"],
        "speech_engine": "deepgram",
    }}})
    service.add_verb("connect", {"to": AGENT})
    service.add_verb("hangup", {})
    return service


@app.route("/transcribe", methods=["GET", "POST"])
def swml():
    return jsonify(build().get_document())


transcripts = {}   # call_id -> [{"who", "text"}]
summaries = {}     # call_id -> text


def classify(event):
    """Return ('partial'|'final'|'summary'|'other', payload)."""
    if event.get("summary") or event.get("type") == "summary":
        return "summary", event
    if event.get("type") == "partial" or event.get("partial") is True:
        return "partial", event
    if event.get("text") or event.get("transcript"):
        return "final", event
    return "other", event


@app.post("/transcript")
def transcript():
    event = request.get_json(force=True, silent=True) or {}
    kind, e = classify(event)
    call_id = e.get("call_id", "unknown")
    if kind == "final":
        transcripts.setdefault(call_id, []).append(
            {"who": e.get("direction") or e.get("channel", "?"),
             "text": e.get("text") or e.get("transcript")})
    elif kind == "summary":
        summaries[call_id] = e.get("summary") or e.get("text")
    else:
        print("partial:", e.get("text") or e.get("transcript"))
    return "", 204


if __name__ == "__main__":
    app.run(port=int(os.getenv("PORT", "8080")))

# Start live transcription - partial and final transcripts of both legs POSTed
# to your webhook while the call is up, and an AI summary when it ends.
# live_transcribe starts before connect so the bridged conversation is covered.
version: "1.0.0"
sections:
  main:
    - answer: {}
    - live_transcribe:
        action:
          start:
            webhook: "https://your-host.example.com/transcript"
            lang: en
            live_events: true
            ai_summary: true
            direction:
              - remote-caller
              - local-caller
            speech_engine: deepgram
    - connect: { to: "+15550100001" }
    - hangup: {}

Run it

Markup only: paste swml/agent.yaml into a SWML Script, set webhook and the destination number, assign a phone number.

Python:

cd python
pip install -r requirements.txt
PUBLIC_URL=https://<your-host> AGENT_NUMBER=+1555... python app.py

Point a phone number’s SWML webhook at https://<your-host>/transcribe.

Verify it

python verify.py          # from the recipe folder, not python/

Both surfaces validate against the SWML schema. The verifier asserts live_transcribe precedes connect with live_events, ai_summary and both directions on, and that the webhook keeps finals and the summary while dropping partials.

Report an issue