Collect speech input and branch on it

Voicespeech recognition IVR

Ask an open question, recognise the answer, and take a different path for each.

stt

The claim

A menu the caller talks to instead of pressing. A recogniser turns the utterance into a word, a switch picks the section, and the call is transferred. There is no ai verb, no prompt to write and no model in the path.

Why it holds

Speech input is not the same thing as an AI agent. This is the cheap version: fixed vocabulary, deterministic routing, and nothing to govern.

How it works

prompt plays a question and waits. What it listens for is decided by which parameters you set, and the schema is precise about it. Speech detection is off unless a speech parameter is present. Digit detection is off if only speech parameters are present.

- prompt:
    play: "say:Are you calling about sales, support, or your account?"
    speech_timeout: 12
    speech_end_timeout: 1.5
    speech_language: en-US
    speech_hints: [sales, support, account, billing]

No max_digits, so the keypad is never armed. Setting one would arm both.

speech_hints is the vocabulary. The recogniser is told what it is likely to hear, which is the difference between matching “billing” and matching “Bill Ng”.

The branch reads prompt_value, which holds what the caller said. prompt_result is a different variable, carrying the status of the attempt: values like match_speech or no_input. Branch on the status instead of the content and the menu stops working.

- switch:
    variable: prompt_value
    case:
      sales:   [{ transfer: { dest: sales } }]
      account: [{ transfer: { dest: billing } }]

Two phrases can share a destination. “Account” and “billing” both reach the billing team. That is why the Python surface keeps a table and builds the cases from it, rather than repeating YAML.

Limitations

The vocabulary is fixed. A caller who says “I need to pay my invoice” matches nothing and is asked again. That is what you give up by having no model in the path.

Recognition is not exact. A hint improves the odds and does not guarantee them, so the default branch is load bearing rather than decorative.

What to change first

Add max_digits: 1 to the prompt and a "1" case beside sales. Both detectors arm, and the same switch serves both, because a digit arrives in prompt_value exactly as a word does. Add the digit without the case and every keypress falls to the default.

python/app.py

"""Collect speech input and branch on it.

`prompt` asks a question and waits. What it listens for depends on which
parameters are set: speech detection turns on only when a speech parameter is
present, and digit detection turns on only when a digit parameter is. Set just
the speech ones and the keypad stays disarmed.

What the caller said arrives in `prompt_value`. `prompt_result` is the status
of the attempt, not its content, so the branch reads the first.

There is no AI agent here. A recogniser matches against hints and a `switch`
picks the section.

Written against signalwire-sdk 3.0.1 (SWMLService) and Flask.
"""
import os

from dotenv import load_dotenv
from flask import Flask, jsonify
from signalwire import SWMLService

# the SDK does not read .env for you
load_dotenv()

# Destination per recognised phrase. Two phrases reach billing, which is the
# reason this is a table rather than three lines of YAML.
ROUTES = {
    "sales": ("sales", os.getenv("SALES_NUMBER", "+15550100001")),
    "support": ("support", os.getenv("SUPPORT_NUMBER", "+15550100002")),
    "account": ("billing", os.getenv("BILLING_NUMBER", "+15550100003")),
    "billing": ("billing", os.getenv("BILLING_NUMBER", "+15550100003")),
}

app = Flask(__name__)


def build(service=None):
    service = service or SWMLService(name="menu", route="/menu")
    service.reset_document()
    service.add_verb("answer", {})
    service.add_verb("prompt", {
        "play": "say:Are you calling about sales, support, or your account?",
        # speech parameters only, so digit collection stays off
        "speech_timeout": 12,
        "speech_end_timeout": 1.5,
        "speech_language": "en-US",
        "speech_hints": sorted(ROUTES),
    })
    service.add_verb("switch", {
        # what the caller said, not whether the attempt succeeded
        "variable": "prompt_value",
        "case": {
            phrase: [{"transfer": {"dest": section}}]
            for phrase, (section, _) in ROUTES.items()
        },
        "default": [
            {"play": {"url": "say:Sorry, I did not catch that."}},
            {"transfer": {"dest": "main"}},
        ],
    })
    for section, number in {v for v in ROUTES.values()}:
        service.add_section(section)
        service.add_verb_to_section(section, "play", {
            "url": f"say:Connecting you to {section}."})
        service.add_verb_to_section(section, "connect", {"to": number})
        service.add_verb_to_section(section, "hangup", {})
    return service


@app.route("/menu", methods=["GET", "POST"])
def swml():
    return jsonify(build().get_document())


if __name__ == "__main__":
    app.run(port=int(os.getenv("PORT", "8080")))

# Branch on what the caller says, with no AI agent in the document.
#
# `prompt` plays a question and waits. Speech detection is not enabled unless at
# least one speech parameter is set, and setting only speech parameters leaves
# digit detection off, so this document listens for words and nothing else.
version: "1.0.0"
sections:
  main:
    - answer: {}
    - prompt:
        play: "say:Are you calling about sales, support, or your account?"
        # speech parameters only: no max_digits, so the keypad is not armed
        speech_timeout: 12
        speech_end_timeout: 1.5
        speech_language: en-US
        # the words the recogniser should expect to hear
        speech_hints:
          - sales
          - support
          - account
          - billing
    - switch:
        # prompt_value is what the caller said; prompt_result is the status
        variable: prompt_value
        case:
          sales:
            - transfer: { dest: sales }
          support:
            - transfer: { dest: support }
          account:
            - transfer: { dest: billing }
          billing:
            - transfer: { dest: billing }
        default:
          - play: { url: "say:Sorry, I did not catch that." }
          - transfer: { dest: main }
  sales:
    - play: { url: "say:Connecting you to sales." }
    - connect: { to: "+15550100001" }
    - hangup: {}
  support:
    - play: { url: "say:Connecting you to support." }
    - connect: { to: "+15550100002" }
    - hangup: {}
  billing:
    - play: { url: "say:Connecting you to the billing team." }
    - connect: { to: "+15550100003" }
    - hangup: {}

Run it

cd python
pip install -r requirements.txt
cp ../.env.example .env
python app.py

Point a phone number’s SWML webhook at https://<your-host>/menu. The SWML surface can be pasted into a hosted SWML Script instead.

Verify it

No network, no account:

python verify.py          # from the recipe folder, not python/

Both surfaces validate against the SWML schema. The verifier asserts:

the verbs open answer, prompt, switch, and no section contains an ai verb
the prompt sets speech parameters and no digit parameters
the switch reads prompt_value, not prompt_result
every case and the default target a section that exists
every hint given to the recogniser is a phrase the switch can act on
two phrases share the billing destination, and three destinations remain

Report an issue