Run an SMS survey over several messages

MessagingSMS survey (text message survey)

Text a customer a question, keep track of where each number is in the survey, ask the next question when a reply arrives, re-ask when it does not fit, and stop the moment they say STOP.

webhooksstateopt-out

The claim

Every inbound text is a fresh webhook with no memory of the last one. A survey is therefore a small state machine of your own. The sender’s number is the key, the step is the value, and the reply you return is the next question. This recipe keeps that state in a file, advances it one reply at a time, and answers with a messaging SWML reply.

Why it holds

Four things the handler does that a survey needs.

An answer that does not fit the question is re-asked, and the step does not move.
The last answer gets a closing line, and any text after that gets an empty document: nothing sent, nothing recorded.
STOP, or any of the other stop words, marks the number and ends the survey. A number that has stopped is never texted a first question, and the refusal happens before any request is made.
The webhook checks the platform’s signature before it touches state, because an unsigned POST could otherwise fill your survey with anything.
A webhook delivered twice gets the same reply twice and moves nothing, even when the retry is late or the survey is already complete. The record keeps every message_id it acted on, with the reply it gave. Without that, a retried rating would be parsed at the comment step and stored as the comment. A retry that arrives after STOP gets silence, not the cached question.
/begin sends texts at your expense, so it is behind a key the server holds and your systems present as X-Survey-Key. The public internet gets a 403.

The vendored REST spec documents the inbound payload as {message, vars, params}, where message carries from, to and a body that may be null. The first question goes out as POST /api/messaging/messages, which requires to and from.

How it works

def handle_inbound(message):
    sender = message["from"]
    state = _load()
    record = state.get(sender)
    word = keyword(message.get("body"))
    if word in STOP_WORDS:
        state[sender] = {**(record or {"step": 0, "answers": {}}), "stopped": True}
        _save(state)
        return reply(STOPPED)
    if not record or record["stopped"] or record["step"] >= len(QUESTIONS):
        return silence()
    key, _, kind = QUESTIONS[record["step"]]
    answer = parse(kind, message.get("body"))
    if answer is None:
        return reply(REASK.get(kind, QUESTIONS[record["step"]][1]))
    record["answers"][key] = answer
    record["step"] += 1
    _save(state)
    return reply(DONE if record["step"] == len(QUESTIONS) else QUESTIONS[record["step"]][1])

What the webhook returns after a valid first answer:

{"version": "1.0.0",
 "sections": {"main": [{"reply": {"body": "Would you recommend us to a friend? Reply YES or NO."}}]}}

parse is where the survey stops being a chat. A rating is one of five digits after trim; yes or no accepts y and n; the comment accepts anything and treats SKIP as an empty answer. Anything else returns None, and the same question comes back.

The state is a JSON file keyed by number, written through a rename so a crash mid-write leaves the old file intact. In your app that is a table with the number as the key. The two documented commands, the server and begin, are two processes, which is why the state cannot live in a dictionary.

The TypeScript surface is the same handler on @signalwire/sdk, with a small node:http server for the two routes.

Limitations

The verifier proves the handler and the requests, not delivery. Whether a question reaches the phone, and how long a customer takes to answer, are live.

The state file is a stand-in. Two server processes sharing one file would race; a database with the number as the key is the real version.

A survey texts people who did not text first. Consent for that is yours to collect and record before begin, and the recipe does not claim otherwise.

What to change first

Change "4" in the verifier’s second turn to "four" and run it. The re-ask comes back instead of the next question and the step stays put. That is the point: the survey advances only on an answer it can store.

Where this sits

Before this

Reply to an inbound SMS Handle SMS STOP and START in your own code

Register a 10DLC brand and campaign Verify a webhook signature Send an SMS Keep per-call state server-side, keyed by call id

python/app.py

"""Run an SMS survey over several messages.

A survey is one outbound text and then a conversation the platform does not
hold for you. Each reply arrives at your inbound webhook on its own, so the
state that says which question a number is on lives in a file here, keyed by
the sender. The webhook answers with a messaging SWML `reply` carrying the next
question, a re-ask for an answer that does not parse, or the closing line.

STOP ends everything for that number, and a number that has stopped is never
sent a first question. Only SignalWire may post to the webhook: the signature
is checked before any state changes.

Written against signalwire-sdk 3.0.1 (RestClient) and the documented inbound
message webhook and messaging SWML.

    python app.py                       # serve /inbound and /begin
    python app.py begin +14155550123    # text the first question
"""
import hashlib
import hmac
import json
import os
import sys
from pathlib import Path

from dotenv import load_dotenv
from flask import Flask, abort, jsonify, request
from signalwire.rest import RestClient

# the SDK does not read .env for you
load_dotenv()

client = RestClient()
# 3.0.1 has no messaging namespace; the send goes through the shared HttpClient
http = client._http

FROM = os.getenv("SMS_FROM")
SIGNING_KEY = os.getenv("SIGNALWIRE_SIGNING_KEY")
INBOUND_URL = os.getenv("INBOUND_URL")
# /begin sends billable texts with your credentials, so it is behind a key the
# server holds and your own systems present as X-Survey-Key
ADMIN_KEY = os.getenv("SURVEY_ADMIN_KEY")

# where each number's progress lives; swap for your database
STATE_PATH = Path(os.getenv("SURVEY_STATE_PATH", "survey-state.json"))

# the platform signs its webhooks: hex(HMAC(signing_key, url + raw_body)), SHA-256
# on call requests, SHA-1 on every signed request (docs/swml/guides/webhook-security)
DIGESTS = {"X-Signalwire-SHA256-Signature": hashlib.sha256,
           "X-Signalwire-Signature": hashlib.sha1}

# the questions, in order: a key for the answer, the text, and how to read a reply
QUESTIONS = [
    ("rating", "Thanks for visiting Ridgeline Cycles. How was your service today? "
               "Reply with a number from 1 to 5.", "scale"),
    ("recommend", "Would you recommend us to a friend? Reply YES or NO.", "yes_no"),
    ("comment", "Anything we should know? Reply with a sentence, or SKIP.", "text"),
]
DONE = "That is everything. Thank you. Reply STOP at any time to opt out."
REASK = {"scale": "Please reply with a single number from 1 to 5.",
         "yes_no": "Please reply YES or NO."}

# the single words that end the survey, compared whole after trim and lowercase
STOP_WORDS = {"stop", "stopall", "unsubscribe", "cancel", "end", "quit"}
STOPPED = "You will receive no more messages from Ridgeline Cycles."


class OptedOut(Exception):
    """Raised before any request is made."""


def _load():
    if not STATE_PATH.exists():
        return {}
    return json.loads(STATE_PATH.read_text(encoding="utf-8"))


def _save(state):
    tmp = STATE_PATH.with_suffix(".tmp")
    tmp.write_text(json.dumps(state, indent=2), encoding="utf-8")
    os.replace(tmp, STATE_PATH)


def keyword(body):
    return (body or "").strip().lower()


def parse(kind, body):
    """The answer a reply means, or None when it does not fit the question."""
    word = keyword(body)
    if kind == "scale":
        return int(word) if word in {"1", "2", "3", "4", "5"} else None
    if kind == "yes_no":
        return {"yes": True, "y": True, "no": False, "n": False}.get(word)
    return "" if word == "skip" else (body or "").strip() or None


def reply(text):
    """A messaging SWML document with one verb: the text back to the sender."""
    return {"version": "1.0.0", "sections": {"main": [{"reply": {"body": text}}]}}


def silence():
    return {"version": "1.0.0", "sections": {"main": []}}


def begin(to):
    """Text the first question. A number that said STOP is refused, not texted."""
    state = _load()
    record = state.get(to, {})
    if record.get("stopped"):
        raise OptedOut(f"{to} opted out; no message sent")
    state[to] = {"step": 0, "answers": {}, "stopped": False}
    _save(state)
    body = {"to": to, "from": FROM, "body": QUESTIONS[0][1]}
    return http.post("/api/messaging/messages", body=body)


def handle_inbound(message):
    """Advance one number's survey by one reply. Returns the SWML to run."""
    sender = message["from"]
    state = _load()
    record = state.get(sender)
    word = keyword(message.get("body"))

    if word in STOP_WORDS:
        state[sender] = {**(record or {"step": 0, "answers": {}}), "stopped": True}
        _save(state)
        return reply(STOPPED)
    if not record or record["stopped"]:
        # not in a survey, or out of it: nothing is sent and nothing is
        # recorded. This runs before the cache, or a late retry of an earlier
        # answer would text a question to a number that said STOP
        return silence()
    seen = record.get("seen", {})
    if message.get("message_id") in seen:
        # the webhook was delivered again, maybe late: the answer it got the
        # first time, and the state is not touched. A retried rating parsed at
        # the comment step would otherwise be stored as the comment
        return reply(seen[message["message_id"]])
    if record["step"] >= len(QUESTIONS):
        return silence()

    key, _, kind = QUESTIONS[record["step"]]
    answer = parse(kind, message.get("body"))
    if answer is None:
        return reply(REASK.get(kind, QUESTIONS[record["step"]][1]))

    record["answers"][key] = answer
    record["step"] += 1
    text = DONE if record["step"] == len(QUESTIONS) else QUESTIONS[record["step"]][1]
    record.setdefault("seen", {})[message["message_id"]] = text
    _save(state)
    return reply(text)


def signed(headers, url, raw_body, key=None):
    """True only when a signature header is present and matches."""
    key = key or SIGNING_KEY
    for header, digest in DIGESTS.items():
        sent = headers.get(header)
        if sent:
            expected = hmac.new(key.encode(), url.encode() + raw_body, digest).hexdigest()
            return hmac.compare_digest(sent, expected)
    return False


app = Flask(__name__)


@app.before_request
def gate():
    """Only SignalWire may advance a survey."""
    if request.path == "/inbound":
        url = INBOUND_URL
        if request.query_string:
            url += "?" + request.query_string.decode()
        if not signed(request.headers, url, request.get_data()):
            abort(403)


@app.post("/inbound")
def inbound():
    payload = request.get_json(force=True)
    return jsonify(handle_inbound(payload["message"]))


@app.post("/begin")
def begin_route():
    """Your systems start a survey here, never the public internet."""
    if not ADMIN_KEY or request.headers.get("X-Survey-Key") != ADMIN_KEY:
        abort(403)
    try:
        return jsonify(begin(request.get_json(force=True)["to"]))
    except OptedOut as exc:
        return jsonify({"error": str(exc)}), 409


if __name__ == "__main__":
    for name, value in (("SMS_FROM", FROM), ("SIGNALWIRE_SIGNING_KEY", SIGNING_KEY),
                        ("INBOUND_URL", INBOUND_URL), ("SURVEY_ADMIN_KEY", ADMIN_KEY)):
        if not value:
            raise SystemExit(f"{name} is required; see .env.example")
    if len(sys.argv) == 3 and sys.argv[1] == "begin":
        print(begin(sys.argv[2]))
    else:
        app.run(port=int(os.getenv("PORT", "8080")))

/**
 * Run an SMS survey over several messages.
 *
 * A survey is one outbound text and then a conversation the platform does not
 * hold for you. Each reply arrives at your inbound webhook on its own, so the
 * state that says which question a number is on lives in a file here, keyed by
 * the sender. The webhook answers with a messaging SWML `reply` carrying the
 * next question, a re-ask for an answer that does not parse, or the closing
 * line.
 *
 * STOP ends everything for that number, and a number that has stopped is never
 * sent a first question. Only SignalWire may post to the webhook: the signature
 * is checked before any state changes.
 *
 * Written against @signalwire/sdk 2.0.5 (RestClient) and the documented inbound
 * message webhook and messaging SWML.
 *
 *     npm start                          # serve /inbound and /begin
 *     npm start begin +14155550123       # text the first question
 */
import "dotenv/config";
import { createHmac, timingSafeEqual } from "node:crypto";
import { existsSync, readFileSync, renameSync, writeFileSync } from "node:fs";
import { createServer, type IncomingMessage } from "node:http";
import { RestClient } from "@signalwire/sdk";

export const client = new RestClient();
// 2.0.5 has no messaging namespace; the send goes through the shared HttpClient
const http = (client.calling as unknown as {
  _http: { post(path: string, body: unknown): Promise<unknown> };
})._http;

export const FROM = process.env["SMS_FROM"] ?? "";
export const SIGNING_KEY = process.env["SIGNALWIRE_SIGNING_KEY"] ?? "";
export const INBOUND_URL = process.env["INBOUND_URL"] ?? "";
// /begin sends billable texts with your credentials, so it is behind a key the
// server holds and your own systems present as X-Survey-Key
export const ADMIN_KEY = process.env["SURVEY_ADMIN_KEY"] ?? "";

// where each number's progress lives; swap for your database
export const STATE_PATH = process.env["SURVEY_STATE_PATH"] ?? "survey-state.json";

// the platform signs its webhooks: hex(HMAC(signing_key, url + raw_body)), SHA-256
// on call requests, SHA-1 on every signed request (docs/swml/guides/webhook-security)
const DIGESTS: Record<string, string> = {
  "x-signalwire-sha256-signature": "sha256", "x-signalwire-signature": "sha1",
};

type Kind = "scale" | "yes_no" | "text";
// the questions, in order: a key for the answer, the text, and how to read a reply
export const QUESTIONS: [string, string, Kind][] = [
  ["rating", "Thanks for visiting Ridgeline Cycles. How was your service today? "
             + "Reply with a number from 1 to 5.", "scale"],
  ["recommend", "Would you recommend us to a friend? Reply YES or NO.", "yes_no"],
  ["comment", "Anything we should know? Reply with a sentence, or SKIP.", "text"],
];
export const DONE = "That is everything. Thank you. Reply STOP at any time to opt out.";
const REASK: Partial<Record<Kind, string>> = {
  scale: "Please reply with a single number from 1 to 5.",
  yes_no: "Please reply YES or NO.",
};

// the single words that end the survey, compared whole after trim and lowercase
const STOP_WORDS = new Set(["stop", "stopall", "unsubscribe", "cancel", "end", "quit"]);
export const STOPPED = "You will receive no more messages from Ridgeline Cycles.";

export type Record_ = {
  step: number; answers: Record<string, unknown>; stopped: boolean;
  seen?: Record<string, string>;
};
type State = Record<string, Record_>;
export type Message = {
  message_id: string; from: string; to: string; body: string | null;
};
type Swml = { version: string; sections: { main: unknown[] } };

export class OptedOut extends Error {}

function load(): State {
  return existsSync(STATE_PATH) ? JSON.parse(readFileSync(STATE_PATH, "utf-8")) : {};
}

function save(state: State) {
  writeFileSync(`${STATE_PATH}.tmp`, JSON.stringify(state, null, 2), "utf-8");
  renameSync(`${STATE_PATH}.tmp`, STATE_PATH);
}

const keyword = (body: string | null | undefined) => (body ?? "").trim().toLowerCase();

/** The answer a reply means, or null when it does not fit the question. */
export function parse(kind: Kind, body: string | null): unknown {
  const word = keyword(body);
  if (kind === "scale") {
    return ["1", "2", "3", "4", "5"].includes(word) ? Number(word) : null;
  }
  if (kind === "yes_no") {
    const answers: Record<string, boolean> = { yes: true, y: true, no: false, n: false };
    return answers[word] ?? null;
  }
  return word === "skip" ? "" : (body ?? "").trim() || null;
}

/** A messaging SWML document with one verb: the text back to the sender. */
export const reply = (text: string): Swml =>
  ({ version: "1.0.0", sections: { main: [{ reply: { body: text } }] } });

export const silence = (): Swml => ({ version: "1.0.0", sections: { main: [] } });

/** Text the first question. A number that said STOP is refused, not texted. */
export async function begin(to: string) {
  const state = load();
  if (state[to]?.stopped) throw new OptedOut(`${to} opted out; no message sent`);
  state[to] = { step: 0, answers: {}, stopped: false };
  save(state);
  return http.post("/api/messaging/messages", { to, from: FROM, body: QUESTIONS[0][1] });
}

/** Advance one number's survey by one reply. Returns the SWML to run. */
export function handleInbound(message: Message): Swml {
  const sender = message.from;
  const state = load();
  const record = state[sender];
  const word = keyword(message.body);

  if (STOP_WORDS.has(word)) {
    state[sender] = { ...(record ?? { step: 0, answers: {} }), stopped: true };
    save(state);
    return reply(STOPPED);
  }
  if (!record || record.stopped) {
    // not in a survey, or out of it: nothing is sent and nothing is recorded.
    // This runs before the cache, or a late retry of an earlier answer would
    // text a question to a number that said STOP
    return silence();
  }
  const seen = record.seen ?? {};
  if (message.message_id in seen) {
    // the webhook was delivered again, maybe late: the answer it got the first
    // time, and the state is not touched. A retried rating parsed at the
    // comment step would otherwise be stored as the comment
    return reply(seen[message.message_id]);
  }
  if (record.step >= QUESTIONS.length) return silence();

  const [key, question, kind] = QUESTIONS[record.step];
  const answer = parse(kind, message.body);
  if (answer === null) return reply(REASK[kind] ?? question);

  record.answers[key] = answer;
  record.step += 1;
  const text = record.step === QUESTIONS.length ? DONE : QUESTIONS[record.step][1];
  (record.seen ??= {})[message.message_id] = text;
  save(state);
  return reply(text);
}

/** True only when a signature header is present and matches. */
export function signed(headers: Record<string, string | undefined>, url: string,
                       rawBody: Buffer, key = SIGNING_KEY): boolean {
  for (const [header, algorithm] of Object.entries(DIGESTS)) {
    const sent = headers[header];
    if (sent) {
      const expected = createHmac(algorithm, key)
        .update(Buffer.concat([Buffer.from(url), rawBody])).digest("hex");
      return sent.length === expected.length
        && timingSafeEqual(Buffer.from(sent), Buffer.from(expected));
    }
  }
  return false;
}

function readBody(req: IncomingMessage): Promise<Buffer> {
  return new Promise((resolve) => {
    const chunks: Buffer[] = [];
    req.on("data", (c: Buffer) => chunks.push(c));
    req.on("end", () => resolve(Buffer.concat(chunks)));
  });
}

/** The two routes: SignalWire posts to /inbound, you post to /begin. */
export function serve(port: number) {
  return createServer(async (req, res) => {
    const raw = await readBody(req);
    const send = (status: number, body: unknown) => {
      res.writeHead(status, { "content-type": "application/json" });
      res.end(JSON.stringify(body));
    };
    if (req.method === "POST" && req.url?.startsWith("/inbound")) {
      const query = req.url.includes("?") ? req.url.slice(req.url.indexOf("?")) : "";
      if (!signed(req.headers as Record<string, string>, INBOUND_URL + query, raw)) {
        return send(403, { error: "unsigned" });
      }
      return send(200, handleInbound(JSON.parse(raw.toString()).message));
    }
    if (req.method === "POST" && req.url === "/begin") {
      // your systems start a survey here, never the public internet
      if (!ADMIN_KEY || req.headers["x-survey-key"] !== ADMIN_KEY) {
        return send(403, { error: "no survey key" });
      }
      try {
        return send(200, await begin(JSON.parse(raw.toString()).to));
      } catch (error) {
        if (error instanceof OptedOut) return send(409, { error: error.message });
        throw error;
      }
    }
    send(404, { error: "no such route" });
  }).listen(port);
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].replace(/\\/g, "/"))) {
  for (const [name, value] of Object.entries({ SMS_FROM: FROM, SIGNALWIRE_SIGNING_KEY:
      SIGNING_KEY, INBOUND_URL, SURVEY_ADMIN_KEY: ADMIN_KEY })) {
    if (!value) throw new Error(`${name} is required; see .env.example`);
  }
  if (process.argv[2] === "begin" && process.argv[3]) {
    console.log(await begin(process.argv[3]));
  } else {
    serve(Number(process.env["PORT"] ?? 8080));
    console.log("listening: POST /inbound (signed), POST /begin {\"to\": ...}");
  }
}

Run it

cd python
pip install -r requirements.txt
cp ../.env.example .env          # project id, API token, space, and the three below
python app.py                    # serves POST /inbound and POST /begin on :8080
python app.py begin +14155550123 # texts the first question

Or the TypeScript surface, on Node 20.18.1 or newer:

cd typescript
npm ci
cp ../.env.example .env
npm start                        # the same two routes

Point the number’s message handler at your public /inbound URL, and put that exact URL in INBOUND_URL: the signature is computed over the URL as SignalWire called it. SMS_FROM is that number, and SIGNALWIRE_SIGNING_KEY is the project’s signing key. SURVEY_ADMIN_KEY is whatever your own systems will send as X-Survey-Key when they start a survey.

Survey traffic from a 10DLC number needs a registered campaign; see [Register a 10DLC brand and campaign](../register-a-10dlc-brand-and-campaign/). Opt-outs are yours to honour. STOP is handled here, and in full in [Handle SMS STOP and START in your own code](../handle-opt-outs-yourself/).

Verify it

No network, no account.

python verify.py          # from the recipe folder, not python/

The verifier points the state at a temporary file, swaps the HTTP layer for a recorder, and asserts the following.

begin sends one documented POST /api/messaging/messages with to, from and the first question, and writes step 0
the module is reloaded before the replies, so the state came from the file and not from memory
a reply that does not fit is re-asked and the step stays at 0; 4, Yes and a comment advance through the second and third questions to the closing line, and the three answers are on disk
a text after the last question, and a text from a number not in a survey, each get an empty document and change nothing
STOP marks the number, a second begin for it raises with no request made
the Flask route returns 403 to an unsigned POST and 200 to one signed with the documented HMAC
the same message_id delivered twice gets the same reply and leaves the state exactly as it was, a late retry of an earlier answer does too, a retried final answer gets the closing line again, and a retry after STOP gets an empty document
/begin refuses a missing or wrong X-Survey-Key with 403 and no request, and sends with the right one
the TypeScript surface runs the same turns from an empty file to the same replies and state, answers a redelivered message without moving, refuses the same begin, and its server gates the signature and the key the same way

Report an issue