Put an AI agent in a web chat widget

AI AgentsAI web chat widget

A small gateway on your server holds the API token, mints a signed handle per visitor and forwards the page's messages to the AI Chat API. The page learns a publishable key and nothing else.

Also called website chatbot from a voice AI agent, chat gateway, publishable key for AI chat

chatbrowserrest

The claim

[Run the same voice AI agent over text chat](../run-the-same-agent-over-text/) drives an agent from a server. A web page is not a server: the API token carries the whole project, and anything in a page is public. This recipe puts a gateway between the two. The browser sends POST /chat/ with a publishable key in Authorization: Bearer and a method of start, chat, log or end. The gateway does the rest.

Why it holds

*It injects config_url.* The agent’s URL lives in the gateway’s environment and is sent on every create_conversation. A config_url in the page’s request is ignored, so a visitor cannot pick which agent runs.
*It signs the handle.* start mints a conversation id and returns <id>.<HMAC> in an X-Chat-Handle header. Every later method carries the handle back, and one flipped character is a 403 before any request, so an id cannot be forged or guessed.
*It caps the spend.* New conversations per window and turns per conversation are counted in the process, and the platform is not asked once a cap is hit. Those two numbers bound what a leaked key can cost, because a leaked key opens many one-turn conversations rather than hammering one.
*It filters the log.* log returns user and assistant entries only, never the prompt or the tool traffic.

The wire is modelled on signalwire.ai_chat.ChatGateway, which the SDK ships in releases newer than the pinned 3.0.1, with the same methods, header and coarse {"error": reason} refusals. Its reference is the source for the design. A publishable key is public by definition. The origin allowlist is leak containment rather than access control. The caps are what bound the cost. Upstream, every method is one JSON-RPC 2.0 POST to /api/ai/chat, held to the vendored REST spec.

How it works

@app.post("/chat/")
def gateway():
    check_key_and_origin()                       # 401, 403; nothing forwarded
    body = request.get_json(silent=True) or {}
    if body.get("method") == "start":
        charge_mint()                            # 429 past MAX_NEW_CONVERSATIONS
        cid, handle = mint_handle()              # "chat-<uuid>.<hmac>"
        made = rpc("create_conversation", {"id": cid, "config_url": CONFIG_URL,
                                           "conversation_timeout": TIMEOUT})
        out = jsonify({"greeting": made.get("initial_message"), ...})
        out.headers["X-Chat-Handle"] = handle
        return out
    cid = read_handle(body.get("handle")) or refuse(403, "handle")
    if body.get("method") == "chat":
        charge_turn(cid)                         # 429 past MAX_TURNS
        return jsonify(rpc("chat", {"id": cid, "message": body["message"]}))

What the page sends for a turn, and what the platform receives for it:

{"method": "chat", "handle": "chat-9c41....3f2a...", "message": "When are you open?"}

{"jsonrpc": "2.0", "id": "1f3a...", "method": "chat",
 "params": {"id": "chat-9c41...", "message": "When are you open?"}}

The page is web/widget.html, served by both surfaces at / with the key substituted in. It calls start on load, shows the greeting, and sends each typed line as a chat.

Limitations

The counters live in the serving process. Behind several replicas each keeps its own, so the effective cap multiplies by the replica count. Put a shared limiter in front, or accept that.

The origin allowlist stops a key pasted into someone else’s page. It does not stop curl, which omits the header. The caps are what bound that.

A page refresh loses the handle, so the visitor starts a new conversation. Keeping it in sessionStorage is a page change, not a gateway change. The end sent on pagehide is best effort; the turn counter expires on its own if it never arrives.

The verifier proves the gateway and the envelopes, not the model’s replies.

What to change first

Remove charge_turn(cid) from the chat branch and run the verifier. The third turn now reaches the platform and the cap assertion fails. That is the point: without it a leaked key spends without limit.

Where this sits

Before this

Run the same voice AI agent over text chat

Embed a state-aware voice agent in your web page Publish events to browsers with PubSub Embed a call widget with no backend

Seen in a build

One of 8 recipes composed by Order status desk: one omnichannel AI agent for web chat and phone.

python/app.py

"""Put an AI agent in a web chat widget.

A page cannot hold a SignalWire API token: the token carries the whole
project. This gateway holds it, and the page learns two things, the gateway's
URL and a publishable key. The gateway injects `config_url` itself, so a
visitor cannot pick which agent runs; it signs a handle per conversation, so
an id cannot be forged or guessed; and it caps new conversations per window
and turns per conversation, which bounds what a leaked key can cost.

The wire is modelled on the SDK's own `ChatGateway` (newer than the pinned
3.0.1): `POST /chat/` with the key in `Authorization: Bearer`, a `method` of
`start`, `chat`, `log` or `end`, and an `X-Chat-Handle` header on the
response that minted the handle. Upstream, every method is one JSON-RPC 2.0
POST to `/api/ai/chat` (the vendored REST spec, `tools/openapi/rest.json`).

Written against signalwire-sdk 3.0.1 (RestClient) and Flask.

    python app.py          # the page at /, the gateway at /chat/
"""
import hashlib
import hmac
import json
import os
import time
import uuid
from collections import deque
from pathlib import Path

from dotenv import load_dotenv
from flask import Flask, Response, abort, jsonify, request
from signalwire.rest import RestClient

# the SDK does not read .env for you
load_dotenv()

# reads SIGNALWIRE_PROJECT_ID / SIGNALWIRE_API_TOKEN / SIGNALWIRE_SPACE from the env
client = RestClient()

CHAT = "/api/ai/chat"
# the agent this gateway may talk to; never accepted from the request
CONFIG_URL = os.environ["AGENT_CONFIG_URL"]
KEY = os.environ["CHAT_GATEWAY_KEY"]              # the page carries this
SECRET = os.environ["CHAT_GATEWAY_SECRET"].encode()   # the page never sees this
ALLOWED_ORIGINS = {o.strip() for o in os.getenv("ALLOWED_ORIGINS", "").split(",")
                   if o.strip()}
# int() refuses a typo, so a mistyped cap stops the process rather than
# silently disabling the cap
MAX_NEW = int(os.getenv("MAX_NEW_CONVERSATIONS", "60"))
WINDOW = int(os.getenv("WINDOW_SECONDS", "60"))
MAX_TURNS = int(os.getenv("MAX_TURNS", "200"))
TIMEOUT = int(os.getenv("CONVERSATION_TIMEOUT", "1800"))
MAX_BODY = 16 * 1024   # a chat message, not an upload
PAGE = (Path(__file__).resolve().parent.parent / "web" / "widget.html").read_text("utf-8")
VISIBLE_ROLES = ("user", "assistant")   # a page redraws these; never the prompt or tools

# counters live in this process; behind replicas each keeps its own
_mints = deque()
_turns = {}   # conversation id -> [turns used, last seen]


class UpstreamError(Exception):
    """A JSON-RPC error from the service. It arrives inside an HTTP 200."""


def rpc(method, params, http=None):
    """One JSON-RPC 2.0 request."""
    body = {"jsonrpc": "2.0", "id": uuid.uuid4().hex, "method": method, "params": params}
    envelope = (http or client._http).post(CHAT, body=body)
    if "error" in envelope:
        err = envelope["error"] or {}
        raise UpstreamError(f"{err.get('code')}: {err.get('message', '')}")
    return envelope.get("result", {})


def sign(cid):
    return hmac.new(SECRET, cid.encode(), hashlib.sha256).hexdigest()[:32]


def mint_handle():
    """A fresh conversation id and the handle that proves this server made it."""
    cid = "chat-" + uuid.uuid4().hex
    return cid, f"{cid}.{sign(cid)}"


def read_handle(handle):
    """The conversation id inside a handle, or None when it was not ours."""
    cid, _, sig = (handle or "").rpartition(".")
    if not cid or not hmac.compare_digest(sig, sign(cid)):
        return None
    return cid


def refuse(status, reason):
    """Coarse on purpose: a finer reason lets a caller map the caps by probing."""
    abort(Response(json.dumps({"error": reason}), status=status,
                   mimetype="application/json"))


def check_key_and_origin():
    if request.headers.get("Authorization") != f"Bearer {KEY}":
        refuse(401, "key")
    origin = request.headers.get("Origin")
    local = origin and (origin.startswith("http://localhost")
                        or origin.startswith("http://127.0.0.1"))
    if origin and not local and origin not in ALLOWED_ORIGINS:
        refuse(403, "origin")


def charge_mint(now=None):
    now = now or time.time()
    while _mints and _mints[0] < now - WINDOW:
        _mints.popleft()
    if len(_mints) >= MAX_NEW:
        refuse(429, "limit")
    _mints.append(now)


def prune(now=None):
    """Forget conversations idle past the service's own timeout, so a visitor
    who closes the tab without `end` does not leave a counter behind forever."""
    now = now or time.time()
    for cid in [c for c, (_, last) in _turns.items() if last < now - TIMEOUT]:
        del _turns[cid]


def charge_turn(cid, now=None):
    now = now or time.time()
    prune(now)
    used, _ = _turns.get(cid, (0, now))
    if used >= MAX_TURNS:
        refuse(429, "limit")
    _turns[cid] = [used + 1, now]


app = Flask(__name__)
app.config["MAX_CONTENT_LENGTH"] = MAX_BODY


@app.errorhandler(413)
def too_large(_exc):
    return jsonify({"error": "malformed"}), 413


@app.errorhandler(UpstreamError)
def upstream_failed(_exc):
    # the page always reads JSON; a refusal it can show beats an HTML 500
    return jsonify({"error": "upstream"}), 502


@app.get("/")
def index():
    return Response(PAGE.replace("__CHAT_KEY__", KEY), mimetype="text/html")


@app.post("/chat/")
def gateway():
    check_key_and_origin()
    body = request.get_json(silent=True) or {}
    method = body.get("method", "chat")
    if method == "start":
        charge_mint()
        cid, handle = mint_handle()
        made = rpc("create_conversation",
                   {"id": cid, "config_url": CONFIG_URL, "conversation_timeout": TIMEOUT})
        out = jsonify({"greeting": made.get("initial_message"),
                       "status": made.get("status"), "timeout": TIMEOUT})
        out.headers["X-Chat-Handle"] = handle
        return out
    cid = read_handle(body.get("handle"))
    if not cid:
        refuse(403, "handle")
    if method == "chat":
        message = body.get("message")
        if not isinstance(message, str) or not message.strip():
            refuse(400, "malformed")
        charge_turn(cid)
        # the visitor is always the user; a role or a config_url in the body is ignored
        return jsonify(rpc("chat", {"id": cid, "message": message}))
    if method == "log":
        entries = rpc("chat_log", {"id": cid}).get("chat_log", [])
        visible = [e for e in entries if e.get("role") in VISIBLE_ROLES]
        return jsonify({"messages": visible})
    if method == "end":
        rpc("end_conversation", {"id": cid})
        _turns.pop(cid, None)
        return jsonify({"status": "ended"})
    refuse(400, "malformed")


if __name__ == "__main__":
    app.run(port=int(os.getenv("PORT", "8080")))

/**
 * Put an AI agent in a web chat widget.
 *
 * A page cannot hold a SignalWire API token: the token carries the whole
 * project. This gateway holds it, and the page learns two things, the
 * gateway's URL and a publishable key. The gateway injects `config_url`
 * itself, so a visitor cannot pick which agent runs; it signs a handle per
 * conversation, so an id cannot be forged or guessed; and it caps new
 * conversations per window and turns per conversation, which bounds what a
 * leaked key can cost.
 *
 * The wire is modelled on the Python SDK's `ChatGateway`: `POST /chat/` with
 * the key in `Authorization: Bearer`, a `method` of `start`, `chat`, `log` or
 * `end`, and an `X-Chat-Handle` header on the response that minted the
 * handle. Upstream, every method is one JSON-RPC 2.0 POST to `/api/ai/chat`.
 *
 * Written against @signalwire/sdk 2.0.5 (RestClient) and node:http.
 *
 *     npm start          # the page at /, the gateway at /chat/
 */
import "dotenv/config";
import { createHmac, randomUUID, timingSafeEqual } from "node:crypto";
import { readFileSync } from "node:fs";
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
import { RestClient } from "@signalwire/sdk";

// reads SIGNALWIRE_PROJECT_ID / SIGNALWIRE_API_TOKEN / SIGNALWIRE_SPACE
export const client = new RestClient();
type Envelope = {
  result?: Record<string, unknown>; error?: { code?: number; message?: string };
};
type Http = { post(path: string, body: unknown): Promise<Envelope> };
const http = (client.chat as unknown as { _http: Http })._http;

export const CHAT = "/api/ai/chat";
const env = (name: string) => {
  const value = process.env[name];
  if (!value) throw new Error(`${name} is required; see .env.example`);
  return value;
};
/** A cap that fails to parse would disable itself: `NaN >= n` is always false. */
const num = (name: string, fallback: number) => {
  const raw = process.env[name];
  const value = raw === undefined ? fallback : Number(raw);
  if (!Number.isFinite(value) || value <= 0) {
    throw new Error(`${name} must be a positive number, not '${raw}'`);
  }
  return value;
};
// the agent this gateway may talk to; never accepted from the request
export const CONFIG_URL = env("AGENT_CONFIG_URL");
export const KEY = env("CHAT_GATEWAY_KEY");                 // the page carries this
const SECRET = env("CHAT_GATEWAY_SECRET");                  // the page never sees this
const ALLOWED_ORIGINS = new Set((process.env["ALLOWED_ORIGINS"] ?? "")
  .split(",").map((o) => o.trim()).filter(Boolean));
const MAX_NEW = num("MAX_NEW_CONVERSATIONS", 60);
const WINDOW = num("WINDOW_SECONDS", 60);
const MAX_TURNS = num("MAX_TURNS", 200);
export const TIMEOUT = num("CONVERSATION_TIMEOUT", 1800);
export const MAX_BODY = 16 * 1024;   // a chat message, not an upload
const PAGE = readFileSync(new URL("../../web/widget.html", import.meta.url), "utf-8");
const VISIBLE_ROLES = new Set(["user", "assistant"]);   // never the prompt or tools

// counters live in this process; behind replicas each keeps its own
const mints: number[] = [];
export const turns = new Map<string, { used: number; last: number }>();

/** One JSON-RPC 2.0 request. An error arrives inside an HTTP 200. */
export async function rpc(method: string, params: Record<string, unknown>,
                          transport: Http = http) {
  const envelope = await transport.post(CHAT, {
    jsonrpc: "2.0", id: randomUUID().replace(/-/g, ""), method, params,
  });
  if ("error" in envelope) {
    const err = envelope.error ?? {};
    throw new Error(`${err.code}: ${err.message ?? ""}`);
  }
  return envelope.result ?? {};
}

const sign = (cid: string) =>
  createHmac("sha256", SECRET).update(cid).digest("hex").slice(0, 32);

/** A fresh conversation id and the handle that proves this server made it. */
export function mintHandle() {
  const cid = "chat-" + randomUUID().replace(/-/g, "");
  return { cid, handle: `${cid}.${sign(cid)}` };
}

/** The conversation id inside a handle, or null when it was not ours. */
export function readHandle(handle: unknown) {
  if (typeof handle !== "string") return null;
  const at = handle.lastIndexOf(".");
  if (at < 1) return null;
  const cid = handle.slice(0, at);
  const sig = Buffer.from(handle.slice(at + 1));
  const want = Buffer.from(sign(cid));
  return sig.length === want.length && timingSafeEqual(sig, want) ? cid : null;
}

/** Coarse on purpose: a finer reason lets a caller map the caps by probing. */
class Refusal extends Error {
  constructor(public status: number, public reason: string) { super(reason); }
}

function checkKey(req: IncomingMessage) {
  if (req.headers.authorization !== `Bearer ${KEY}`) throw new Refusal(401, "key");
}

function checkOrigin(req: IncomingMessage) {
  const origin = req.headers.origin;
  const local = origin?.startsWith("http://localhost")
    || origin?.startsWith("http://127.0.0.1");
  if (origin && !local && !ALLOWED_ORIGINS.has(origin)) {
    throw new Refusal(403, "origin");
  }
}

function chargeMint(now = Date.now() / 1000) {
  while (mints.length && mints[0]! < now - WINDOW) mints.shift();
  if (mints.length >= MAX_NEW) throw new Refusal(429, "limit");
  mints.push(now);
}

/**
 * Forget conversations idle past the service's own timeout, so a visitor who
 * closes the tab without `end` does not leave a counter behind forever.
 */
export function prune(now = Date.now() / 1000) {
  for (const [cid, entry] of turns) {
    if (entry.last < now - TIMEOUT) turns.delete(cid);
  }
}

function chargeTurn(cid: string, now = Date.now() / 1000) {
  prune(now);
  const used = turns.get(cid)?.used ?? 0;
  if (used >= MAX_TURNS) throw new Refusal(429, "limit");
  turns.set(cid, { used: used + 1, last: now });
}

type Body = { method?: string; handle?: unknown; message?: unknown };
type Reply = { status: number; json: unknown; headers?: Record<string, string> };

/** The gateway, as a function of the request so a test can drive it directly. */
export async function gateway(req: IncomingMessage, body: Body,
                              transport: Http = http): Promise<Reply> {
  checkKey(req);
  checkOrigin(req);
  const method = body.method ?? "chat";
  if (method === "start") {
    chargeMint();
    const { cid, handle } = mintHandle();
    const params = { id: cid, config_url: CONFIG_URL, conversation_timeout: TIMEOUT };
    const made = await rpc("create_conversation", params, transport);
    return { status: 200, headers: { "x-chat-handle": handle },
             json: { greeting: made["initial_message"] ?? null, status: made["status"],
                     timeout: TIMEOUT } };
  }
  const cid = readHandle(body.handle);
  if (!cid) throw new Refusal(403, "handle");
  if (method === "chat") {
    const message = body.message;
    if (typeof message !== "string" || !message.trim()) {
      throw new Refusal(400, "malformed");
    }
    chargeTurn(cid);
    // the visitor is always the user; a role or a config_url in the body is ignored
    return { status: 200, json: await rpc("chat", { id: cid, message }, transport) };
  }
  if (method === "log") {
    const result = await rpc("chat_log", { id: cid }, transport);
    const entries = (result["chat_log"] ?? []) as { role?: string }[];
    const messages = entries.filter((e) => VISIBLE_ROLES.has(e.role ?? ""));
    return { status: 200, json: { messages } };
  }
  if (method === "end") {
    await rpc("end_conversation", { id: cid }, transport);
    turns.delete(cid);
    return { status: 200, json: { status: "ended" } };
  }
  throw new Refusal(400, "malformed");
}

/**
 * The body, or a 413 once more than MAX_BODY bytes have arrived. Past the cap
 * the rest is drained rather than the socket destroyed, so the refusal can
 * still be written; the response then closes the connection.
 */
function readBody(req: IncomingMessage): Promise<string> {
  return new Promise((resolve, reject) => {
    const chunks: Buffer[] = [];
    let size = 0;
    const onData = (c: Buffer) => {
      size += c.length;
      if (size > MAX_BODY) {
        req.off("data", onData);
        req.resume();
        reject(new Refusal(413, "malformed"));
        return;
      }
      chunks.push(c);
    };
    req.on("data", onData);
    req.on("end", () => resolve(Buffer.concat(chunks).toString()));
  });
}

async function handle(req: IncomingMessage, res: ServerResponse, transport: Http) {
  if (req.method === "GET" && req.url === "/") {
    res.writeHead(200, { "content-type": "text/html; charset=utf-8" });
    return res.end(PAGE.replace("__CHAT_KEY__", KEY));
  }
  if (req.method !== "POST" || req.url !== "/chat/") {
    res.writeHead(404);
    return res.end();
  }
  let reply: Reply;
  try {
    // the key is checked before a byte of the body is buffered, and a declared
    // length past the cap is refused before it is read
    checkKey(req);
    if (Number(req.headers["content-length"] ?? 0) > MAX_BODY) {
      throw new Refusal(413, "malformed");
    }
    let body: Body = {};
    try { body = JSON.parse(await readBody(req) || "{}"); } catch (error) {
      if (error instanceof Refusal) throw error;   // the body was too large
      /* otherwise malformed JSON: an empty body, refused below */
    }
    reply = await gateway(req, body, transport);
  } catch (error) {
    const r = error instanceof Refusal ? error : new Refusal(502, "upstream");
    reply = { status: r.status, json: { error: r.reason } };
  }
  const headers: Record<string, string> = { "content-type": "application/json",
                                            ...(reply.headers ?? {}) };
  if (reply.status === 413) headers["connection"] = "close";   // stop reading the rest
  res.writeHead(reply.status, headers);
  res.end(JSON.stringify(reply.json));
}

/** The page and the gateway on one port. */
export function serve(port: number, transport: Http = http) {
  // a rejection that escaped would end the process; everything answers 500 instead
  return createServer((req, res) => {
    handle(req, res, transport).catch(() => {
      res.writeHead(500, { "content-type": "application/json" });
      res.end(JSON.stringify({ error: "server" }));
    });
  }).listen(port);
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].replace(/\\/g, "/"))) {
  serve(Number(process.env["PORT"] ?? 8080));
}

Run it

cd python
pip install -r requirements.txt
cp ../.env.example .env      # token with the chat scope, the agent URL, a key, a secret
python app.py                # the page at http://localhost:8080/

AGENT_CONFIG_URL is the running agent from the prerequisite recipe, with its basic-auth pair in the URL. Open the page and type.

The TypeScript surface serves the same page and wire on Node 20.18.1 or newer:

cd typescript
npm ci
cp ../.env.example .env
npm start

Set ALLOWED_ORIGINS to the site that will embed the page before deploying; localhost is always allowed so development works unconfigured.

Verify it

No network, no account.

python verify.py          # from the recipe folder, not python/

The verifier drives the Flask app with its test client, swaps the SDK’s HTTP layer for a recorder, and asserts the following.

the page carries the key and neither the token, the secret nor the agent URL
a request without the key is a 401, one from a foreign origin a 403, a local origin is allowed, and none of the refusals reach the platform
start sends create_conversation with the handle’s id, the configured config_url and the timeout, every param documented and every required one present, and answers with the greeting and an X-Chat-Handle header
a handle with one flipped character, or another id under a valid signature, is a 403 with no request
a turn forwards the message under the handle’s id and drops a role and a config_url the page tried to send; an empty message is a 400
the third turn on a two-turn cap and the third start on a two-conversation cap are 429s with no request
log returns user and assistant entries only, end sends end_conversation, and an unknown method is a 400
the token appears in no response body or header
a JSON-RPC error from the service is a JSON 502 the page can show, not an HTML 500
a body past 16 KiB is a 413 with no request, and in Node the key is checked before a byte of it is buffered
a turn counter left by a visitor who never sent end is forgotten after the conversation timeout, and a live one is kept
a cap that does not parse stops the Node process instead of disabling itself
the page’s form is disabled until start answers and while a turn is out
the TypeScript surface, on a real port, serves the same page, refuses the same requests, caps the same counts and sends the same six envelopes

Report an issue