< 1.2s

typical AI response latency

0

vendor hops in the media path

2.7B+

minutes processed on the platform

4+

codec conversions in a typical multi-vendor audio path

The problem nobody warns you about

Silent Failures Across Vendor Boundaries

Codec mismatch produces no error

Phone networks speak G.711 mu-law at 8kHz. Your STT model expects PCM at 16kHz. Your TTS generates audio at 24kHz. When the transcoding between vendors fails, there is no error message. Only garbled audio and confused callers.

WebSocket connects, audio vanishes

The HTTP 101 handshake completes successfully. Media packets drop silently between services. Developers spend hours correlating logs across providers before discovering the audio never arrived.

TTS chunks fragment at codec boundaries

When your TTS provider generates audio in chunks and your transcoding layer uses fixed frame sizes, boundaries misalign. The result: glitches at every chunk transition that require custom buffer management per provider.

Barge-in races across external services

Old TTS audio keeps playing because the buffer was not cleared. The 5ms window between voice activity detection and buffer flush is nearly impossible to hit when audio processing spans services with variable network latency.

Build a Voice AI Agent

from signalwire_agents import AgentBase
from signalwire_agents.core.function_result import SwaigFunctionResult

class SupportAgent(AgentBase):
    def __init__(self):
        super().__init__(name="Support Agent", route="/support")
        self.prompt_add_section("Instructions",
            body="You are a customer support agent. "
                 "Greet the caller and resolve their issue.")
        self.add_language("English", "en-US", "rime.spore:mistv2")

    @AgentBase.tool(name="check_order")
    def check_order(self, order_id: str):
        """Check the status of a customer order.

        Args:
            order_id: The order ID to look up
        """
        return SwaigFunctionResult(f"Order {order_id}: shipped, ETA April 2nd")

agent = SupportAgent()
agent.run()

import { AgentBase, FunctionResult } from '@signalwire/sdk';

const agent = new AgentBase({
  name: 'Support Agent',
  route: '/support',
});

agent.promptAddSection('Instructions',
  'You are a customer support agent. Greet the caller and resolve their issue.');

agent.addLanguage({ name: 'English', code: 'en-US', voice: 'rime.spore:mistv2' });

agent.defineTool({
  name: 'check_order',
  description: 'Check the status of a customer order',
  parameters: {
    type: 'object',
    properties: {
      order_id: { type: 'string', description: 'The order ID to look up' },
    },
    required: ['order_id'],
  },
  handler: (args) => {
    return new FunctionResult(`Order ${args.order_id}: shipped, ETA April 2nd`);
  },
});

agent.run();

package main

import (
	"fmt"

	"github.com/signalwire/signalwire-go/pkg/agent"
	"github.com/signalwire/signalwire-go/pkg/swaig"
)

func main() {
	a := agent.NewAgentBase(
		agent.WithName("Support Agent"),
		agent.WithRoute("/support"),
	)

	a.PromptAddSection("Instructions",
		"You are a customer support agent. Greet the caller and resolve their issue.")

	a.AddLanguage(map[string]any{
		"name": "English", "code": "en-US", "voice": "rime.spore:mistv2",
	})

	a.DefineTool(agent.ToolDefinition{
		Name:        "check_order",
		Description: "Check the status of a customer order",
		Parameters: map[string]any{
			"type": "object",
			"properties": map[string]any{
				"order_id": map[string]any{
					"type": "string", "description": "The order ID to look up",
				},
			},
			"required": []string{"order_id"},
		},
		Handler: func(args map[string]any, rawData map[string]any) *swaig.FunctionResult {
			orderID := args["order_id"]
			return swaig.NewFunctionResult(
				fmt.Sprintf("Order %v: shipped, ETA April 2nd", orderID),
			)
		},
	})

	a.Run()
}

import com.signalwire.sdk.agent.AgentBase;
import com.signalwire.sdk.swaig.FunctionResult;

import java.util.List;
import java.util.Map;

public class SupportAgent {
    public static void main(String[] args) throws Exception {
        var agent = AgentBase.builder()
                .name("Support Agent")
                .route("/support")
                .build();

        agent.promptAddSection("Instructions",
                "You are a customer support agent. "
              + "Greet the caller and resolve their issue.");

        agent.addLanguage("English", "en-US", "rime.spore:mistv2");

        agent.defineTool(
                "check_order",
                "Check the status of a customer order",
                Map.of("type", "object",
                       "properties", Map.of(
                           "order_id", Map.of(
                               "type", "string",
                               "description", "The order ID to look up")),
                       "required", List.of("order_id")),
                (toolArgs, rawData) -> {
                    var orderId = toolArgs.get("order_id");
                    return new FunctionResult(
                        "Order " + orderId + ": shipped, ETA April 2nd");
                }
        );

        agent.run();
    }
}

# frozen_string_literal: true

require 'signalwire'

agent = SignalWire::AgentBase.new(name: 'Support Agent', route: '/support')

agent.prompt_add_section('Instructions',
  'You are a customer support agent. Greet the caller and resolve their issue.')

agent.add_language(name: 'English', code: 'en-US', voice: 'rime.spore:mistv2')

agent.define_tool(
  name:        'check_order',
  description: 'Check the status of a customer order',
  parameters:  {
    'order_id' => { 'type' => 'string', 'description' => 'The order ID to look up' }
  }
) do |args, _raw|
  SignalWire::Swaig::FunctionResult.new(
    "Order #{args['order_id']}: shipped, ETA April 2nd"
  )
end

agent.run

<?php
require 'vendor/autoload.php';

use SignalWire\Agent\AgentBase;
use SignalWire\SWAIG\FunctionResult;

$agent = new AgentBase(['name' => 'Support Agent', 'route' => '/support']);

$agent->promptAddSection('Instructions',
    'You are a customer support agent. Greet the caller and resolve their issue.');

$agent->addLanguage('English', 'en-US', 'rime.spore:mistv2');

$agent->defineTool(
    name: 'check_order',
    description: 'Check the status of a customer order',
    parameters: [
        'order_id' => ['type' => 'string', 'description' => 'The order ID to look up'],
    ],
    handler: function (array $args): FunctionResult {
        return new FunctionResult("Order {$args['order_id']}: shipped, ETA April 2nd");
    }
);

$agent->run();

#!/usr/bin/env perl
use strict;
use warnings;
use lib 'lib';
use SignalWire::Agent::AgentBase;
use SignalWire::SWAIG::FunctionResult;

my $agent = SignalWire::Agent::AgentBase->new(
    name  => 'Support Agent',
    route => '/support',
);

$agent->prompt_add_section('Instructions',
    'You are a customer support agent. Greet the caller and resolve their issue.');

$agent->add_language(name => 'English', code => 'en-US', voice => 'rime.spore:mistv2');

$agent->define_tool(
    name        => 'check_order',
    description => 'Check the status of a customer order',
    parameters  => {
        order_id => { type => 'string', description => 'The order ID to look up' },
    },
    handler => sub {
        my ($args, $raw) = @_;
        return SignalWire::SWAIG::FunctionResult->new(
            response => "Order $args->{order_id}: shipped, ETA April 2nd"
        );
    },
);

$agent->run;

#include <signalwire/agent/agent_base.hpp>

using namespace signalwire;
using json = nlohmann::json;

class SupportAgent : public agent::AgentBase {
public:
    SupportAgent() : AgentBase("Support Agent", "/support") {
        prompt_add_section("Instructions",
            "You are a customer support agent. "
            "Greet the caller and resolve their issue.");

        add_language({"English", "en-US", "rime.spore:mistv2"});

        define_tool({
            .name = "check_order",
            .description = "Check the status of a customer order",
            .parameters = {
                {"order_id", {{"type", "string"},
                              {"description", "The order ID to look up"}}}
            },
            .handler = [](const json& args, const json&) {
                auto order_id = args.value("order_id", "unknown");
                return swaig::FunctionResult(
                    "Order " + order_id + ": shipped, ETA April 2nd");
            }
        });
    }
};

int main() {
    SupportAgent().run();
}

using SignalWire.Agent;
using SignalWire.SWAIG;

var agent = new AgentBase(new AgentOptions { Name = "Support Agent", Route = "/support" });

agent.PromptAddSection("Instructions",
    "You are a customer support agent. Greet the caller and resolve their issue.");

agent.AddLanguage("English", "en-US", "rime.spore:mistv2");

agent.DefineTool("check_order", "Check the status of a customer order",
    new { type = "object", properties = new {
        order_id = new { type = "string", description = "The order ID to look up" }
    }, required = new[] { "order_id" } },
    (args, rawData) =>
    {
        var orderId = args.TryGetValue("order_id", out var id) ? id : "unknown";
        return new FunctionResult($"Order {orderId}: shipped, ETA April 2nd");
    });

agent.Run();

use signalwire::agent::AgentBase;
use signalwire::swaig::FunctionResult;
use serde_json::json;

fn main() {
    let mut agent = AgentBase::builder()
        .name("Support Agent")
        .route("/support")
        .build();

    agent
        .prompt_add_section("Instructions",
            "You are a customer support agent. Greet the caller and resolve their issue.", &[])
        .add_language("English", "en-US", "rime.spore:mistv2");

    agent.define_tool(
        "check_order",
        "Check the status of a customer order",
        json!({"type": "object", "properties": {
            "order_id": {"type": "string", "description": "The order ID to look up"}
        }, "required": ["order_id"]}),
        Box::new(|args, _raw| {
            let order_id = args.get("order_id").and_then(|v| v.as_str()).unwrap_or("unknown");
            FunctionResult::with_response(&format!("Order {order_id}: shipped, ETA April 2nd"))
        }),
    );

    agent.run();
}

Multi-vendor audio pipeline vs. single media engine

Multi-Vendor Pipeline

G.711 to PCM transcoding between telephony and STT providers
WebSocket relay adds encoding, decoding, and network hops
TTS chunk boundaries misalign with codec frame sizes
Barge-in requires coordinating buffer flush across external services
Silent failures produce no errors, no logs, no alerts

SignalWire

PSTN audio enters the media engine and gets processed in place
No WebSocket relay, no base64 encoding, no sample rate mismatch
TTS streaming and STT processing share the same audio buffer
Barge-in handled natively with sub-millisecond buffer management
Format mismatches caught and reported as structured errors

How codec transcoding fails silently

Failure	What callers hear	Error message
Mu-law sent to a service expecting PCM	Garbled, unintelligible audio	None
TTS buffer fragmented at codec boundaries	Choppy, stuttering responses	None
Sample rate mismatch after WebSocket relay	Chipmunk or slow-motion audio	None
Base64 encoding error in the mu-law path	Static or silence	None
Media packets dropped between services	Complete silence mid-call	WebSocket reports connected

Twilio sends mulaw 8kHz. If the STT expects PCM 16kHz, you get garbled audio. No error. No log. The call sounds broken and you have no idea why.
Developer community, Custom voice AI pipelines

From five-vendor audio pipeline to one

1

Define your agent

Write a YAML document or Python class. Point a phone number at it.

2

Audio handled internally

The media engine negotiates codecs, processes STT and TTS, and manages buffers in one process.

3

Test with real calls

Call the number. Audio quality is consistent because there are no vendor boundaries to cross.

4

Ship to production

Same platform, same audio path, same behavior at scale. No integration surprises.

Upsampling 8kHz audio to 16kHz does not recover lost frequency information. STT accuracy is determined by the input codec, not the model. If your audio pipeline transcodes through G.711 before reaching the AI, you have already lost the data that matters.

FAQ

Can I bring my own STT or TTS provider?

Yes. The platform supports multiple STT and TTS providers. Codec negotiation and format conversion happen inside the media engine regardless of which provider you choose.

What codecs does SignalWire support?

G.711 (mu-law and A-law), G.722, and Opus with automatic negotiation. Conversion between PSTN codecs and AI model requirements happens inside the media engine.

How does barge-in work without external WebSocket coordination?

Voice activity detection, TTS streaming, and STT processing share the same audio buffer inside the media engine. Barge-in is a native operation, not a race condition across external services.

Who built this platform?

The team that wrote FreeSWITCH, the open-source telephony engine that processes trillions of minutes across the industry. Production audio engineering is the foundation.

Trusted by 2,000+ companies

Stop debugging codecs. Start shipping agents.

One media engine. No vendor gaps. No silent failures. Built by the team that wrote FreeSWITCH.

Get Started Free Read the Docs

Your Audio Pipeline Fails Without Errors