Act One Is Not the Whole Play: The AI Gold Rush Is a Phase

ClueCon turned 22 this year. Any baby born the year we started it can legally drink now, which is not arithmetic I recommend doing out loud on stage.

I opened the keynote with a timeline. It starts in 2006. Everyone's eye goes to the red section at the top, which starts in 2023, when the industry decided AI was the whole story. The part I care about is underneath. We were excited about what goes under AI before we knew when AI would arrive. People had been promising it since the 1950s, so nobody had a date. What we did know was that the future would run on software, and somebody had to build the layer below it.

Two arguments came out of that talk. I will make both again here.

The first: this industry has run this movie before and the ending is known. Gold rushes consolidate, and the value moves up the stack.

The second, which matters more right now: voice AI demos die on contact with production for reasons that have almost nothing to do with the model. They die because a generation of builders threw out twenty-five years of real-time engineering in order to reach the model faster.

This industry has run this movie before

I lived the web hosting version of this. The dot-com bubble was around the corner and everyone was building everything. There was a new thing, and nobody wanted to be the person who did not build it. Grab a pickaxe, go dig in the hills, and maybe somebody writes a fifty million dollar check. Then the slow failure starts. A couple of companies survive, usually the ones stubborn enough to keep going when it stops being fun. Then the value moves up the stack, to the people building products instead of foundations.

The Innovator's Dilemma was written in the 1990s and it is still correct. Clayton Christensen used hard drive speed as his example, because drives were shrinking and accelerating faster than incumbents could react. AI moves at something like a hundred times that rate, in part because AI compresses its own timeline. The tooling that builds the next generation is built by the current one.

So we are in Act One. Everyone is taking AI and adding it to things. I am guilty of it too. ChatGPT shipped, an API followed a month later, and I immediately started thinking about where it belonged in what we already ran.

What worries me about Act One is not that the models get worse. It is that a large number of talented people will spend the entire act rebuilding the bottom of the stack. Then they run out of money before shipping the top of it.

Nobody builds their own kernel

The first FreeSWITCH website had a caveman at the top. A little animated GIF of a guy hitting a wheel with a hammer. Underneath were pictures of wheels that already existed, and the message was that we were not going to reinvent them.

That was the working rule. When we needed a new capability, the first question was whether it already existed somewhere. If it did, we wrote a module and exposed what it offered rather than rebuilding it so we could claim we invented it. We thought about the platform the way we think about an operating system: a set of tools underneath, extended by the software you write on top.

Nobody in this field writes their own Linux kernel. There is a voice on your shoulder saying you could, and it sounds fun, and the answer is still no. Same with the database. Same with TLS, which I would happily redesign given a free decade, and which I am nonetheless going to keep using as shipped. Our job in that stack was the soft switch, and we did it so that nobody else would have to do it again.

The versatility was deliberate, not lucky. In 2006 nobody cared that FreeSWITCH supported multiple sampling rates, because the phone system was 8 kHz and that was the end of the discussion. We built it anyway, because we were guessing at what would need to attach later. When WebRTC arrived and suddenly 48 kHz and 16 kHz had to interoperate on the same call, that work was already done. We were not solving it under deadline in the middle of a shift in the industry.

Someone in the audience asked about this directly. His premise was that open source in this space is unopinionated. Every integrator has to make a hundred decisions from scratch, and that is why assembly takes so long. Yes. That is exactly right, and it was a design choice I made. I came up through Perl and I have been a Unix person since my first computer, which was literally a terminal. My instinct is to give people options. When I wrote FreeSWITCH I could not afford an opinion, because one person's feature is the next person's bug. So the answer was parameters, then more parameters, then detection logic for specific situations.

SignalWire is where we finally get to have an opinion. That is my favorite part of it. Somebody has already picked the protocol, arranged the servers, and decided where the media runs. The answer is not a configuration exercise handed to a customer at their most time-constrained moment.

The escape room

Nobody should have to solve an escape room to ship a product.

Here is the sequence I keep watching. A team connects an AI to a conventional CPaaS. It is slow, and it disconnects, and they chase it. They fix that. Then someone convinces them to pump all the call audio into a service they write themselves in Python. Now they are back in microservices, and one of them is dead, and it is not obvious which one. There was a hint about it somewhere, worth coming back to, except five servers have now crashed. And there is a meeting with the person who wanted to buy the thing, and it dies in the middle of the demo.

Meanwhile the actual product sits there rotting. The part of the business they control most, the part that is actually theirs, gets no attention. All of it is going to somebody else's layer.

Engineers love solving problems. That is not the issue. The issue is that with a thousand problems in front of you, choosing which to attack first is its own hard problem. And most of these were never chosen by the team.

TCP is no place for audio

The standard integration pattern handed to new builders right now: a WebRTC gateway feeding a streaming protocol, carrying audio as base64 inside JSON, over TCP.

Everyone who has done voice over IP for a living should feel something when they read that sentence.

TCP is a reliable transport, and reliability is the correct property for streaming Netflix. Netflix buffers. There is no human on the other end waiting to hear you finish a sentence. A conversation has different physics. When there is a storm somewhere in the middle of the internet, TCP does its job. Its job is to hold the line and deliver everything eventually. What arrives is a five-second gap.

The failure that actually breaks conversations is subtler than a gap. Picture a stack assembled from seven providers where one leg carries media over TCP. A barge-in fires, so the system sends a message to cut the audio. Both the message and the audio are stuck behind the same stall. One of them comes free first. Either the cut arrives before the system has said the thing it was supposed to stop saying. Or the audio lands all at once, long after the moment it mattered. The turn-taking model collapses, and nothing in the logs says "TCP."

We made voice over IP work on UDP. TCP is the last resort, for the case where nothing else survives. That means TURN from behind a corporate network that passes nothing unless it looks like a web page. We implement the popular streaming protocol, because compatibility matters and people need a migration path. I do not like it, and I would rather people avoid it. The secret to low latency is getting audio there fast, and that path is one of the specific ways it does not get there fast.

The mermaids on the rocks

The next move in that story is always the same. The pipeline felt slow, so the team reaches for a speech-to-speech model, and the whole problem is declared solved.

This is mermaids luring you onto the rocks.

Speech-to-speech from the large labs is a business model wearing a latency fix. Adopt one and the choices go away. One or two voices if you are lucky. One vendor's roadmap, one vendor's outage window, one vendor's pricing decision.

We support speech-to-speech at SignalWire. It sits in beta and we do not sell it, because I do not want to. The thing it does very well is talk. It does not perform. What a production agent needs most is to execute the right tool at the right moment. For the voice models that remains an afterthought they are still working on. Maybe it gets solved. By the time it does, the economics of running your own will look different than they do today.

The position we take is boring on purpose. Run a cascaded pipeline by default, stay model-agnostic, and treat which model is in the slot as configuration rather than architecture. We do not bet the company on foundation models. We build the system that makes them work, and commoditization of that layer is our tailwind rather than our exposure.

Prompt and pray is not engineering

It took about three days from first prototype to a working prompt-and-pray agent. That part is genuinely fun. You write a paragraph describing a role, you talk to it, and it says funny things back.

Then you want it to do something, and it is a completely different exercise.

Consider handing a pilot who has never flown a sheet of paper with the rules on it, right before you board. That is what a thousand-line prompt full of exceptions is. Exceptions are worse than length. Give a model two instructions that conflict, turn thinking on, and watch it burn five paragraphs deciding which one wins. Models get something like a migraine from being torn.

Everyone reaching for a coding agent has already learned this lesson without naming it. Those agents are on a leash. When one vendor's implementation got out and people rebuilt it from what they saw, the finding was that it was almost all instructions. An April 2026 arXiv analysis of Claude Code put a number on it: 98.4% of the system is scaffolding governing the 1.6% that is model calls. I found that enormously validating, because we had been building on that premise for a while, and it is where the control actually comes from. Left alone, what a model wants to do is emit tokens until the end of time.

There is a double standard here worth naming. We do not ask our code to please not segfault. We do not ask it to please avoid deadlocks and please have no race conditions. When FreeSWITCH went from two cores to four, then eight, then to boxes with 192 of them, we did not hope. We found the races and we fixed them, because that is the job.

The industry's current answer when an AI agent misbehaves is to delete the feature. It keeps getting that one wrong, so take it out. Then the next one. Keep going and you arrive at an agent whose only remaining capability is asking whether the caller has read the FAQ. At that point, record a WAV file, put it on an extension, and point the number at it. That is an IVR. We built those twenty years ago, and they cost less and answer faster.

That is the part I want people to sit with. A large share of what ships today as a voice AI agent is an IVR with higher latency and a bigger invoice.

Which is not an argument for going back to humans reading scripts. Replacing IVRs with people is like building the pyramids by dragging rocks up a hill with a rope. The work worth automating is the work you cannot staff: you cannot keep ten thousand people on payroll against the possibility of a call surge.

Control the scenery, not the whole play

Most of these models were trained into something like an actor's frame. They behave as though they are performing a role. Once you accept that, the programmer's job becomes obvious, and it is not writing better lines.

Control the scenery. Which scene are we in right now? Do not hand the model the entire play and ask it to perform the whole thing end to end. It does not have the capacity, and it will improvise the parts it cannot hold. Give it scene one: introduce yourself, offer these two choices. Let it handle the language. When the choice comes back, wipe the stage, set the next one, put the new actor in place, and let your code decide which scene follows.

There is an older version of this that everyone in the room had already built. An HTML form. You fill in the answers, hit submit, and a different form comes back. In a well-built agent the model becomes the form asking the questions. That is the whole job. Solving the problem is still ours.

The corollary is that a model handed a tool will use it. If an agent can find movies and book restaurants and both toolsets are live at once, it will eventually reach for the wrong one. Scope tools to the step and the question stops being a question.

The last piece is testing, and this is where I embarrassed myself early. It worked. Maybe that was luck, so I ran it three more times, and it worked. The next day it was wrong on every attempt. Something in the sampling shifted, or a cache had been quietly flattering me. "It was working yesterday" is an ancient engineering complaint, and it gets much louder when a probabilistic component sits in the path. Run it a hundred times. Even then, expect one of the hundred to do something strange, and design for that instead of hoping.

Signaling and media were always two problems

Telecom solved this split decades before anyone needed it for AI, and the field promptly forgot.

SS7 switches drove enormous fleets of media servers. We do the same thing now with SIP: a proxy out front deciding where everything goes, media engines behind it moving audio and keeping calls up. Those are two different problems with two different build-outs. A media server needs particular hardware and carefully chosen network settings so a thousand concurrent streams can pass through one box without a single one stalling. Signaling needs almost none of that and cares about entirely different failure modes.

Telling an AI what to do is the signaling layer of an AI product.

So that is where we put the seam. The AI runs inside the media path, where the audio, the timing, and the call state already live. That is why barge-in and endpointing get detected at the audio level rather than over a network hop. Typical conversational turns land in the 1100 to 1500 millisecond range, closer to 800 with an optimized stack. Meanwhile the control surface comes back out over a fast interface, so the developer's code steers. Transfer the call, hold it, drop it into a conference, bridge two people, take it back to AI afterward.

The important property is that the call, the session, and the live media stay anchored in one place for the whole interaction. It is not trunked out and handed across four systems that each hold a different piece of the truth.

We followed the same rule when we shipped chat. The obvious move was a second engine for text, and we did not build one. The chat implementation runs the same code already written for voice. An agent with steps, scoped tools, and real logic behind it works over either channel. That went generally available the Friday before the conference.

I demonstrated it on stage by switching a live session from chat to voice mid-conversation, and my keyboard froze solid in front of several hundred people. I stood there for two minutes explaining that our decimator was decimating my keyboard. It came back. The demo worked. Dangerous Demos was the next day and I appear to have entered early.

Someone asked afterward whether an assistant like Siri or Alexa would be better if it were built this way. Those systems have an unfair and legitimate advantage: dedicated hardware sitting in the room with you. But they are endpoints, not communications channels. The interesting version of that question is what happens when the same capability answers the support line, or picks up from a website. Then it serves everyone at once from the cloud, not only the person holding the device.

The latency number nobody wants

At the ClueCon AI roundtable, a panelist put human conversational turn-taking at 200 milliseconds. That became the target voice AI supposedly has to hit.

I interrupted to disagree, which Adam Kalsey, moderating, immediately pointed out was a live demonstration of endpointing and barge-in. I will take the assist.

The 200 millisecond figure describes face-to-face conversation. It is not what phone calls do. We run a latency analyzer over recorded calls, and humans on the phone routinely take three to four seconds to respond. People think before they speak on a call. Ask someone to leave a voicemail and they slow down further, composing the whole thing before saying any of it. They cannot see who they are talking to, so they are careful.

So we do something almost nobody advertises. We turn the wait knob up, to roughly 75%, on purpose.

Cutting a caller off is worse than being slow. Say a phone number out loud, slowly, the way people do when they want it written down correctly. An agent tuned to answer within 500 milliseconds jumps in after the first three digits and asks what the number is. Now the call is a repair job. Being a beat late reads as thoughtful. Interrupting reads as broken.

While we are on numbers: I keep seeing 500 millisecond claims, then 200, then 100. That is the rules of physics being broken. Getting a response out of a large language model under 300 milliseconds is hard unless you are Groq, who built different hardware to do it. Buy the most expensive card available and run the model on the same machine and 400 of your milliseconds are already committed. Place an ordinary cell phone call and 500 are gone before anyone has said a word. A number like 500 usually comes from a browser talking to the server that answers it, which is not a phone call. And even speech-to-speech models, the fastest architecture available, clock in around 600 to 1000 milliseconds. That is the floor for the approach people give up tool calling and voice choice to get.

The related lesson is about scale, and it is the one people learn latest. At the tenth call, a one-in-a-million failure is a rounding error. At real volume, one in a million is every five minutes. It stops being lightning and starts being weather, and weather calls for lightning rods. That means fallbacks for every dependency, the AI services included. Everyone is running in move-fast mode, and something is broken every morning.

What I might be wrong about

Adam Kalsey, moderating the ClueCon AI roundtable, asked the panel what we would turn out to be wrong about in two years. Fair question to ask a room full of people with strong opinions.

My exposure is the hardware bet. I argue that widespread self-hosted inference stays impractical until somebody breaks the current hardware bottleneck. I also argue we hit an electricity wall before we hit a model wall. I could be wrong. Small models keep getting better, and enough people may find them sufficient that the economics never need to change.

I still think I am right. But being wrong on purpose for a while is how this works. We were wrong about plenty for years, and the way you find out is by bouncing off the problem until it gives.

Let's go mess stuff up

The best thing about ClueCon is that the people who built the plumbing are in the room. Multiple decades each, on pieces of this problem most of the industry has never had to think about. That is the entire point. Nobody has to think about them because those people already did. Give them a hand when you see them.

Start higher up the stack than you think you need to. Not because the lower layers are uninteresting. It is that the interesting part of your product lives higher up, and your customers were never going to know how good your telecom stack was.

This industry is not going to get swept up by one or two companies who have decided they are the ending. The people who build things are still here.

Go mess stuff up.