--- title: 'Ship an AI voice agent that answers calls: a 2026 developer guide' description: >- Learn how to build an AI voice agent for the phone in 2026: telephony, STT, LLM, TTS, turn-taking and latency, with real JavaScript. summary: >- A pillar guide to building a phone AI voice agent: the telephony, STT, LLM, and TTS layers, turn-based vs full-duplex builds, and where Sautikit fits as the voice layer. date: 2026-06-28T00:00:00.000Z type: blog --- An AI voice agent is a program that answers or places a phone call, listens, thinks, and talks back: end to end, no human on the line. Building one in 2026 is mostly about wiring four layers together and keeping latency low enough that the caller never feels the machine. > **TL;DR** > > - A phone AI voice agent has four layers: telephony (place/receive calls + fork audio), speech-to-text, an LLM for reasoning, and text-to-speech. > - Two build paths: **turn-based** (record → STT → LLM → TTS → loop) is simplest; **realtime full-duplex** via `` + a realtime LLM gives natural barge-in. > - Sautikit is the telephony layer any LLM plugs into: a working number costs ~KES 116 and activates instantly, billed in KES. ## The anatomy of a phone AI voice agent Every phone AI voice agent, regardless of framework, is the same pipeline: 1. **Telephony layer**: places and receives the PSTN/SIP call, and exposes the live audio. This is Sautikit: `POST /v1/calls` to dial out, a voice callback to handle inbound, and the `` verb to fork media to your code. 2. **Speech-to-text (STT)**: turns caller audio into text. Deepgram, Whisper, or a realtime provider. 3. **LLM reasoning**: decides what to say and which tools to call. GPT, Claude, or Gemini. 4. **Text-to-speech (TTS)**: turns the LLM's reply back into audio the caller hears. Two cross-cutting concerns make or break the experience: **turn-taking / barge-in** (letting the caller interrupt) and the **latency budget** (how long from "caller stops talking" to "agent starts talking"). > > > Sautikit is the telephony layer only. It is deliberately model-agnostic: you bring any STT, LLM, and TTS. That separation is what gives you control over cost, prompt, and voice, instead of a black-box framework. > > ## The latency budget The caller perceives a natural conversation when the agent responds in under ~800 ms after they stop speaking. That budget is shared: - Endpointing (deciding the caller is done): 100–300 ms - STT final transcript: 100–200 ms - LLM first token: 200–500 ms - TTS first audio frame: 100–300 ms You cannot afford to run these strictly in series and stay under budget. The trick is **streaming**: start LLM inference on partial transcripts, and start TTS on the first LLM tokens. Which path you pick (turn-based or full-duplex) decides how much streaming you get. ## Path A: turn-based (record → STT → LLM → TTS → loop) The simplest agent uses Sautikit voice actions and never touches raw audio. Each turn: `record` the caller, transcribe, ask the LLM, `say` the answer, and redirect back into the loop. Your voice callback returns JSON actions. ```js import express from "express"; const app = express(); app.use(express.urlencoded({ extended: false })); // First turn: greet, then record the caller and post to /turn app.post("/voice", (req, res) => { res.json({ actions: [ { say: { text: "Hi, this is Sauti support. How can I help?", language: "en-KE" } }, { record: { action: "/turn", maxLength: 20, transcribe: true } }, ], }); }); // Each subsequent turn: LLM answers, then we record again app.post("/turn", async (req, res) => { const caller = req.body.transcription_text ?? ""; const reply = await askLLM(caller); // your GPT/Claude/Gemini call res.json({ actions: [ { say: { text: reply, language: "en-KE" } }, { record: { action: "/turn", maxLength: 20, transcribe: true } }, ], }); }); app.listen(3000); ``` This is robust and easy to debug, but the caller cannot interrupt; they must wait for `say` to finish before the next `record` starts. Good for IVR-style flows, form-filling, and confirmations. Not good for open conversation. ## Path B: realtime full-duplex via `` For a natural agent that supports **barge-in**, you fork the live call audio to your own WebSocket with the `` verb and pipe it into a realtime LLM. Return the raw XML form from your voice callback: ```js app.post("/voice", (req, res) => { res.type("application/xml").send( `` ); }); ``` Sautikit then opens a WebSocket to your server (which must accept the `audio.drachtio.org` subprotocol) and streams raw binary PCM frames. The socket is **bidirectional**: send binary PCM frames back on the same socket to play audio into the call. That is where your STT → realtime LLM → TTS pipeline lives, and where barge-in happens: when the caller starts talking, you stop your outbound audio. ```js import { WebSocketServer } from "ws"; const wss = new WebSocketServer({ port: 8080, handleProtocols: () => "audio.drachtio.org" }); wss.on("connection", (ws) => { ws.on("message", (frame, isBinary) => { if (!isBinary) return handleStatusEvent(JSON.parse(frame.toString())); // frame is raw 16 kHz PCM from the caller; feed to STT / realtime LLM pipeline.write(frame); }); // to speak: ws.send(pcmChunk) with a binary Buffer of PCM pipeline.on("audio", (pcm) => ws.send(pcm)); }); ``` For a complete realtime engine using Gemini as the reasoning model, see [Building an AI voice engine with Gemini and Sautikit](/blog/ai-voice-engine-gemini-sautikit). See [voice actions](/developers/concepts/voice-actions) for every `` attribute and status event. ## Placing the call Whichever path you build, the outbound trigger is the same request. Dial the number, and Sautikit hits your voice callback for the actions above. ```js const res = await fetch("https://api.sautikit.com/v1/calls", { method: "POST", headers: { Authorization: `Bearer ${process.env.SAUTIKIT_API_KEY}`, "Content-Type": "application/json", "Idempotency-Key": crypto.randomUUID(), }, body: JSON.stringify({ from: "+254700000000", to: ["+254711111111"] }), }); const { call_id, status, stream_url } = await res.json(); // 202, status "ringing" ``` Inbound is free (KES 0): point a claimed number at your voice callback and answer for nothing. Outbound is KES 3.00/min with a billed per second from the moment the call connects. ## Function calling and tools A useful agent does more than talk: it looks up an order, checks a balance, books a slot. That is standard LLM tool calling: your LLM emits a tool call, your handler runs it (query your DB, hit M-Pesa, etc.), and you feed the result back before generating the spoken reply. The telephony layer does not change; only your reasoning loop grows. Keep tool calls under ~400 ms or speak a filler line ("Let me check that for you") while the tool runs, so the latency budget holds. ## Build-your-own vs managed frameworks Managed platforms like **Vapi, Retell, and Bland** bundle telephony + STT + LLM + TTS behind one config. They are fast to start and fine for prototypes. Teams move to build-your-own when they need: control over the exact prompt and voice, the freedom to swap models, data residency, and predictable pricing. Building on a raw voice API like Sautikit gives you that control, plus KES-denominated, prepaid billing instead of USD-per-minute markup. A working Sautikit number is ~KES 116 and activates instantly, versus the ~KES 5,000 setup plus ~KES 2,500/month a legacy Kenyan provider charges to provision a number. See the full breakdown in [Sautikit vs Vapi, Retell & Bland](/compare/vapi-retell-bland). ## FAQ ### What is an AI voice agent? A program that handles a phone call end to end: it answers or places the call, transcribes what the caller says, uses an LLM to decide a response, and speaks back with TTS, with no human on the line. ### Do I need machine learning expertise to build one? No. You call hosted STT, LLM, and TTS APIs and wire them together. The hard parts (endpointing, barge-in, and staying inside the latency budget) are engineering, not model training. ### Turn-based or realtime: which should I build? Start turn-based (`record` → STT → LLM → `say`) for confirmations, OTPs, and form-filling. Move to realtime `` full-duplex when you need natural interruption and open-ended conversation. ### Can I use any LLM with Sautikit? Yes. Sautikit is the telephony layer only and is model-agnostic: plug in GPT, Claude, Gemini, or a self-hosted model. It just delivers and receives the audio. ### How much does it cost to run? Sautikit charges KES 3.00/min outbound (billed per second from the moment the call connects) and KES 0 inbound. Numbers are from KES 116/month. Your STT/LLM/TTS providers bill separately. ## Get started 1. [Create a Sautikit workspace](/) and claim a number (from KES 116, instant). 2. Top up over **M-Pesa**; no card required. 3. Wire your voice callback to return `say`/`record` for a turn-based agent, or `` for full-duplex. **[Start with Sautikit →](/)**  ·  **[See pricing →](/pricing)**  ·  **[Need SMS, WhatsApp & an agent desk? Helloduty →](https://helloduty.com)** ## Further reading - [Building an AI voice engine with Gemini and Sautikit](/blog/ai-voice-engine-gemini-sautikit) - [Sautikit vs Vapi, Retell & Bland](/compare/vapi-retell-bland) - [Voice actions concepts](/developers/concepts/voice-actions)