By the end of this guide you will have a voice AI agent that can answer a phone call or web call, understand what the caller says, reply in a natural voice, and hand off to a person when it should. To build a voice AI agent you combine three pieces: speech-to-text to hear the caller, a language model to decide what to say, and text-to-speech to say it, all running with low enough delay that the conversation feels natural.
The practical choice is how much of that stack you assemble yourself. This guide covers the options, walks through a first build and test, and compares seven tools from our Developer platforms category, with links into Audio and voice for the speech side.
Key takeaways
- A voice agent is speech-to-text, a language model and text-to-speech joined by telephony. You can assemble the parts with a framework, use a platform that bundles them, or buy a managed phone agent.
- Cost is usually per minute and depends on which models you choose, so estimate a call before you commit.
- Interruptions, turn-taking and delay decide whether callers tolerate the agent, so test with real calls early.
- Tell callers they are speaking to an AI where required, get consent before recording, and keep a human handoff for sensitive calls.
What you need before you start
- A clear call goal. One job to start, such as booking appointments, qualifying leads or answering common support questions.
- A script and a source of facts. The questions callers ask, the answers, and the systems the agent may need to read or update.
- A developer, for most options. Frameworks and APIs need code. Some phone-agent platforms offer visual builders.
- A phone number or web entry point. Several platforms provide numbers, or let you bring your own carrier.
- Time. A first test agent can be talking in under a day. Getting it production-ready takes much longer.
- Difficulty. Moderate to hard. Real-time audio is less forgiving than text chat.
- Consent and disclosure. Recording and AI-voice rules vary by country and state, so have someone check the rules for your calls.
Which kind of AI tool fits this job
Pick by how much control you want against how much you want handled for you.
A real-time framework and cloud for full control. LiveKit has an open-source Agents framework for Python and Node.js, in which you assemble an agent from speech-to-text, a language model and text-to-speech components with turn detection and interruption handling. LiveKit Cloud adds hosting, observability, simulations and telephony. OpenAI Agents SDK is an open-source Python framework that includes realtime and voice agents, either through the realtime API or a speech-to-text, agent, text-to-speech pipeline.
A voice-agent platform that bundles the pieces. Vapi combines speech-to-text, a language model and text-to-speech for each agent, lets you choose providers or bring your own model, and handles inbound and outbound calls and web calls. Agents can call your APIs and tools.
Speech building blocks you plug in. Deepgram offers speech-to-text, text-to-speech and a Voice Agent API. Its Flux models add end-of-turn detection and interruption handling. Cartesia offers Sonic text-to-speech, Ink speech-to-text and managed agents. Rime supplies text-to-speech and speech-to-text for production agents and plugs into LiveKit, Pipecat and Vapi.
A managed phone-agent platform for regulated calling. Bland runs its own telephony, transcription, language model and voice stack, with a visual call-flow designer and a per-minute rate that covers the LLM, speech and voices. It lists compliance certifications aimed at healthcare, insurance and finance.
Step 1: Define the call and what counts as success
You finish this step with a one-page spec the whole build can follow.
- Write the single goal of the call and what the agent must collect or do.
- List what the agent must never do, such as giving advice in a regulated area.
- Decide when to hand off to a person, and how.
- Choose measures of success, such as completed bookings or calls that needed a human.
Check it worked by letting a colleague read the spec and describe an ideal call in their own words.
Step 2: Choose the build approach
Pick the level of control that matches your team.
- If you have developers and want control over each component, use a framework such as LiveKit Agents or the OpenAI Agents SDK.
- If you want fewer moving parts, use a platform like Vapi that bundles the pipeline.
- If you want the supplier to run the whole phone stack, consider Bland.
- Estimate cost per minute for your chosen models using the table below, remembering that provider costs are often passed through.
Check it worked by writing the approach in one line, including who builds it.
Step 3: Pick the speech and language models
Latency and voice quality are decided here.
- Choose a speech-to-text model that handles your callers' languages and accents. Deepgram lists dozens of languages with automatic language detection; Rime's speech-to-text covers English and Spanish only.
- Choose a text-to-speech voice. Rime offers voices tuned for expression or for the lowest latency, and Cartesia offers Sonic voices, with instant voice cloning from the Pro plan up.
- Choose a language model, balancing quality against speed. A smaller model often feels better on a phone call.
- Prefer streaming everywhere, so the agent starts speaking before the full answer exists.
Check it worked by recording a short test call and judging whether the voice and delay are acceptable.
Step 4: Write the conversation design
A phone agent needs short turns and clear recovery paths.
- Write a short system prompt: role, goal, tone and limits.
- Keep spoken replies brief. Long answers are tiring to listen to.
- Script how the agent confirms details, such as repeating back a name or date.
- Define tools the agent can call, such as checking a calendar or looking up an order. Vapi and the OpenAI Agents SDK both support tool calling.
- Add a fallback for when it does not understand, and an escalation path to a person.
Check it worked by reading the script aloud with a colleague playing the caller.
Step 5: Connect telephony and tools
Give the agent a way to receive calls and reach your systems.
- Get a phone number, or connect your own carrier. LiveKit Cloud lists telephony with phone numbers, and Vapi agents can make and receive calls on phone numbers.
- For browser or app calls, use the platform's client SDKs.
- Connect the tools the agent needs, and give each the minimum permission required.
- Keep API keys on the server, never in client code.
Check it worked by calling the number and hearing the agent's greeting.
Step 6: Test with real calls, including bad ones
Voice agents fail in ways text bots do not, so test the hard cases.
- Call yourself from different phones and networks.
- Interrupt the agent mid-sentence and talk over it.
- Try accents, background noise, silence and off-topic questions.
- Use simulation tools if the platform has them. LiveKit lists simulations and Vapi lists testing and call simulation.
- Read transcripts and listen to recordings to find where the agent went wrong.
Check it worked by completing your Step 1 goal in several calls in a row, including ones with interruptions.
Step 7: Launch small, monitor and improve
Start with a limited slice of real traffic.
- Route a small share of calls to the agent, with a clear way to reach a person.
- Tell callers they are speaking to an AI where the rules require it, and handle recording consent.
- Watch cost per call, call length, handoff rate and failures. Check platform limits such as concurrency caps.
- Review calls weekly and refine the prompt, tools and voice.
- Have a person review any flow involving health, money or legal advice.
Check it worked by comparing the first week's results with the success measures from Step 1.
Tools for this job compared
| Tool | Good fit for | Pricing | Stated limits |
|---|---|---|---|
| LiveKit | Code-first agents with an open-source framework and cloud hosting | Freemium, paid from $50/mo (checked 24 Oct 2026) | Building an agent requires writing code; per-minute cost depends on the models you choose |
| Vapi | Bundled agent platform for phone and web calls | Paid, from $0.05, usage-based (checked 24 Oct 2026) | No-package tier capped at 4 concurrent calls and 14 days of raw data retention; telephony fees extra |
| Deepgram | Speech-to-text, text-to-speech and a Voice Agent API | Freemium | Does not provide the reasoning LLM; Flux models run only on the /v2/listen endpoint |
| Cartesia | Sonic text-to-speech, Ink speech-to-text and managed agents | Freemium, paid from $5/mo (checked 24 Oct 2026) | Agent per-minute rates exclude LLM usage and phone number costs; requests fail once monthly credits are used if overages are off |
| Rime | Low-latency voices for production agents; self-hosting on Enterprise | Free trial, paid from $0.03, usage-based (checked 24 Oct 2026) | Speech-to-text supports English and Spanish only; custom voices and HIPAA BAA are Enterprise |
| Bland | Managed phone agents with a visual call-flow builder | Freemium, paid from $0.14, usage-based (checked 24 Oct 2026) | Start allows 100 calls per day and 10 concurrent calls; SMS, warm transfers, BAA and SSO are Enterprise only |
| OpenAI Agents SDK | Python framework with realtime and voice agents | Open source | Requires Python 3.10 or newer; model calls are billed by your provider |
Prices and plan limits change, so confirm the current terms on each vendor's site before you commit.
Common mistakes to avoid
- Ignoring latency. Even good answers fail if there is a long pause before each reply. Fix it by streaming speech and text, choosing faster models, and testing over a real phone line.
- Not planning for interruptions. Callers talk over agents, and an agent that keeps reading frustrates them. Fix it by using turn detection and interruption handling, as LiveKit and Deepgram Flux describe, and by testing barge-in.
- Underestimating cost. Per-minute platform fees often exclude model, speech and telephony costs. Fix it by adding up each layer, as in the table, and by running a pilot.
- Skipping disclosure and consent. Recording and AI-voice rules vary by place. Fix it by having a qualified person check them before launch, and by announcing the AI where required.
- Letting the agent handle everything. Sensitive calls need a person. Fix it by defining handoff triggers in Step 1 and testing them.
How we put this together
This is an evidence-based guide built from each tool's listing in our directory, which is compiled from the vendor's own site and pricing page. It is not a review based on using the tools. We don't rank tools, and nobody paid to be included. Some outbound links may be affiliate links; see our disclosure.
Frequently asked questions
Is there a free way to build a voice AI agent?
You can start free on several tools. LiveKit has a Build plan at $0 a month, Cartesia has a $0 Free plan, Vapi gives $5 in free credits, Deepgram's Pay As You Go starts with a free credit, and the OpenAI Agents SDK is open source. Model, speech and telephony usage may still cost money.
How much does a voice AI agent cost per call minute?
It depends on the models and telephony you choose. Vapi lists $0.05 per minute for hosting plus provider costs at cost, Bland's Start plan is $0.14 per minute, and LiveKit's pricing page gives an estimate of about $0.0479 per minute for its own sample. Add up every layer for your design.
Do I need to code to build a voice AI agent?
For most options in this guide, yes. LiveKit, the OpenAI Agents SDK, Vapi and the speech APIs are developer tools. Bland offers a visual call-flow designer, but teams still integrate through its API for programmatic work.
Can a voice AI agent make outbound calls?
Yes. Vapi and Bland both list inbound and outbound calls. Outbound calling is often regulated, so check consent, calling-hour and disclosure rules with a qualified person before you dial.
Can I keep call data private?
Some tools offer self-hosting or private deployment: Deepgram runs in-VPC or self-hosted, and Rime offers VPC or on-prem hosting on Enterprise. Bland lists compliance certifications. Retention and HIPAA features often sit on higher plans, so read the terms before you handle sensitive calls.
Compare more options in our Developer platforms category.
