Set up RetellAI with an AI agent

name: retellai
description: Use when building, testing, deploying, or managing AI voice agents
  and chat agents that handle phone calls, SMS, and web interactions. Reach for
  this skill when you need to create conversational AI agents, configure
  telephony, set up integrations, test agent behavior, or monitor call
  performance.
metadata:
  mintlify-proj: retellai
  version: "1.0"

Retell AI Skill

Product summary

Retell AI is a platform for building, testing, deploying, and monitoring AI voice and chat agents that handle phone calls, SMS, and web interactions. Agents run on four layers: telephony (call routing), speech-to-text (transcription), LLM (response generation), and text-to-speech (voice synthesis). Retell manages all four, so you configure the agent behavior instead of connecting providers.

Key files and concepts:

• Agent types: Single prompt (recommended start) or conversation flow (for fixed-order steps)
• Response engines: Retell LLM (text-based), conversation flow (node-based), or custom LLM (your own model)
• Phone numbers: Retell-managed (US/Canada) or imported via SIP trunking
• API base: https://api.retellai.com/v2/
• Primary docs: https://docs.retellai.com

When to use

Reach for Retell AI when:

• Building agents: Creating a voice agent for customer support, lead qualification, appointment booking, or patient intake
• Testing before deployment: Running simulation tests, batch tests, or web call tests to validate agent behavior
• Deploying calls: Making outbound calls, receiving inbound calls, or embedding voice in web/mobile apps
• Integrating systems: Connecting CRM (HubSpot, Salesforce, Dynamics 365, GoHighLevel, Zoho), helpdesk (Zendesk), or calendar (Calendly, Cal.com) tools
• Monitoring performance: Tracking call metrics, analyzing transcripts, running AI quality assurance, or setting up alerts
• Managing agent versions: Publishing versions, comparing changes, or A/B testing agents
• Handling complex workflows: Running pre-call functions (CRM lookups) or post-call functions (logging, follow-ups)

Quick reference

Agent types decision

Use case	Agent type	When
Most agents (recommended start)	Single prompt	Task fits in one clear prompt; model decides conversation flow
Fixed-order compliance calls	Conversation flow	Call must follow strict step sequence; prompt too long to fit in one
Text-only interactions	Chat agent	SMS, web widget, or API-only (no phone)
Custom model/compliance	Custom LLM	Must run your own model or process data in your infrastructure

Core API endpoints

Task	Endpoint	Method
Create agent	/create-agent	POST
Update agent	/update-agent/{agent_id}	POST
Publish agent	/publish-agent-version/{agent_id}	POST
Make outbound call	/create-phone-call	POST
Get call details	/get-call/{call_id}	GET
Create phone number	/create-phone-number	POST
Create Retell LLM	/create-retell-llm	POST
Create conversation flow	/create-conversation-flow	POST
Register webhook	Dashboard only (as of Aug 2026)	—

LLM model options

Model	Tier	Best for
gpt-6.1-sol	standard, fast, ultrafast	Latest OpenAI; recommended for most agents
gpt-5.5	standard, fast	Balanced cost/performance
claude-5.5-sonnet	standard, fast	Complex reasoning; good for tool use
gemini-3.5-flash	standard, fast	Fast, cost-effective
gpt-realtime-2.1	—	Speech-to-speech (full-duplex); no separate voice needed
gpt-live-1	—	Full-duplex voice with cascading backend text LLM

Voice configuration

Setting	Purpose	Example
voice_id	Platform or custom voice	"11labs_sarah" or cloned voice ID
voice_speed	Speech rate (0.5–2)	1.0 (default), 1.2 (faster)
voice_temperature	Voice stability (0–2)	1.0 (default), 0.5 (more stable)
enable_expressive_mode	Emotion/effect tags	true to enable [happy], [pause] tags
interruption_sensitivity	How easily interrupted (0–1)	1.0 (default), 0.5 (harder to interrupt)
responsiveness	Response speed (0–1)	1.0 (default), 0.8 (slower, more thoughtful)

Dynamic variables (template syntax)

Use {{variable_name}} in prompts, tool inputs, and webhook URLs:

Variable	Available	Example
{{user_number}}	All calls	Caller's phone number
{{agent_number}}	All calls	Your agent's phone number
{{call_id}}	All calls	Unique call identifier
{{call_summary}}	Post-call only	AI-generated call summary
{{call_successful}}	Post-call only	Boolean: call succeeded
{{user_sentiment}}	Post-call only	Caller's sentiment (positive/negative/neutral)
Custom fields	Post-call only	From Post Call Extraction setup

Decision guidance

Single prompt vs. conversation flow

Aspect	Single prompt	Conversation flow
Design effort	Low; describe task once	High; design nodes and transitions
Flexibility	High; model adapts to caller	Lower; fixed node order
Best for	Lead qualification, support, booking	Compliance scripts, patient intake
Context per turn	Full prompt available	Only current node's instructions
When to switch	If working reliably, keep it	Only if single prompt fails or prompt too long

Cascading vs. speech-to-speech models

Model type	Latency	Features	Cost	Use when
Cascading (text LLM + TTS)	200–500ms	Full tools, knowledge base, transfers	Standard	Most agents; full control needed
Speech-to-speech (GPT Realtime)	100–300ms	Native voice, lower latency	Higher	Real-time responsiveness critical
Duplex (GPT Live)	150–400ms	Voice + backend text LLM	Higher	Full-duplex feel with tool support

Testing method selection

Goal	Method	Cost	Time
Iterate on prompt/flow	LLM Playground (text)	~$0.001–0.05/message	Seconds
Catch regressions	Simulation + batch testing	~$0.001–0.05/message	Minutes
Validate voice/interruptions	Web call testing	~$0.10–0.50/call	Minutes
Check transfers/DTMF/carrier	Phone call testing	~$0.01–0.10/call	Minutes

Workflow

Build and deploy a single prompt agent

1. Create the agent

   • Dashboard: Agents → Create Agent → Single Prompt
   • Or API: POST /create-agent with response_engine type retell-llm
   • Set general_prompt (task description), start_speaker (user or agent), begin_message (optional opening)

2. Configure the response engine

   • Pick LLM model (e.g., gpt-6.1-sol), temperature (0 for deterministic, 0.7 for creative)
   • Attach knowledge base if needed (URLs, documents, or text)
   • Add tools: end_call, transfer_call, integration tools, custom functions

3. Set voice and call behavior

   • Pick voice from platform voices or custom provider
   • Tune voice_speed, interruption_sensitivity, responsiveness
   • Configure voicemail detection, DTMF handling, max call duration
   • Enable expressive mode if using platform voice

4. Test the draft

   • LLM Playground: Chat by hand, inspect tool calls
   • Simulation testing: Save test cases, run batch tests
   • Web call: Hear real audio and interruptions
   • No phone number needed; draft agents are free to test

5. Publish the version

   • Dashboard: Version panel → Publish
   • Or API: POST /publish-agent-version/{agent_id} with version number
   • Published versions are immutable; create new version to change

6. Deploy to phone number

   • Buy number: POST /create-phone-number (Retell-managed, US/Canada only)
   • Or import: Connect SIP trunk from Twilio, Telnyx, Vonage, or custom provider
   • Bind agent: Set inbound_agents or outbound_agents with agent_id and version

7. Make calls

   • Inbound: Callers dial your number; agent answers automatically
   • Outbound: POST /create-phone-call with from_number, to_number, agent_id
   • Monitor: Webhooks (call_started, call_ended, call_analyzed) or polling Get Call API

Set up pre-call and post-call functions

1. Open Workflow page in agent editor
2. Add pre-call function (runs before agent speaks)
   • CRM lookup: Search Contact by phone number
   • Load state: Fetch tickets, orders, or account flags
   • Chain functions: Sequential (waits for output) or parallel (runs alongside)
3. Map outputs to dynamic variables so agent and downstream functions can use them
4. Add post-call function (runs after call ends)
   • Log to CRM: Write summary and disposition
   • Create follow-up: Open task or ticket
   • Send SMS: Text confirmation to caller
   • Gate on condition: Only run if call_successful = true, etc.

Test before going live

1. LLM Playground: Chat with agent, inspect each turn
2. Simulation testing: Create test cases with expected outcomes
3. Batch testing: Run all cases, track pass rate
4. Web call: Hear voice, test interruptions, transfers
5. Phone call: Validate DTMF, carrier audio, real transfers

Common gotchas

• Prompt too long: Single prompt agents struggle with 5000+ token prompts. Shorten or switch to conversation flow.
• Tool definitions bloat context: Keep tool descriptions concise (1–2 sentences). Move "when to call" instructions to the main prompt.
• Dynamic variables in webhook URLs: Test webhook URLs with variables using the dashboard Test button; it only works if variables are resolvable at setup time (not call-time-only variables).
• Pre-call functions delay inbound calls: Budget is 1 minute for inbound, 5 minutes for outbound (before dialing). Keep chains shallow; run slow lookups as in-conversation tools instead.
• Published versions are immutable: Create a new version to make changes. Agents can reference "latest" or "latest_published" to auto-update.
• Multi-prompt agents deprecated: Ending 12/15/2026 (no new creation) and 02/01/2027 (no editing). Migrate to single prompt or conversation flow.
• Custom LLM agents can't use Playground or simulation: Only web/phone call testing works. No batch testing or A/B testing.
• Speech-to-speech models don't support tools: GPT Realtime models can't call functions. Use cascading or duplex if you need tool calling.
• Webhook signature verification required: Always verify X-Retell-Signature header in production. Use raw request body (not re-serialized JSON) for HMAC-SHA256.
• Idempotency keys for retries: Use same key, API key, phone numbers, and request body to safely retry call creation without duplicates.

Verification checklist

Before submitting an agent to production:

• Prompt is clear and concise — Describe the whole task; avoid "keep speaking even if interrupted" (use settings instead)
• Tool definitions are minimal — 1–2 sentences per tool; move "when to call" to main prompt
• Voice sounds natural — Test web call; check voice_speed, interruption_sensitivity, responsiveness
• Knowledge base is attached (if needed) — Agent retrieves accurate answers, not hallucinations
• Pre-call functions load critical data — CRM lookup, account flags, etc. run before opening message
• Post-call functions log outcomes — Summary, disposition, follow-ups written to CRM or endpoint
• Simulation tests pass — Batch test covers happy path and edge cases; pass rate ≥ 90%
• Web call test succeeds — Agent handles interruptions, speaks clearly, transfers work
• Phone call test validates telephony — DTMF, transfers, carrier audio work end-to-end
• Webhook signature verified — Production code checks X-Retell-Signature header
• Idempotency keys used — Outbound calls include unique idempotency_key for safe retries
• Concurrency and CPS limits set — Check /get-concurrency and configure CPS for your provider
• Monitoring configured — Webhooks or polling in place to track call status and errors
• Version published — Agent version is published (immutable) before binding to phone number

Resources

Comprehensive navigation: https://docs.retellai.com/llms.txt

Critical docs:

• Quick Start (15 min) — Build and deploy your first agent
• API Reference — All endpoints, request/response schemas
• Single Prompt Best Practices — Prompt engineering, tool design, common patterns

---

For additional documentation and navigation, see: https://docs.retellai.com/llms.txt