---
name: tavus
description: Use when building real-time conversational video AI agents (PALs),
  creating and training custom faces, deploying conversations to web apps,
  integrating tool calling, or managing multi-turn video interactions with
  perception and speech recognition.
metadata:
  mintlify-proj: tavus
  version: "1.0"
---

# Tavus Skill

## Product Summary

Tavus is a platform for building real-time, multimodal conversational video interfaces (CVI). You build a **PAL** (Personified Application Layer) — a configurable AI agent with behavior, knowledge, tools, and pipeline layers — pair it with a **Face** (a photorealistic avatar trained from video or image), and deploy it as a live video conversation. The core API endpoint is `https://tavusapi.com/v2/`. Authenticate with an API key in the `x-api-key` header. Key resources: [API Overview](https://docs.tavus.io/api-reference/overview), [CVI Quickstart](https://docs.tavus.io/sections/conversational-video-interface/quickstart/cvi-quickstart), [PAL Maker](https://maker.tavus.io/dev).

## When to Use

Reach for Tavus when:
- Building a real-time video conversation agent (sales rep, interviewer, customer support, medical intake)
- Creating or training a custom face from video or image (Phoenix-4.5 or Phoenix-4 models)
- Deploying a conversation to a website as a widget, embed, or landing page
- Wiring tool calls into a PAL so it can look things up, write to a CRM, or trigger actions
- Managing multi-turn conversations with perception (visual/audio understanding) and speech recognition
- Storing participant memory across sessions (pinned facts and learned context)
- Setting objectives (workflow steps) or guardrails (behavioral boundaries) for structured conversations

## Quick Reference

### Core API Endpoints

| Operation | Endpoint | Use |
|-----------|----------|-----|
| Create PAL | `POST /v2/pals` | Define behavior, system prompt, pipeline layers |
| Create Face | `POST /v2/faces` | Train from video or image (Phoenix-4.5/4) |
| Create Conversation | `POST /v2/conversations` | Start a live video call session |
| Create Deployment | `POST /v2/deployments` | Publish PAL to widget/embed/landing-page |
| Create Tool | `POST /v2/tools` | Define a function the PAL can call |
| Attach Tool | `POST /v2/pals/{pal_id}/tools` | Wire tool onto a PAL |
| Create Memory Store | `POST /v2/memory-stores` | Store participant context across calls |
| Create Objectives | `POST /v2/objectives` | Define workflow steps and branching |
| Create Guardrails | `POST /v2/guardrails` | Set behavioral boundaries |

### PAL Configuration Fields

| Field | Type | Purpose |
|-------|------|---------|
| `pal_name` | string | Display name |
| `system_prompt` | string | Core behavior and tone (keep under 5,000 tokens) |
| `default_face_id` | string | Stock face ID or trained face ID |
| `pipeline_mode` | string | `full` (perception + STT + LLM + TTS) or `llm_only` |
| `layers.perception.perception_model` | string | `raven-1` for visual/audio understanding |
| `layers.conversational_flow.turn_detection_model` | string | `sparrow-2` (recommended) for turn-taking |
| `layers.llm.model` | string | `tavus-gemma` (default, fastest), `tavus-gpt-sol`, `tavus-gpt-terra` |
| `layers.tts.voice_id` | string | Voice for speech synthesis |
| `objectives_id` | string | Workflow structure (templated conversations) |
| `guardrails_id` | string | Behavioral boundaries |

### Face Training Models

| Model | Input | Speed | Best For |
|-------|-------|-------|----------|
| Phoenix-4.5 | Image or video | Fast (preview in ~1 min) | Portrait, quick turnaround |
| Phoenix-4 | Video only | Medium (3–4 hours) | Full-body, high quality |
| Phoenix-3 | Video only | Legacy | Existing integrations only |

### Deployment Channels

| Channel | Element | Use |
|---------|---------|-----|
| Widget | `<tavus-widget deployment-id="...">` | Floating launcher on existing site |
| Embed | `<tavus-embed deployment-id="...">` | Inline conversation in your layout |
| Landing page | Hosted by Tavus | Standalone link, no website needed |

### Tool Types

| Type | Trigger | Use |
|------|---------|-----|
| LLM tool | User speech | PAL decides to call based on conversation |
| Perception tool | Audio/video stream | Raven detects something and calls tool |
| Post-call action | Conversation end | Run work after call ends (transcript + analysis) |
| System tool | Always available | Built-in (e.g., `end_call`) |

### MCP & CLI

| Surface | Best For | Auth |
|---------|----------|------|
| MCP server (`https://mcp.tavus.io/mcp`) | AI agents building PALs in editor | Browser OAuth or API key |
| CLI (`tavus` command) | Humans at terminal, CI automation | `TAVUS_API_KEY` env var |

## Decision Guidance

### When to Use Full Pipeline vs. LLM-Only

| Scenario | Use Full Pipeline | Use LLM-Only |
|----------|-------------------|--------------|
| Need perception (see/hear user) | ✓ | |
| Need speech recognition (STT) | ✓ | |
| Text-only or external STT | | ✓ |
| Lowest latency critical | | ✓ |
| Multimodal interactions | ✓ | |

### When to Use Objectives vs. Guardrails

| Goal | Use Objectives | Use Guardrails |
|------|----------------|----------------|
| Collect info in defined order | ✓ | |
| Branch based on answers | ✓ | |
| Enforce behavioral rules | | ✓ |
| Flag policy violations | | ✓ |
| Workflow progression | ✓ | |
| Safety/compliance steering | | ✓ |

### When to Use Deployment vs. Direct API

| Scenario | Use Deployment | Use Direct API |
|----------|----------------|----------------|
| Drop into website with one tag | ✓ | |
| No backend needed | ✓ | |
| Full control over UI | | ✓ |
| Programmatic conversation creation | | ✓ |
| Managed call limits/bot protection | ✓ | |

### When to Use MCP vs. CLI

| Scenario | Use MCP | Use CLI |
|----------|---------|---------|
| Agent building PAL in editor | ✓ | |
| Human at terminal | | ✓ |
| CI/CD automation | | ✓ |
| Interactive tool wiring | ✓ | |

## Workflow

### Build and Deploy a PAL (API Path)

1. **Authenticate**: Generate API key in [PAL Maker](https://maker.tavus.io/dev). Send it in `x-api-key` header on every request.

2. **Create a Face** (if using custom):
   - For image: `POST /v2/faces` with `model_name: "phoenix-4.5"`, `train_image_url`, and `voice_id`
   - For video: `POST /v2/faces` with `model_name: "phoenix-4.5"`, `train_video_url`, and `voice_id`
   - Or use a stock face ID (e.g., `rc9cff32ceba` for Anna)

3. **Create a PAL**: `POST /v2/pals` with:
   - `pal_name`, `system_prompt` (clear, under 5k tokens)
   - `default_face_id` (stock or trained)
   - `pipeline_mode: "full"` (for perception + STT)
   - `layers` with perception model, turn-taking model, LLM model, TTS voice
   - Optional: `objectives_id`, `guardrails_id`, `documents` (knowledge base)

4. **Attach Tools** (if needed):
   - `POST /v2/tools` to define each tool (name, description, parameters, delivery)
   - `POST /v2/pals/{pal_id}/tools` to attach to PAL

5. **Create a Conversation**: `POST /v2/conversations` with:
   - `pal_id`
   - `conversational_context` (optional: participant info, session data)
   - `participant_tags` (optional: for memory store)
   - Response includes `conversation_url` to join

6. **Join the Conversation**: Click `conversation_url` or embed it in your app.

### Deploy to Website (Deployment Path)

1. **Create a PAL** (steps 1–4 above)
2. **Create a Deployment**: `POST /v2/deployments` with:
   - `pal_id`
   - `channel: "widget"` or `"embed"` or `"landing-page"`
   - `allowed_origins` (restrict to your domains)
   - `limits` (daily/total call caps)
3. **Add to HTML**:
   ```html
   <tavus-embed deployment-id="YOUR_DEPLOYMENT_ID"></tavus-embed>
   <script src="https://unpkg.com/@tavus/embed"></script>
   ```
4. **No backend needed**: Deployment ID is safe to expose; Tavus manages conversations server-side.

### Wire Tool Calls (API Delivery)

1. **Create Tool**: `POST /v2/tools` with:
   - `origin: "llm"` (LLM calls) or `"vision"` (perception calls)
   - `delivery.type: "api"`
   - `delivery.api.url` (your endpoint)
   - `delivery.api.auth.type` (none, bearer, api_key, basic, oauth2_client_credentials, hmac)

2. **Attach to PAL**: `POST /v2/pals/{pal_id}/tools`

3. **Handle Requests**: Your endpoint receives POST with tool parameters. Return 2xx with result JSON.

4. **Test**: Start a conversation and watch tool fire. Check response handling.

### Build with MCP (Agent-Driven)

1. **Connect MCP client** to `https://mcp.tavus.io/mcp` with API key in `Authorization: Bearer <key>` header.
2. **Agent calls `tavus_pal_create`** with PAL config.
3. **Agent calls `tavus_tool_create`** to define tools.
4. **Agent calls `tavus_pal_tools_attach`** to wire them.
5. **Agent calls `tavus_conversation_create`** to test.
6. **Agent scaffolds handlers** in your codebase for each tool.

## Common Gotchas

- **System prompt too long**: Keep under 5,000 tokens (~20k characters). Longer prompts degrade speed and instruction-following.
- **Forgetting `x-api-key` header**: Every API request requires it. Requests without it return 401.
- **Reusing face IDs incorrectly**: Stock face IDs (e.g., `rc9cff32ceba`) are fixed; trained faces have unique IDs. Don't mix them up.
- **Tool parameters mismatch**: If tool definition doesn't match your handler's request shape, the PAL won't call it correctly. Test with `tavus_conversation_create` first.
- **Memory store not persisting**: Use stable `participant_tags` (e.g., user ID) across calls. Changing the tag creates a new store. Learned memory processes asynchronously after calls end; wait a few minutes before starting the next conversation.
- **Objectives conflicting with system prompt**: If system prompt says "be a tutor" but objectives say "qualify leads," the PAL will be confused. Align them.
- **Deployment ID exposed**: It's safe—it's meant to be public. Protect it with `allowed_origins`, call limits, and bot protection (Cloudflare Turnstile).
- **Face training rejected**: Check image/video requirements (full face visible, facing camera, no accessories, clear speech). Phoenix-4.5 preview has a watermark; tuning removes it.
- **Latency issues**: Use `tavus-gemma` (fastest LLM), `sparrow-2` (turn-taking), and full pipeline. Avoid oversized prompts. Test with Daily's network check.
- **Legacy inline tools**: Don't define tools in `layers.llm.tools` anymore. Create them at `/v2/tools` and attach them. Inline tools are deprecated.

## Verification Checklist

Before submitting a PAL or deployment:

- [ ] API key is valid and in `x-api-key` header
- [ ] PAL `system_prompt` is under 5,000 tokens and clear
- [ ] `default_face_id` is a valid stock or trained face ID
- [ ] `pipeline_mode` matches your use case (full for perception, llm_only for text-only)
- [ ] LLM model is one of: `tavus-gemma`, `tavus-gpt-sol`, `tavus-gpt-terra`
- [ ] Turn-taking model is `sparrow-2` (recommended)
- [ ] All tools have `origin` (llm/vision) and `delivery` configured
- [ ] Tool parameters JSON Schema matches your handler's request shape
- [ ] Objectives (if used) don't conflict with system prompt
- [ ] Guardrails are clear, specific, and testable
- [ ] Deployment has `allowed_origins` set to your domains
- [ ] Deployment has call limits configured
- [ ] Conversation created successfully and `conversation_url` is accessible
- [ ] Test a full conversation: user speaks, PAL responds, tools fire (if any)
- [ ] Memory store persists context across calls (if using `participant_tags`)

## Resources

- **Full docs index**: [https://docs.tavus.io/llms.txt](https://docs.tavus.io/llms.txt) — comprehensive page-by-page navigation for agents
- **API Reference**: [https://docs.tavus.io/api-reference/overview](https://docs.tavus.io/api-reference/overview) — all endpoints and schemas
- **CVI Quickstart**: [https://docs.tavus.io/sections/conversational-video-interface/quickstart/cvi-quickstart](https://docs.tavus.io/sections/conversational-video-interface/quickstart/cvi-quickstart) — create PAL and conversation in minutes
- **PAL Overview**: [https://docs.tavus.io/sections/conversational-video-interface/pal/overview](https://docs.tavus.io/sections/conversational-video-interface/pal/overview) — all PAL fields and layers
- **Deployments**: [https://docs.tavus.io/sections/deployments/overview](https://docs.tavus.io/sections/deployments/overview) — widget, embed, landing page
- **MCP Server**: [https://docs.tavus.io/sections/agent-tools/mcp-server](https://docs.tavus.io/sections/agent-tools/mcp-server) — agent-driven PAL building
- **Prompting Guide**: [https://docs.tavus.io/sections/onboarding-guide/prompting-guide](https://docs.tavus.io/sections/onboarding-guide/prompting-guide) — system prompt best practices

---

> For additional documentation and navigation, see: https://docs.tavus.io/llms.txt