Set up HeyGen with an AI agent

name: heygen
description: Use when generating AI avatar videos, creating digital twins from
  video or photos, cloning voices, translating videos, or building video
  generation workflows. Agents should reach for this skill when users request
  video creation, avatar management, voice synthesis, or video translation
  tasks.
metadata:
  mintlify-proj: heygen
  version: "1.0"

HeyGen API Skill

Product summary

HeyGen is an AI video generation platform with a REST API for creating avatar videos, managing digital twins, cloning voices, and translating videos. Agents use it to generate presenter videos from prompts or scripts, create custom avatars from photos or video footage, synthesize speech, and translate videos into 175+ languages with lip-sync. The primary API endpoint is https://api.heygen.com/v3/ (v3 only; v1/v2 are deprecated). Authenticate with X-Api-Key header. Key files: API keys at https://platform.heygen.com/api-keys, usage dashboard at https://platform.heygen.com/usage. CLI available via curl -fsSL https://static.heygen.ai/cli/install.sh | bash. MCP server available for Claude, Cursor, and other agents.

When to use

• Video generation: User requests "make a video," "create a presenter video," "generate an explainer," or "make a talking-head video"
• Avatar creation: User wants to create a digital twin from video footage, a photo avatar, or a prompt-based avatar
• Voice work: User needs voice cloning (instant or professional), text-to-speech, or speech-to-speech conversion
• Video translation: User wants to translate a video into another language with lip-sync
• Batch operations: User needs to generate multiple videos, translations, or lipsyncs at scale
• Workflow automation: Building video generation into a larger application or pipeline

Not for: avatar appearance description (describe the presenter, not the avatar), streaming/live avatars, or cinematic b-roll without a presenter.

Quick reference

Authentication priority (detect in order)

Method	Trigger	Billing	Setup
MCP server	mcp__heygen__* tools visible	Existing HeyGen plan credits	OAuth in agent settings
CLI	HEYGEN_API_KEY env var set	Direct API usage	export HEYGEN_API_KEY=<key>
Raw API	No MCP, no CLI	Direct API usage	X-Api-Key: <key> header

Core endpoints (v3 only)

Task	Endpoint	Method
Create video from prompt	POST /v3/video-agents	Create session, poll for video_id
Create avatar video	POST /v3/videos	Direct render with avatar + script
List/get video status	GET /v3/videos/{video_id}	Poll until completed or failed
Create digital twin	POST /v3/avatars	Upload video, poll training status
Clone voice (instant)	POST /v3/models/audio/voices	mode: "instant", ready in seconds
Text-to-speech	POST /v3/models/audio/tts	Returns WAV URL
Translate video	POST /v3/video-translations	Specify output languages
Batch videos	POST /v3/videos/batches	Up to 100 items, poll batch status
Webhooks	POST /v3/webhooks/endpoints	Register callback for completion

Concurrency limits

Plan	Concurrent workflows	Burst
Pay-As-You-Go	10	None
Enterprise	20	+50 at 1.5× cost

Concurrent jobs: Video Agent sessions, avatar video renders, video translations. Exceeding limit returns 429 Too Many Requests with Retry-After header.

Common status values

Status	Meaning	Action
pending / processing	Job in progress	Keep polling
completed	Done, ready to use	Fetch result URL
failed	Generation failed	Check failure_code and failure_message
waiting_for_input	Agent paused for user reply	Send message via POST /v3/video-agents/{session_id}

Decision guidance

When to use Video Agent vs. Avatar Video

Scenario	Use Video Agent	Use Avatar Video
User has a prompt/idea	✓ Agent scripts, picks avatar, renders	—
User has a written script	✓ Paste script in prompt	✓ Pass script directly
User has a specific avatar	✓ Pass avatar_id	✓ Pass avatar_id
User wants full control	—	✓ Specify every detail
Speed matters	✓ Faster for vague ideas	✓ Faster for known inputs

Default: Use Video Agent (POST /v3/video-agents) for most requests. It handles scripting, avatar selection, and composition. Use Avatar Video (POST /v3/videos) only when the user has already decided on avatar, voice, and script.

When to poll vs. webhook

Scenario	Use polling	Use webhook
Interactive session, user waiting	✓ Poll every 10–30s	—
Batch job, fire-and-forget	—	✓ Register callback_url
Production workflow	—	✓ Async notification
Testing/debugging	✓ Simpler to debug	—

Default: Use webhooks in production. Pass callback_url in creation request. For interactive sessions, poll with exponential backoff.

When to use instant vs. professional voice clone

Scenario	Instant	Professional
One recording, quick turnaround	✓ Ready in seconds	—
Studio-grade quality	—	✓ 20+ min audio, best quality
Free during preview	✓	—
Fine-tuning control	—	✓ Seed, pitch, speed params

Workflow

Generate a video from a prompt (Video Agent)

1. Detect auth: Check for MCP tools, CLI, or HEYGEN_API_KEY env var. Verify with GET /v3/users/me.
2. Gather requirements: Ask user for prompt, duration, tone, avatar preference, language. Keep questions conversational (1–2 items at a time).
3. Craft prompt: Write a clear, specific prompt including duration, tone, media type (motion graphics / AI-generated / stock), and a style block at the end.
4. Create session: POST /v3/video-agents with prompt, optional avatar_id, voice_id, brand_kit_id, callback_url.
5. Poll session: GET /v3/video-agents/{session_id} every 10s until status is completed, failed, or waiting_for_input. Stop on failed or waiting_for_input.
6. Get video_id: Once session has video_id, start polling video.
7. Poll video: GET /v3/videos/{video_id} every 10–30s until status is completed or failed. Typical time: 5–10 minutes.
8. Deliver result: Return video_url from completed video. Provide session URL https://app.heygen.com/video-agent/{session_id} for editing.

Create a digital twin from video

1. Upload video: POST /v3/assets with video file (MP4, WebM, <100 MB, <2K resolution). Get asset_id.
2. Create avatar: POST /v3/avatars with type: "asset_id", asset_id, optionally avatar_group_id (for same person, reuse group).
3. Poll training: GET /v3/avatars/looks/{look_id} every 10s until status is completed or failed. Training usually takes minutes.
4. Collect consent: POST /v3/avatars/{group_id}/consent returns a link (valid 24h). Subject records a short statement on camera.
5. Render video: Once look is completed and consent is cleared, use avatar_id in POST /v3/videos or POST /v3/video-agents.

Clone a voice and generate speech

1. Instant clone (recommended): POST /v3/models/audio/voices with mode: "instant", audio file. Returns voice_id with status PENDING.
2. Poll voice: GET /v3/models/audio/voices/{voice_id} until status is ACTIVE. Usually ready in seconds.
3. Generate speech: POST /v3/models/audio/tts with voice_id, text, language. Returns WAV URL.
4. Use in video: Pass voice_id to POST /v3/videos or POST /v3/video-agents.

Translate a video

1. Create translation: POST /v3/video-translations with video_id, output_languages (array of language codes).
2. Poll translation: GET /v3/video-translations/{video_translation_id} every 10–30s until status is completed or failed.
3. Get result: Fetch video_url, captioned_video_url, subtitle_url from completed translation.
4. Batch multiple: Use POST /v3/video-translations/batches for 100+ videos. Poll GET /v3/video-translations/batches/{batch_id}.

Common gotchas

• Never describe the avatar's appearance in the prompt when avatar_id is set. Say "the selected presenter" instead. Describing it causes avatar mismatch.
• Always include a style block at the end of Video Agent prompts. Without it, visuals drift scene-to-scene.
• Polling has a ceiling. Most videos complete in 1–5 minutes. If a video has been processing >24 hours, contact support with the video_id.
• Don't ask users to paste API keys into chat. Direct them to https://platform.heygen.com/api-keys, have them export HEYGEN_API_KEY=<key>, then continue.
• Respect Retry-After on 429 errors. Back off exponentially. Enterprise plans can enable burst concurrency for higher limits.
• v1/v2 endpoints are deprecated and may fail intermittently. Always use v3 endpoints.
• Asset URLs must be publicly accessible. Private URLs, authentication-required URLs, or files behind redirects will fail with download_failed.
• Voice clones expire if not used. Professional voice clones (PVC) may fail if the source clip is lost or the voice provider revokes access.
• Batch operations are fire-and-forget. Items surface their IDs immediately; you can start fetching finished items while the batch is still processing.
• Brand kits and glossaries must be ready before use. Poll GET /v3/brand-kits/{brand_kit_id} until status is completed before applying.

Verification checklist

Before submitting work:

• Auth verified: Ran GET /v3/users/me or equivalent and got a 200 response with account details.
• Prompt is clear and specific: Includes duration, tone, media type, and a style block (for Video Agent).
• Avatar ID is correct: If using a custom avatar, verified it exists and is trained (preview_image_url is non-null).
• No avatar description in prompt: If avatar_id is set, prompt says "the selected presenter," not "a person in a blue shirt."
• Polling strategy chosen: Decided between polling (interactive) or webhook (batch/production).
• Concurrency checked: Confirmed request won't exceed plan limits (10 for Pay-As-You-Go, 20 for Enterprise).
• Video completed: Polled until status is completed or failed, not just processing.
• Result URL valid: video_url is a presigned HTTPS URL, not null or empty.
• Error handling in place: Code checks failure_code and failure_message on failed videos.
• Session URL captured: For Video Agent, saved session_id immediately for editing link https://app.heygen.com/video-agent/{session_id}.

Resources

Comprehensive navigation: https://developers.heygen.com/llms.txt

Critical pages:

• Quick Start — First video in 3 steps
• For AI Agents — Auth detection, MCP/CLI/API ladder, agent rules
• Error Codes — Full error catalog with troubleshooting
• Usage Limits — Concurrency, rate limits, input caps
• Video Agent — Prompt-to-video workflow
• API Key — Key creation and configuration

---

For additional documentation and navigation, see: https://developers.heygen.com/llms.txt