Initial release — OpenMontage: the first open-source agentic video production system
11 production pipelines, 47 tools, 124 agent skills. Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,103 @@
|
||||
---
|
||||
name: heygen
|
||||
description: |
|
||||
[DEPRECATED] Use `create-video` for prompt-based video generation or `avatar-video` for precise avatar/scene control. This legacy skill combines both workflows — the newer focused skills provide clearer guidance.
|
||||
homepage: https://docs.heygen.com/reference/generate-video-agent
|
||||
allowed-tools: mcp__heygen__*
|
||||
metadata:
|
||||
openclaw:
|
||||
requires:
|
||||
env:
|
||||
- HEYGEN_API_KEY
|
||||
primaryEnv: HEYGEN_API_KEY
|
||||
---
|
||||
|
||||
# HeyGen API (Deprecated)
|
||||
|
||||
> **This skill is deprecated.** Use the focused skills instead:
|
||||
> - **`create-video`** — Generate videos from a text prompt (Video Agent API)
|
||||
> - **`avatar-video`** — Build videos with specific avatars, voices, scripts, and scenes (v2 API)
|
||||
|
||||
This skill remains for backward compatibility but will be removed in a future release.
|
||||
|
||||
---
|
||||
|
||||
AI avatar video creation API for generating talking-head videos, explainers, and presentations.
|
||||
|
||||
## Tool Selection
|
||||
|
||||
If HeyGen MCP tools are available (`mcp__heygen__*`), **prefer them** over direct HTTP API calls — they handle authentication and request formatting automatically.
|
||||
|
||||
| Task | MCP Tool | Fallback (Direct API) |
|
||||
|------|----------|----------------------|
|
||||
| Generate video from prompt | `mcp__heygen__generate_video_agent` | `POST /v1/video_agent/generate` |
|
||||
| Check video status / get URL | `mcp__heygen__get_video` | `GET /v2/videos/{video_id}` |
|
||||
| List account videos | `mcp__heygen__list_videos` | `GET /v2/videos` |
|
||||
| Delete a video | `mcp__heygen__delete_video` | `DELETE /v2/videos/{video_id}` |
|
||||
|
||||
If no HeyGen MCP tools are available, use direct HTTP API calls with `X-Api-Key: $HEYGEN_API_KEY` header as documented in the reference files.
|
||||
|
||||
## Default Workflow
|
||||
|
||||
**Prefer Video Agent** for most video requests.
|
||||
Always use [prompt-optimizer.md](references/prompt-optimizer.md) guidelines to structure prompts with scenes, timing, and visual styles.
|
||||
|
||||
**With MCP tools:**
|
||||
1. Write an optimized prompt using [prompt-optimizer.md](references/prompt-optimizer.md) → [visual-styles.md](references/visual-styles.md)
|
||||
2. Call `mcp__heygen__generate_video_agent` with prompt and config (duration_sec, orientation, avatar_id)
|
||||
3. Call `mcp__heygen__get_video` with the returned video_id to poll status and get the download URL
|
||||
|
||||
**Without MCP tools (direct API):**
|
||||
1. Write an optimized prompt using [prompt-optimizer.md](references/prompt-optimizer.md) → [visual-styles.md](references/visual-styles.md)
|
||||
2. `POST /v1/video_agent/generate` — see [video-agent.md](references/video-agent.md)
|
||||
3. `GET /v2/videos/<id>` — see [video-status.md](references/video-status.md)
|
||||
|
||||
Only use v2/video/generate when user explicitly needs:
|
||||
- Exact script without AI modification
|
||||
- Specific voice_id selection
|
||||
- Different avatars/backgrounds per scene
|
||||
- Precise per-scene timing control
|
||||
- Programmatic/batch generation with exact specs
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | MCP Tool | Read |
|
||||
|------|----------|------|
|
||||
| Generate video from prompt (easy) | `mcp__heygen__generate_video_agent` | [prompt-optimizer.md](references/prompt-optimizer.md) → [visual-styles.md](references/visual-styles.md) → [video-agent.md](references/video-agent.md) |
|
||||
| Generate video with precise control | — | [video-generation.md](references/video-generation.md), [avatars.md](references/avatars.md), [voices.md](references/voices.md) |
|
||||
| Check video status / get download URL | `mcp__heygen__get_video` | [video-status.md](references/video-status.md) |
|
||||
| Add captions or text overlays | — | [captions.md](references/captions.md), [text-overlays.md](references/text-overlays.md) |
|
||||
| Transparent video for compositing | — | [video-generation.md](references/video-generation.md) (WebM section) |
|
||||
| Use with Remotion | — | [remotion-integration.md](references/remotion-integration.md) |
|
||||
|
||||
## Reference Files
|
||||
|
||||
### Foundation
|
||||
- [references/authentication.md](references/authentication.md) - API key setup and X-Api-Key header
|
||||
- [references/quota.md](references/quota.md) - Credit system and usage limits
|
||||
- [references/video-status.md](references/video-status.md) - Polling patterns and download URLs
|
||||
- [references/assets.md](references/assets.md) - Uploading images, videos, audio
|
||||
|
||||
### Core Video Creation
|
||||
- [references/avatars.md](references/avatars.md) - Listing avatars, styles, avatar_id selection
|
||||
- [references/voices.md](references/voices.md) - Listing voices, locales, speed/pitch
|
||||
- [references/scripts.md](references/scripts.md) - Writing scripts, pauses, pacing
|
||||
- [references/video-generation.md](references/video-generation.md) - POST /v2/video/generate and multi-scene videos
|
||||
- [references/video-agent.md](references/video-agent.md) - One-shot prompt video generation
|
||||
- [references/prompt-optimizer.md](references/prompt-optimizer.md) - Writing effective Video Agent prompts (core workflow + rules)
|
||||
- [references/visual-styles.md](references/visual-styles.md) - 20 named visual styles with full specs
|
||||
- [references/prompt-examples.md](references/prompt-examples.md) - Full production prompt example + ready-to-use templates
|
||||
- [references/dimensions.md](references/dimensions.md) - Resolution and aspect ratios
|
||||
|
||||
### Video Customization
|
||||
- [references/backgrounds.md](references/backgrounds.md) - Solid colors, images, video backgrounds
|
||||
- [references/text-overlays.md](references/text-overlays.md) - Adding text with fonts and positioning
|
||||
- [references/captions.md](references/captions.md) - Auto-generated captions and subtitles
|
||||
|
||||
### Advanced Features
|
||||
- [references/templates.md](references/templates.md) - Template listing and variable replacement
|
||||
- [references/photo-avatars.md](references/photo-avatars.md) - Creating avatars from photos
|
||||
- [references/webhooks.md](references/webhooks.md) - Webhook endpoints and events
|
||||
|
||||
### Integration
|
||||
- [references/remotion-integration.md](references/remotion-integration.md) - Using HeyGen in Remotion compositions
|
||||
@@ -0,0 +1,339 @@
|
||||
---
|
||||
name: assets
|
||||
description: Uploading images, videos, and audio for use in HeyGen video generation
|
||||
---
|
||||
|
||||
# Asset Upload and Management
|
||||
|
||||
HeyGen allows you to upload custom assets (images, videos, audio) for use in video generation, such as backgrounds, talking photo sources, and custom audio.
|
||||
|
||||
## Upload Flow
|
||||
|
||||
Asset uploads are a single-step process: POST the raw file binary directly to the upload endpoint. The Content-Type header must match the file's MIME type.
|
||||
|
||||
## Uploading an Asset
|
||||
|
||||
**Endpoint:** `POST https://upload.heygen.com/v1/asset`
|
||||
|
||||
### Request
|
||||
|
||||
| Header | Required | Description |
|
||||
|--------|:--------:|-------------|
|
||||
| `X-Api-Key` | ✓ | Your HeyGen API key |
|
||||
| `Content-Type` | ✓ | MIME type of the file (e.g. `image/jpeg`) |
|
||||
|
||||
The request body is the raw binary file data. No JSON or form fields are needed.
|
||||
|
||||
### Response
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `code` | number | Status code (`100` = success) |
|
||||
| `data.id` | string | Unique asset ID for use in video generation |
|
||||
| `data.name` | string | Asset name |
|
||||
| `data.file_type` | string | `image`, `video`, or `audio` |
|
||||
| `data.url` | string | Accessible URL for the uploaded file |
|
||||
| `data.image_key` | string \| null | Key for creating uploaded photo avatars (images only) |
|
||||
| `data.folder_id` | string | Folder ID (empty if not in a folder) |
|
||||
| `data.meta` | string \| null | Asset metadata |
|
||||
| `data.created_ts` | number | Unix timestamp of creation |
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X POST "https://upload.heygen.com/v1/asset" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: image/jpeg" \
|
||||
--data-binary '@./background.jpg'
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
|
||||
interface AssetUploadResponse {
|
||||
code: number;
|
||||
data: {
|
||||
id: string;
|
||||
name: string;
|
||||
file_type: string;
|
||||
url: string;
|
||||
image_key: string | null;
|
||||
folder_id: string;
|
||||
meta: string | null;
|
||||
created_ts: number;
|
||||
};
|
||||
msg: string | null;
|
||||
message: string | null;
|
||||
}
|
||||
|
||||
async function uploadAsset(filePath: string, contentType: string): Promise<AssetUploadResponse["data"]> {
|
||||
const resolvedPath = path.resolve(filePath);
|
||||
const fileBuffer = fs.readFileSync(resolvedPath);
|
||||
|
||||
const response = await fetch("https://upload.heygen.com/v1/asset", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": contentType,
|
||||
},
|
||||
body: fileBuffer,
|
||||
});
|
||||
|
||||
const json: AssetUploadResponse = await response.json();
|
||||
|
||||
if (json.code !== 100) {
|
||||
throw new Error(json.message ?? "Upload failed");
|
||||
}
|
||||
|
||||
return json.data;
|
||||
}
|
||||
|
||||
// Usage
|
||||
const asset = await uploadAsset("./background.jpg", "image/jpeg");
|
||||
console.log(`Uploaded asset: ${asset.id}`);
|
||||
console.log(`Asset URL: ${asset.url}`);
|
||||
```
|
||||
|
||||
### TypeScript (with streams for large files)
|
||||
|
||||
```typescript
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
import { stat } from "fs/promises";
|
||||
|
||||
async function uploadLargeAsset(filePath: string, contentType: string): Promise<AssetUploadResponse["data"]> {
|
||||
const resolvedPath = path.resolve(filePath);
|
||||
const fileStats = await stat(resolvedPath);
|
||||
const fileStream = fs.createReadStream(resolvedPath);
|
||||
|
||||
const response = await fetch("https://upload.heygen.com/v1/asset", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": contentType,
|
||||
"Content-Length": fileStats.size.toString(),
|
||||
},
|
||||
body: fileStream as any,
|
||||
// @ts-ignore - duplex is needed for streaming
|
||||
duplex: "half",
|
||||
});
|
||||
|
||||
const json: AssetUploadResponse = await response.json();
|
||||
|
||||
if (json.code !== 100) {
|
||||
throw new Error(json.message ?? "Upload failed");
|
||||
}
|
||||
|
||||
return json.data;
|
||||
}
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
|
||||
def upload_asset(file_path: str, content_type: str) -> dict:
|
||||
with open(file_path, "rb") as f:
|
||||
response = requests.post(
|
||||
"https://upload.heygen.com/v1/asset",
|
||||
headers={
|
||||
"X-Api-Key": os.environ["HEYGEN_API_KEY"],
|
||||
"Content-Type": content_type
|
||||
},
|
||||
data=f
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
if data.get("code") != 100:
|
||||
raise Exception(data.get("message", "Upload failed"))
|
||||
|
||||
return data["data"]
|
||||
|
||||
|
||||
# Usage
|
||||
asset = upload_asset("./background.jpg", "image/jpeg")
|
||||
print(f"Uploaded asset: {asset['id']}")
|
||||
print(f"Asset URL: {asset['url']}")
|
||||
```
|
||||
|
||||
## Supported Content Types
|
||||
|
||||
| Type | Content-Type | Use Case |
|
||||
|------|--------------|----------|
|
||||
| JPEG | `image/jpeg` | Backgrounds, talking photos |
|
||||
| PNG | `image/png` | Backgrounds, overlays |
|
||||
| MP4 | `video/mp4` | Video backgrounds |
|
||||
| WebM | `video/webm` | Video backgrounds |
|
||||
| MP3 | `audio/mpeg` | Custom audio input |
|
||||
| WAV | `audio/wav` | Custom audio input |
|
||||
|
||||
## Uploading from URL
|
||||
|
||||
If your asset is already hosted online:
|
||||
|
||||
```typescript
|
||||
async function uploadFromUrl(sourceUrl: string, contentType: string): Promise<AssetUploadResponse["data"]> {
|
||||
// 1. Validate and download the file
|
||||
const url = new URL(sourceUrl);
|
||||
if (url.protocol !== "https:") {
|
||||
throw new Error("Only HTTPS URLs are supported");
|
||||
}
|
||||
const sourceResponse = await fetch(sourceUrl);
|
||||
const buffer = Buffer.from(await sourceResponse.arrayBuffer());
|
||||
|
||||
// 2. Upload directly to HeyGen
|
||||
const response = await fetch("https://upload.heygen.com/v1/asset", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": contentType,
|
||||
},
|
||||
body: buffer,
|
||||
});
|
||||
|
||||
const json: AssetUploadResponse = await response.json();
|
||||
|
||||
if (json.code !== 100) {
|
||||
throw new Error(json.message ?? "Upload failed");
|
||||
}
|
||||
|
||||
return json.data;
|
||||
}
|
||||
```
|
||||
|
||||
## Using Uploaded Assets
|
||||
|
||||
### As Background Image
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello, this is a video with a custom background!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "image",
|
||||
url: asset.url, // Use the URL from the upload response
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### As Talking Photo Source
|
||||
|
||||
```typescript
|
||||
const talkingPhotoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "talking_photo",
|
||||
talking_photo_id: asset.id, // Use the ID from the upload response
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello from my talking photo!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### As Audio Input
|
||||
|
||||
```typescript
|
||||
const audioConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "audio",
|
||||
audio_url: asset.url, // Use the URL from the upload response
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
## Complete Upload Workflow
|
||||
|
||||
```typescript
|
||||
async function createVideoWithCustomBackground(
|
||||
backgroundPath: string,
|
||||
script: string
|
||||
): Promise<string> {
|
||||
// 1. Upload background
|
||||
console.log("Uploading background...");
|
||||
const background = await uploadAsset(backgroundPath, "image/jpeg");
|
||||
|
||||
// 2. Create video config
|
||||
const config = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "image",
|
||||
url: background.url,
|
||||
},
|
||||
},
|
||||
],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
};
|
||||
|
||||
// 3. Generate video
|
||||
console.log("Generating video...");
|
||||
const response = await fetch("https://api.heygen.com/v2/video/generate", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(config),
|
||||
});
|
||||
|
||||
const { data } = await response.json();
|
||||
return data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
## Asset Limitations
|
||||
|
||||
- **File size**: 10MB maximum
|
||||
- **Image dimensions**: Recommended to match video dimensions
|
||||
- **Audio duration**: Should match expected video length
|
||||
- **Retention**: Assets may be deleted after a period of inactivity
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Optimize images** - Resize to match video dimensions before uploading
|
||||
2. **Use appropriate formats** - JPEG for photos, PNG for graphics with transparency
|
||||
3. **Validate before upload** - Check file type and size locally first
|
||||
4. **Handle upload errors** - Implement retry logic for failed uploads
|
||||
5. **Cache asset IDs** - Reuse assets across multiple video generations
|
||||
@@ -0,0 +1,232 @@
|
||||
---
|
||||
name: authentication
|
||||
description: API key setup, X-Api-Key header, and authentication patterns for HeyGen
|
||||
---
|
||||
|
||||
# HeyGen Authentication
|
||||
|
||||
All HeyGen API requests require authentication using an API key passed in the `X-Api-Key` header.
|
||||
|
||||
## Getting Your API Key
|
||||
|
||||
1. Go to https://app.heygen.com/settings?from=&nav=API
|
||||
2. Log in if prompted
|
||||
3. Copy your API key
|
||||
|
||||
## Environment Setup
|
||||
|
||||
Store your API key securely as an environment variable:
|
||||
|
||||
```bash
|
||||
export HEYGEN_API_KEY="your-api-key-here"
|
||||
```
|
||||
|
||||
For `.env` files:
|
||||
|
||||
```
|
||||
HEYGEN_API_KEY=your-api-key-here
|
||||
```
|
||||
|
||||
## Making Authenticated Requests
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/avatars" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
### TypeScript/JavaScript (fetch)
|
||||
|
||||
```typescript
|
||||
const response = await fetch("https://api.heygen.com/v2/avatars", {
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
},
|
||||
});
|
||||
const { data } = await response.json();
|
||||
```
|
||||
|
||||
### TypeScript/JavaScript (axios)
|
||||
|
||||
```typescript
|
||||
import axios from "axios";
|
||||
|
||||
const client = axios.create({
|
||||
baseURL: "https://api.heygen.com",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY,
|
||||
},
|
||||
});
|
||||
|
||||
const { data } = await client.get("/v2/avatars");
|
||||
```
|
||||
|
||||
### Python (requests)
|
||||
|
||||
```python
|
||||
import os
|
||||
import requests
|
||||
|
||||
response = requests.get(
|
||||
"https://api.heygen.com/v2/avatars",
|
||||
headers={"X-Api-Key": os.environ["HEYGEN_API_KEY"]}
|
||||
)
|
||||
data = response.json()
|
||||
```
|
||||
|
||||
### Python (httpx)
|
||||
|
||||
```python
|
||||
import os
|
||||
import httpx
|
||||
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
"https://api.heygen.com/v2/avatars",
|
||||
headers={"X-Api-Key": os.environ["HEYGEN_API_KEY"]}
|
||||
)
|
||||
data = response.json()
|
||||
```
|
||||
|
||||
## Creating a Reusable API Client
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
class HeyGenClient {
|
||||
private baseUrl = "https://api.heygen.com";
|
||||
private apiKey: string;
|
||||
|
||||
constructor(apiKey: string) {
|
||||
this.apiKey = apiKey;
|
||||
}
|
||||
|
||||
async request<T>(endpoint: string, options: RequestInit = {}): Promise<T> {
|
||||
const response = await fetch(`${this.baseUrl}${endpoint}`, {
|
||||
...options,
|
||||
headers: {
|
||||
"X-Api-Key": this.apiKey,
|
||||
"Content-Type": "application/json",
|
||||
...options.headers,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.json();
|
||||
throw new Error(error.message || `HTTP ${response.status}`);
|
||||
}
|
||||
|
||||
return response.json();
|
||||
}
|
||||
|
||||
get<T>(endpoint: string): Promise<T> {
|
||||
return this.request<T>(endpoint);
|
||||
}
|
||||
|
||||
post<T>(endpoint: string, body: unknown): Promise<T> {
|
||||
return this.request<T>(endpoint, {
|
||||
method: "POST",
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Usage
|
||||
const client = new HeyGenClient(process.env.HEYGEN_API_KEY!);
|
||||
const avatars = await client.get("/v2/avatars");
|
||||
```
|
||||
|
||||
## API Response Format
|
||||
|
||||
All HeyGen API responses follow this structure:
|
||||
|
||||
```typescript
|
||||
interface ApiResponse<T> {
|
||||
error: null | string;
|
||||
data: T;
|
||||
}
|
||||
```
|
||||
|
||||
Successful response example:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"avatars": [...]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Error response example:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": "Invalid API key",
|
||||
"data": null
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
Common authentication errors:
|
||||
|
||||
| Status Code | Error | Cause |
|
||||
|-------------|-------|-------|
|
||||
| 401 | Invalid API key | API key is missing or incorrect |
|
||||
| 403 | Forbidden | API key doesn't have required permissions |
|
||||
| 429 | Rate limit exceeded | Too many requests |
|
||||
|
||||
### Handling Errors
|
||||
|
||||
```typescript
|
||||
async function makeRequest(endpoint: string) {
|
||||
const response = await fetch(`https://api.heygen.com${endpoint}`, {
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
});
|
||||
|
||||
const json = await response.json();
|
||||
|
||||
if (!response.ok || json.error) {
|
||||
throw new Error(json.error || `HTTP ${response.status}`);
|
||||
}
|
||||
|
||||
return json.data;
|
||||
}
|
||||
```
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
HeyGen enforces rate limits on API requests:
|
||||
- Standard rate limits apply per API key
|
||||
- Some endpoints (like video generation) have stricter limits
|
||||
- Use exponential backoff when receiving 429 errors
|
||||
|
||||
```typescript
|
||||
async function requestWithRetry(
|
||||
fn: () => Promise<Response>,
|
||||
maxRetries = 3
|
||||
): Promise<Response> {
|
||||
for (let i = 0; i < maxRetries; i++) {
|
||||
const response = await fn();
|
||||
|
||||
if (response.status === 429) {
|
||||
const waitTime = Math.pow(2, i) * 1000;
|
||||
await new Promise((resolve) => setTimeout(resolve, waitTime));
|
||||
continue;
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
throw new Error("Max retries exceeded");
|
||||
}
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Never expose API keys in client-side code** - Always make API calls from a backend server
|
||||
2. **Use environment variables** - Don't hardcode API keys in source code
|
||||
3. **Rotate keys periodically** - Generate new API keys regularly
|
||||
4. **Monitor usage** - Check your HeyGen dashboard for unusual activity
|
||||
@@ -0,0 +1,586 @@
|
||||
---
|
||||
name: avatars
|
||||
description: Listing avatars, avatar styles, and avatar_id selection for HeyGen
|
||||
---
|
||||
|
||||
# HeyGen Avatars
|
||||
|
||||
Avatars are the AI-generated presenters in HeyGen videos. You can use public avatars provided by HeyGen or create custom avatars.
|
||||
|
||||
## Previewing Avatars Before Generation
|
||||
|
||||
Always preview avatars before generating a video to ensure they match user preferences. Each avatar has preview URLs that can be opened directly in the browser - no downloading required.
|
||||
|
||||
### List Avatars and Show Previews
|
||||
|
||||
```typescript
|
||||
async function listAndPreviewAvatars(openInBrowser = true): Promise<void> {
|
||||
const response = await fetch("https://api.heygen.com/v2/avatars", {
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
});
|
||||
const { data } = await response.json();
|
||||
|
||||
for (const avatar of data.avatars.slice(0, 5)) {
|
||||
console.log(`\n${avatar.avatar_name} (${avatar.gender})`);
|
||||
console.log(` ID: ${avatar.avatar_id}`);
|
||||
console.log(` Preview: ${avatar.preview_image_url}`);
|
||||
}
|
||||
|
||||
// Preview URLs can be opened directly in any browser
|
||||
for (const avatar of data.avatars.slice(0, 3)) {
|
||||
console.log(`Open in browser: ${avatar.preview_image_url}`);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Workflow: Preview Before Generate
|
||||
|
||||
1. **List available avatars** - get names, genders, and preview URLs
|
||||
2. **Show preview URLs to user** - share `preview_image_url` for visual check
|
||||
3. **User selects** preferred avatar by name or ID
|
||||
4. **Get avatar details** for `default_voice_id`
|
||||
5. **Generate video** with selected avatar
|
||||
|
||||
### Preview Fields in API Response
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `preview_image_url` | Static image of the avatar (JPG) - publicly accessible URL |
|
||||
| `preview_video_url` | Short video clip showing avatar animation |
|
||||
|
||||
Both URLs are publicly accessible - no authentication needed to view.
|
||||
|
||||
## Listing Available Avatars
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/avatars" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface Avatar {
|
||||
avatar_id: string;
|
||||
avatar_name: string;
|
||||
gender: "male" | "female";
|
||||
preview_image_url: string;
|
||||
preview_video_url: string;
|
||||
}
|
||||
|
||||
interface AvatarsResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
avatars: Avatar[];
|
||||
talking_photos: TalkingPhoto[];
|
||||
};
|
||||
}
|
||||
|
||||
async function listAvatars(): Promise<Avatar[]> {
|
||||
const response = await fetch("https://api.heygen.com/v2/avatars", {
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
});
|
||||
|
||||
const json: AvatarsResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data.avatars;
|
||||
}
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
|
||||
def list_avatars() -> list:
|
||||
response = requests.get(
|
||||
"https://api.heygen.com/v2/avatars",
|
||||
headers={"X-Api-Key": os.environ["HEYGEN_API_KEY"]}
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
if data.get("error"):
|
||||
raise Exception(data["error"])
|
||||
|
||||
return data["data"]["avatars"]
|
||||
```
|
||||
|
||||
## Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"avatars": [
|
||||
{
|
||||
"avatar_id": "josh_lite3_20230714",
|
||||
"avatar_name": "Josh",
|
||||
"gender": "male",
|
||||
"preview_image_url": "https://files.heygen.ai/...",
|
||||
"preview_video_url": "https://files.heygen.ai/..."
|
||||
},
|
||||
{
|
||||
"avatar_id": "angela_expressive_20231010",
|
||||
"avatar_name": "Angela",
|
||||
"gender": "female",
|
||||
"preview_image_url": "https://files.heygen.ai/...",
|
||||
"preview_video_url": "https://files.heygen.ai/..."
|
||||
}
|
||||
],
|
||||
"talking_photos": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Avatar Types
|
||||
|
||||
### Public Avatars
|
||||
|
||||
HeyGen provides a library of public avatars that anyone can use:
|
||||
|
||||
```typescript
|
||||
// List only public avatars
|
||||
const avatars = await listAvatars();
|
||||
const publicAvatars = avatars.filter((a) => !a.avatar_id.startsWith("custom_"));
|
||||
```
|
||||
|
||||
### Private/Custom Avatars
|
||||
|
||||
Custom avatars created from your own training footage:
|
||||
|
||||
```typescript
|
||||
const customAvatars = avatars.filter((a) => a.avatar_id.startsWith("custom_"));
|
||||
```
|
||||
|
||||
## Avatar Styles
|
||||
|
||||
Avatars support different rendering styles:
|
||||
|
||||
| Style | Description |
|
||||
|-------|-------------|
|
||||
| `normal` | Full body shot, standard framing |
|
||||
| `closeUp` | Close-up on face, more expressive |
|
||||
| `circle` | Avatar in circular frame (talking head) |
|
||||
| `voice_only` | Audio only, no video rendering |
|
||||
|
||||
### When to Use Each Style
|
||||
|
||||
| Use Case | Recommended Style |
|
||||
|----------|-------------------|
|
||||
| Full-screen presenter video | `normal` |
|
||||
| Personal/intimate content | `closeUp` |
|
||||
| Picture-in-picture overlay | `circle` |
|
||||
| Small corner widget | `circle` |
|
||||
| Podcast/audio content | `voice_only` |
|
||||
| Motion graphics with avatar overlay | `normal` or `closeUp` + transparent bg |
|
||||
|
||||
### Using Avatar Styles
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal", // "normal" | "closeUp" | "circle" | "voice_only"
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello, world!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Circle Style for Talking Heads
|
||||
|
||||
Circle style is ideal for overlay compositions:
|
||||
|
||||
```typescript
|
||||
// Circle avatar for picture-in-picture
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "circle",
|
||||
},
|
||||
voice: { ... },
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#00FF00", // Green for chroma key, or use webm endpoint
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
## Searching and Filtering Avatars
|
||||
|
||||
### By Gender
|
||||
|
||||
```typescript
|
||||
function filterByGender(avatars: Avatar[], gender: "male" | "female"): Avatar[] {
|
||||
return avatars.filter((a) => a.gender === gender);
|
||||
}
|
||||
|
||||
const maleAvatars = filterByGender(avatars, "male");
|
||||
const femaleAvatars = filterByGender(avatars, "female");
|
||||
```
|
||||
|
||||
### By Name
|
||||
|
||||
```typescript
|
||||
function searchByName(avatars: Avatar[], query: string): Avatar[] {
|
||||
const lowerQuery = query.toLowerCase();
|
||||
return avatars.filter((a) =>
|
||||
a.avatar_name.toLowerCase().includes(lowerQuery)
|
||||
);
|
||||
}
|
||||
|
||||
const results = searchByName(avatars, "josh");
|
||||
```
|
||||
|
||||
## Avatar Groups
|
||||
|
||||
Avatars are organized into groups for better management.
|
||||
|
||||
### List Avatar Groups
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/avatar_group.list?include_public=true" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
#### Query Parameters
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `include_public` | bool | false | Include public avatars in results |
|
||||
|
||||
#### TypeScript
|
||||
|
||||
```typescript
|
||||
interface AvatarGroupItem {
|
||||
id: string;
|
||||
name: string;
|
||||
created_at: number;
|
||||
num_looks: number;
|
||||
preview_image: string;
|
||||
group_type: string;
|
||||
train_status: string;
|
||||
default_voice_id: string | null;
|
||||
}
|
||||
|
||||
interface AvatarGroupListResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
avatar_group_list: AvatarGroupItem[];
|
||||
};
|
||||
}
|
||||
|
||||
async function listAvatarGroups(
|
||||
includePublic = true
|
||||
): Promise<AvatarGroupListResponse["data"]> {
|
||||
const params = new URLSearchParams({
|
||||
include_public: includePublic.toString(),
|
||||
});
|
||||
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/avatar_group.list?${params}`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const json: AvatarGroupListResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data;
|
||||
}
|
||||
```
|
||||
|
||||
### Get Avatars in a Group
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/avatar_group/{group_id}/avatars" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
## Using Avatars in Video Generation
|
||||
|
||||
### Basic Avatar Usage
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Welcome to our product demo!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
},
|
||||
],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
};
|
||||
```
|
||||
|
||||
### Multiple Scenes with Different Avatars
|
||||
|
||||
```typescript
|
||||
const multiSceneConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hi, I'm Josh. Let me introduce my colleague.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
},
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "angela_expressive_20231010",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello! I'm Angela. Nice to meet you!",
|
||||
voice_id: "2d5b0e6a8c3f47d9a1b2c3d4e5f60718",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
## Using Avatar's Default Voice
|
||||
|
||||
Many avatars have a `default_voice_id` that's pre-matched for natural results. **This is the recommended approach** rather than manually selecting voices.
|
||||
|
||||
### Recommended Flow
|
||||
|
||||
```
|
||||
1. GET /v2/avatars → Get list of avatar_ids
|
||||
2. GET /v2/avatar/{id}/details → Get default_voice_id for chosen avatar
|
||||
3. POST /v2/video/generate → Use avatar_id + default_voice_id
|
||||
```
|
||||
|
||||
### Get Avatar Details (v2 API)
|
||||
|
||||
Given an `avatar_id`, fetch its details including the default voice:
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/avatar/{avatar_id}/details" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
#### Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"type": "avatar",
|
||||
"id": "josh_lite3_20230714",
|
||||
"name": "Josh",
|
||||
"gender": "male",
|
||||
"preview_image_url": "https://files.heygen.ai/...",
|
||||
"preview_video_url": "https://files.heygen.ai/...",
|
||||
"premium": false,
|
||||
"is_public": true,
|
||||
"default_voice_id": "1bd001e7e50f421d891986aad5158bc8",
|
||||
"tags": ["AVATAR_IV"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### TypeScript
|
||||
|
||||
```typescript
|
||||
interface AvatarDetails {
|
||||
type: "avatar";
|
||||
id: string;
|
||||
name: string;
|
||||
gender: "male" | "female";
|
||||
preview_image_url: string;
|
||||
preview_video_url: string;
|
||||
premium: boolean;
|
||||
is_public: boolean;
|
||||
default_voice_id: string | null;
|
||||
tags: string[];
|
||||
}
|
||||
|
||||
async function getAvatarDetails(avatarId: string): Promise<AvatarDetails> {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/avatar/${avatarId}/details`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const json = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data;
|
||||
}
|
||||
|
||||
// Usage: Get default voice for a known avatar
|
||||
const details = await getAvatarDetails("josh_lite3_20230714");
|
||||
if (details.default_voice_id) {
|
||||
console.log(`Using ${details.name} with default voice: ${details.default_voice_id}`);
|
||||
} else {
|
||||
console.log(`${details.name} has no default voice, select manually`);
|
||||
}
|
||||
```
|
||||
|
||||
#### Complete Example: Generate Video with Any Avatar's Default Voice
|
||||
|
||||
```typescript
|
||||
async function generateWithAvatarDefaultVoice(
|
||||
avatarId: string,
|
||||
script: string
|
||||
): Promise<string> {
|
||||
// 1. Get avatar details to find default voice
|
||||
const avatar = await getAvatarDetails(avatarId);
|
||||
|
||||
if (!avatar.default_voice_id) {
|
||||
throw new Error(`Avatar ${avatar.name} has no default voice`);
|
||||
}
|
||||
|
||||
// 2. Generate video with the avatar's default voice
|
||||
const videoId = await generateVideo({
|
||||
video_inputs: [{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: avatar.id,
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: avatar.default_voice_id,
|
||||
},
|
||||
}],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
});
|
||||
|
||||
return videoId;
|
||||
}
|
||||
```
|
||||
|
||||
### Why Use Default Voice?
|
||||
|
||||
1. **Guaranteed gender match** - Avatar and voice are pre-paired
|
||||
2. **Natural lip sync** - Default voices are optimized for the avatar
|
||||
3. **Simpler code** - No need to fetch and match voices separately
|
||||
4. **Better quality** - HeyGen has tested this combination
|
||||
|
||||
## Selecting the Right Avatar
|
||||
|
||||
### Avatar Categories
|
||||
|
||||
HeyGen avatars fall into distinct categories. Match the category to your use case:
|
||||
|
||||
| Category | Examples | Best For |
|
||||
|----------|----------|----------|
|
||||
| **Business/Professional** | Josh, Angela, Wayne | Corporate videos, product demos, training |
|
||||
| **Casual/Friendly** | Lily, various lifestyle avatars | Social media, informal content |
|
||||
| **Themed/Seasonal** | Holiday-themed, costume avatars | Specific campaigns, seasonal content |
|
||||
| **Expressive** | Avatars with "expressive" in name | Engaging storytelling, dynamic content |
|
||||
|
||||
### Selection Guidelines
|
||||
|
||||
**For business/professional content:**
|
||||
- Choose avatars with neutral attire (business casual or formal)
|
||||
- Avoid themed or seasonal avatars (holiday costumes, casual clothing)
|
||||
- Preview the avatar to verify professional appearance
|
||||
- Consider your audience demographics when selecting gender and appearance
|
||||
|
||||
**For casual/social content:**
|
||||
- More flexibility in avatar choice
|
||||
- Themed avatars can work for specific campaigns
|
||||
- Match avatar energy to content tone
|
||||
|
||||
### Common Mistakes to Avoid
|
||||
|
||||
1. **Using themed avatars for business content** - A holiday-themed avatar looks unprofessional in a product demo
|
||||
2. **Not previewing before generation** - Always check the preview URL to verify appearance
|
||||
3. **Ignoring avatar style** - A `circle` style avatar may not work for full-screen presentations
|
||||
4. **Mismatched voice gender** - Always use the avatar's `default_voice_id` or match genders manually
|
||||
|
||||
### Selection Checklist
|
||||
|
||||
Before generating a video:
|
||||
- [ ] Previewed avatar image/video in browser
|
||||
- [ ] Avatar appearance matches content tone (professional vs casual)
|
||||
- [ ] Avatar style (`normal`, `closeUp`, `circle`) fits the video format
|
||||
- [ ] Voice gender matches avatar gender
|
||||
- [ ] Using `default_voice_id` when available
|
||||
|
||||
## Helper Functions
|
||||
|
||||
### Get Avatar by ID
|
||||
|
||||
```typescript
|
||||
async function getAvatarById(avatarId: string): Promise<Avatar | null> {
|
||||
const avatars = await listAvatars();
|
||||
return avatars.find((a) => a.avatar_id === avatarId) || null;
|
||||
}
|
||||
```
|
||||
|
||||
### Validate Avatar ID
|
||||
|
||||
```typescript
|
||||
async function isValidAvatarId(avatarId: string): Promise<boolean> {
|
||||
const avatar = await getAvatarById(avatarId);
|
||||
return avatar !== null;
|
||||
}
|
||||
```
|
||||
|
||||
### Get Random Avatar
|
||||
|
||||
```typescript
|
||||
async function getRandomAvatar(gender?: "male" | "female"): Promise<Avatar> {
|
||||
let avatars = await listAvatars();
|
||||
|
||||
if (gender) {
|
||||
avatars = avatars.filter((a) => a.gender === gender);
|
||||
}
|
||||
|
||||
const randomIndex = Math.floor(Math.random() * avatars.length);
|
||||
return avatars[randomIndex];
|
||||
}
|
||||
```
|
||||
|
||||
## Common Avatar IDs
|
||||
|
||||
Some commonly used public avatar IDs (availability may vary):
|
||||
|
||||
| Avatar ID | Name | Gender |
|
||||
|-----------|------|--------|
|
||||
| `josh_lite3_20230714` | Josh | Male |
|
||||
| `angela_expressive_20231010` | Angela | Female |
|
||||
| `wayne_20240422` | Wayne | Male |
|
||||
| `lily_20230614` | Lily | Female |
|
||||
|
||||
Always verify avatar availability by calling the list endpoint before using.
|
||||
@@ -0,0 +1,295 @@
|
||||
---
|
||||
name: backgrounds
|
||||
description: Solid colors, images, and video backgrounds for HeyGen videos
|
||||
---
|
||||
|
||||
# Video Backgrounds
|
||||
|
||||
HeyGen supports various background types to customize the appearance of your avatar videos.
|
||||
|
||||
## Background Types
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `color` | Solid color background |
|
||||
| `image` | Static image background |
|
||||
| `video` | Looping video background |
|
||||
|
||||
## Color Backgrounds
|
||||
|
||||
The simplest option - use a solid color:
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello with a colored background!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#FFFFFF", // White background
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Common Color Values
|
||||
|
||||
| Color | Hex Value | Use Case |
|
||||
|-------|-----------|----------|
|
||||
| White | `#FFFFFF` | Clean, professional |
|
||||
| Black | `#000000` | Dramatic, cinematic |
|
||||
| Blue | `#0066CC` | Corporate, trustworthy |
|
||||
| Green | `#00FF00` | Chroma key (for compositing) |
|
||||
| Gray | `#808080` | Neutral, modern |
|
||||
|
||||
### Using Transparent/Green Screen
|
||||
|
||||
For compositing in post-production:
|
||||
|
||||
```typescript
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#00FF00", // Green screen
|
||||
}
|
||||
```
|
||||
|
||||
## Image Backgrounds
|
||||
|
||||
Use a static image as background:
|
||||
|
||||
### From URL
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Check out this custom background!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "image",
|
||||
url: "https://example.com/my-background.jpg",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### From Uploaded Asset
|
||||
|
||||
First upload your image, then use the asset URL:
|
||||
|
||||
```typescript
|
||||
// 1. Upload the image
|
||||
const assetId = await uploadFile("./background.jpg", "image/jpeg");
|
||||
|
||||
// 2. Use in video config
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {...},
|
||||
voice: {...},
|
||||
background: {
|
||||
type: "image",
|
||||
url: `https://files.heygen.ai/asset/${assetId}`,
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Image Requirements
|
||||
|
||||
- **Formats**: JPEG, PNG
|
||||
- **Recommended size**: Match video dimensions (e.g., 1920x1080 for 1080p)
|
||||
- **Aspect ratio**: Should match video aspect ratio
|
||||
- **File size**: Under 10MB recommended
|
||||
|
||||
## Video Backgrounds
|
||||
|
||||
Use a looping video as background:
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Dynamic video background!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "video",
|
||||
url: "https://example.com/background-loop.mp4",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Video Requirements
|
||||
|
||||
- **Format**: MP4 (H.264 codec recommended)
|
||||
- **Looping**: Video will loop if shorter than avatar content
|
||||
- **Audio**: Background video audio is typically muted
|
||||
- **File size**: Under 100MB recommended
|
||||
|
||||
## Different Backgrounds Per Scene
|
||||
|
||||
Use different backgrounds for each scene:
|
||||
|
||||
```typescript
|
||||
const multiBackgroundConfig = {
|
||||
video_inputs: [
|
||||
// Scene 1: Office background
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Let me start with an introduction.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "image",
|
||||
url: "https://example.com/office-bg.jpg",
|
||||
},
|
||||
},
|
||||
// Scene 2: Product showcase
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "closeUp",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Now let me show you our product.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "image",
|
||||
url: "https://example.com/product-bg.jpg",
|
||||
},
|
||||
},
|
||||
// Scene 3: Call to action
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Get started today!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#1a1a2e",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
## Background Helper Functions
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
type BackgroundType = "color" | "image" | "video";
|
||||
|
||||
interface Background {
|
||||
type: BackgroundType;
|
||||
value?: string;
|
||||
url?: string;
|
||||
}
|
||||
|
||||
function createColorBackground(hexColor: string): Background {
|
||||
return { type: "color", value: hexColor };
|
||||
}
|
||||
|
||||
function createImageBackground(imageUrl: string): Background {
|
||||
return { type: "image", url: imageUrl };
|
||||
}
|
||||
|
||||
function createVideoBackground(videoUrl: string): Background {
|
||||
return { type: "video", url: videoUrl };
|
||||
}
|
||||
|
||||
// Preset backgrounds
|
||||
const backgrounds = {
|
||||
white: createColorBackground("#FFFFFF"),
|
||||
black: createColorBackground("#000000"),
|
||||
greenScreen: createColorBackground("#00FF00"),
|
||||
corporate: createColorBackground("#0066CC"),
|
||||
};
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Match dimensions** - Background should match video dimensions
|
||||
2. **Consider avatar position** - Leave space where avatar will appear
|
||||
3. **Use contrasting colors** - Ensure avatar is visible against background
|
||||
4. **Optimize file sizes** - Compress images/videos for faster processing
|
||||
5. **Test with green screen** - For professional post-production workflows
|
||||
6. **Keep backgrounds simple** - Avoid distracting elements behind the avatar
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Background Not Showing
|
||||
|
||||
```typescript
|
||||
// Wrong: missing url/value
|
||||
background: {
|
||||
type: "image"
|
||||
}
|
||||
|
||||
// Correct
|
||||
background: {
|
||||
type: "image",
|
||||
url: "https://example.com/bg.jpg"
|
||||
}
|
||||
```
|
||||
|
||||
### Aspect Ratio Mismatch
|
||||
|
||||
If your background doesn't match the video dimensions, it may be cropped or stretched. Always match your background aspect ratio to your video dimensions:
|
||||
|
||||
```typescript
|
||||
// For 1920x1080 video
|
||||
// Use 1920x1080 background image
|
||||
|
||||
// For 1080x1920 portrait video
|
||||
// Use 1080x1920 background image
|
||||
```
|
||||
|
||||
### Video Background Audio
|
||||
|
||||
Background video audio is typically muted to avoid conflicting with the avatar's voice. If you need background music, add it as a separate audio track in post-production.
|
||||
@@ -0,0 +1,277 @@
|
||||
---
|
||||
name: captions
|
||||
description: Auto-generated captions and subtitle options for HeyGen videos
|
||||
---
|
||||
|
||||
# Video Captions
|
||||
|
||||
HeyGen can automatically generate captions (subtitles) for your videos, improving accessibility and engagement.
|
||||
|
||||
## Enabling Captions
|
||||
|
||||
Captions can be enabled when generating a video:
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello! This video will have automatic captions.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
},
|
||||
],
|
||||
// Caption settings (availability varies by plan)
|
||||
caption: true,
|
||||
};
|
||||
```
|
||||
|
||||
## Caption Configuration Options
|
||||
|
||||
```typescript
|
||||
interface CaptionConfig {
|
||||
// Enable/disable captions
|
||||
enabled: boolean;
|
||||
|
||||
// Caption style
|
||||
style?: {
|
||||
font_family?: string;
|
||||
font_size?: number;
|
||||
font_color?: string;
|
||||
background_color?: string;
|
||||
position?: "top" | "bottom";
|
||||
};
|
||||
|
||||
// Language for caption generation
|
||||
language?: string;
|
||||
}
|
||||
```
|
||||
|
||||
## Caption Styles
|
||||
|
||||
### Basic Captions
|
||||
|
||||
```typescript
|
||||
const config = {
|
||||
video_inputs: [...],
|
||||
caption: true, // Enable with default styling
|
||||
};
|
||||
```
|
||||
|
||||
### Styled Captions
|
||||
|
||||
```typescript
|
||||
const config = {
|
||||
video_inputs: [...],
|
||||
caption: {
|
||||
enabled: true,
|
||||
style: {
|
||||
font_family: "Arial",
|
||||
font_size: 32,
|
||||
font_color: "#FFFFFF",
|
||||
background_color: "rgba(0, 0, 0, 0.7)",
|
||||
position: "bottom",
|
||||
},
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
## Multi-Language Captions
|
||||
|
||||
For videos in different languages, captions are generated based on the voice language:
|
||||
|
||||
```typescript
|
||||
// Spanish video with Spanish captions
|
||||
const spanishConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "¡Hola! Este video tendrá subtítulos en español.",
|
||||
voice_id: "spanish_voice_id",
|
||||
},
|
||||
},
|
||||
],
|
||||
caption: true,
|
||||
};
|
||||
```
|
||||
|
||||
## Working with SRT Files
|
||||
|
||||
### SRT File Format
|
||||
|
||||
Standard SRT format:
|
||||
|
||||
```srt
|
||||
1
|
||||
00:00:00,000 --> 00:00:03,000
|
||||
Hello! This video will have
|
||||
|
||||
2
|
||||
00:00:03,000 --> 00:00:06,000
|
||||
automatic captions generated.
|
||||
|
||||
3
|
||||
00:00:06,000 --> 00:00:09,000
|
||||
They sync with the audio.
|
||||
```
|
||||
|
||||
### Using Custom SRT
|
||||
|
||||
For video translation, you can provide your own SRT:
|
||||
|
||||
```typescript
|
||||
const translationConfig = {
|
||||
input_video_id: "original_video_id",
|
||||
output_languages: ["es-ES", "fr-FR"],
|
||||
srt_key: "path/to/custom.srt", // Custom SRT file
|
||||
srt_role: "input", // "input" or "output"
|
||||
};
|
||||
```
|
||||
|
||||
## Caption Positioning
|
||||
|
||||
### Bottom (Default)
|
||||
|
||||
Standard position for most videos:
|
||||
|
||||
```typescript
|
||||
caption: {
|
||||
enabled: true,
|
||||
style: {
|
||||
position: "bottom"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Top
|
||||
|
||||
For videos where bottom space is occupied:
|
||||
|
||||
```typescript
|
||||
caption: {
|
||||
enabled: true,
|
||||
style: {
|
||||
position: "top"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Accessibility Best Practices
|
||||
|
||||
1. **Always enable captions** - Improves accessibility for deaf/hard-of-hearing viewers
|
||||
2. **Use high contrast** - White text on dark background or vice versa
|
||||
3. **Readable font size** - At least 24px for standard video, larger for mobile
|
||||
4. **Don't cover important content** - Position captions away from key visual elements
|
||||
5. **Sync timing** - Ensure captions match audio timing accurately
|
||||
|
||||
## Caption Helper Functions
|
||||
|
||||
```typescript
|
||||
interface CaptionStyle {
|
||||
font_family: string;
|
||||
font_size: number;
|
||||
font_color: string;
|
||||
background_color: string;
|
||||
position: "top" | "bottom";
|
||||
}
|
||||
|
||||
const captionPresets: Record<string, CaptionStyle> = {
|
||||
default: {
|
||||
font_family: "Arial",
|
||||
font_size: 32,
|
||||
font_color: "#FFFFFF",
|
||||
background_color: "rgba(0, 0, 0, 0.7)",
|
||||
position: "bottom",
|
||||
},
|
||||
minimal: {
|
||||
font_family: "Arial",
|
||||
font_size: 28,
|
||||
font_color: "#FFFFFF",
|
||||
background_color: "transparent",
|
||||
position: "bottom",
|
||||
},
|
||||
bold: {
|
||||
font_family: "Arial",
|
||||
font_size: 36,
|
||||
font_color: "#FFFFFF",
|
||||
background_color: "rgba(0, 0, 0, 0.9)",
|
||||
position: "bottom",
|
||||
},
|
||||
branded: {
|
||||
font_family: "Roboto",
|
||||
font_size: 30,
|
||||
font_color: "#00D1FF",
|
||||
background_color: "rgba(26, 26, 46, 0.9)",
|
||||
position: "bottom",
|
||||
},
|
||||
};
|
||||
|
||||
function createCaptionConfig(preset: keyof typeof captionPresets) {
|
||||
return {
|
||||
enabled: true,
|
||||
style: captionPresets[preset],
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Social Media Caption Considerations
|
||||
|
||||
### TikTok / Instagram Reels
|
||||
|
||||
- Position captions in center or upper portion
|
||||
- Avoid bottom 20% (covered by UI elements)
|
||||
- Use larger font sizes for mobile viewing
|
||||
|
||||
```typescript
|
||||
const socialCaptions = {
|
||||
enabled: true,
|
||||
style: {
|
||||
font_size: 42,
|
||||
position: "top", // Avoid bottom UI elements
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
### YouTube
|
||||
|
||||
- Standard bottom captions work well
|
||||
- YouTube also supports closed captions upload
|
||||
|
||||
### LinkedIn
|
||||
|
||||
- Captions highly recommended (many watch without sound)
|
||||
- Professional styling preferred
|
||||
|
||||
## Limitations
|
||||
|
||||
- Caption styles may be limited depending on your subscription tier
|
||||
- Some advanced caption features may require the web interface
|
||||
- Multi-speaker caption detection may have limited availability
|
||||
- Caption accuracy depends on audio quality and speech clarity
|
||||
|
||||
## Integration with Video Translation
|
||||
|
||||
When using video translation, captions are automatically handled:
|
||||
|
||||
```typescript
|
||||
// Video translation includes caption generation
|
||||
const translationConfig = {
|
||||
input_video_id: "original_video_id",
|
||||
output_languages: ["es-ES"],
|
||||
// Captions generated in target language
|
||||
};
|
||||
```
|
||||
|
||||
See [video-translation.md](video-translation.md) for more details.
|
||||
@@ -0,0 +1,314 @@
|
||||
---
|
||||
name: dimensions
|
||||
description: Resolution options (720p/1080p) and aspect ratios for HeyGen videos
|
||||
---
|
||||
|
||||
# Video Dimensions and Resolution
|
||||
|
||||
HeyGen supports various video dimensions and aspect ratios to fit different platforms and use cases.
|
||||
|
||||
## Standard Resolutions
|
||||
|
||||
### Landscape (16:9)
|
||||
|
||||
| Resolution | Width | Height | Use Case |
|
||||
|------------|-------|--------|----------|
|
||||
| 720p | 1280 | 720 | Standard quality, faster processing |
|
||||
| 1080p | 1920 | 1080 | High quality, most common |
|
||||
|
||||
### Portrait (9:16)
|
||||
|
||||
| Resolution | Width | Height | Use Case |
|
||||
|------------|-------|--------|----------|
|
||||
| 720p | 720 | 1280 | Mobile-first content |
|
||||
| 1080p | 1080 | 1920 | High quality vertical |
|
||||
|
||||
### Square (1:1)
|
||||
|
||||
| Resolution | Width | Height | Use Case |
|
||||
|------------|-------|--------|----------|
|
||||
| 720p | 720 | 720 | Social media posts |
|
||||
| 1080p | 1080 | 1080 | High quality square |
|
||||
|
||||
## Setting Dimensions
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
// Landscape 1080p
|
||||
const landscapeConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: {
|
||||
width: 1920,
|
||||
height: 1080
|
||||
}
|
||||
};
|
||||
|
||||
// Portrait 1080p
|
||||
const portraitConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: {
|
||||
width: 1080,
|
||||
height: 1920
|
||||
}
|
||||
};
|
||||
|
||||
// Square 1080p
|
||||
const squareConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: {
|
||||
width: 1080,
|
||||
height: 1080
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
# Landscape 1080p
|
||||
curl -X POST "https://api.heygen.com/v2/video/generate" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"video_inputs": [...],
|
||||
"dimension": {
|
||||
"width": 1920,
|
||||
"height": 1080
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
## Dimension Helper Functions
|
||||
|
||||
```typescript
|
||||
type AspectRatio = "16:9" | "9:16" | "1:1" | "4:3" | "4:5";
|
||||
type Quality = "720p" | "1080p";
|
||||
|
||||
interface Dimensions {
|
||||
width: number;
|
||||
height: number;
|
||||
}
|
||||
|
||||
function getDimensions(aspectRatio: AspectRatio, quality: Quality): Dimensions {
|
||||
const configs: Record<AspectRatio, Record<Quality, Dimensions>> = {
|
||||
"16:9": {
|
||||
"720p": { width: 1280, height: 720 },
|
||||
"1080p": { width: 1920, height: 1080 },
|
||||
},
|
||||
"9:16": {
|
||||
"720p": { width: 720, height: 1280 },
|
||||
"1080p": { width: 1080, height: 1920 },
|
||||
},
|
||||
"1:1": {
|
||||
"720p": { width: 720, height: 720 },
|
||||
"1080p": { width: 1080, height: 1080 },
|
||||
},
|
||||
"4:3": {
|
||||
"720p": { width: 960, height: 720 },
|
||||
"1080p": { width: 1440, height: 1080 },
|
||||
},
|
||||
"4:5": {
|
||||
"720p": { width: 576, height: 720 },
|
||||
"1080p": { width: 864, height: 1080 },
|
||||
},
|
||||
};
|
||||
|
||||
return configs[aspectRatio][quality];
|
||||
}
|
||||
|
||||
// Usage
|
||||
const youTubeDimensions = getDimensions("16:9", "1080p");
|
||||
const tikTokDimensions = getDimensions("9:16", "1080p");
|
||||
const instagramDimensions = getDimensions("1:1", "1080p");
|
||||
```
|
||||
|
||||
## Platform-Specific Recommendations
|
||||
|
||||
### YouTube
|
||||
|
||||
```typescript
|
||||
const youtubeConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: { width: 1920, height: 1080 }, // 16:9 landscape
|
||||
};
|
||||
```
|
||||
|
||||
### TikTok / Instagram Reels / YouTube Shorts
|
||||
|
||||
```typescript
|
||||
const shortFormConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: { width: 1080, height: 1920 }, // 9:16 portrait
|
||||
};
|
||||
```
|
||||
|
||||
### Instagram Feed Post
|
||||
|
||||
```typescript
|
||||
const instagramFeedConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: { width: 1080, height: 1080 }, // 1:1 square
|
||||
};
|
||||
```
|
||||
|
||||
### LinkedIn
|
||||
|
||||
```typescript
|
||||
const linkedinConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: { width: 1920, height: 1080 }, // 16:9 landscape preferred
|
||||
};
|
||||
```
|
||||
|
||||
### Twitter/X
|
||||
|
||||
```typescript
|
||||
const twitterConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: { width: 1280, height: 720 }, // 16:9, 720p is common
|
||||
};
|
||||
```
|
||||
|
||||
## Avatar IV Dimensions
|
||||
|
||||
For Avatar IV (photo-based avatars), dimensions are set via orientation:
|
||||
|
||||
```typescript
|
||||
type VideoOrientation = "portrait" | "landscape" | "square";
|
||||
|
||||
function getAvatarIVDimensions(orientation: VideoOrientation): Dimensions {
|
||||
switch (orientation) {
|
||||
case "portrait":
|
||||
return { width: 720, height: 1280 };
|
||||
case "landscape":
|
||||
return { width: 1280, height: 720 };
|
||||
case "square":
|
||||
return { width: 720, height: 720 };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Custom Dimensions
|
||||
|
||||
HeyGen supports custom dimensions within limits:
|
||||
|
||||
```typescript
|
||||
const customConfig = {
|
||||
video_inputs: [...],
|
||||
dimension: {
|
||||
width: 1600,
|
||||
height: 900 // Custom 16:9 at non-standard resolution
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### Dimension Constraints
|
||||
|
||||
- **Minimum**: 128px on any side
|
||||
- **Maximum**: 4096px on any side
|
||||
- **Must be even numbers**: Both width and height must be divisible by 2
|
||||
|
||||
```typescript
|
||||
function validateDimensions(width: number, height: number): boolean {
|
||||
if (width < 128 || height < 128) {
|
||||
throw new Error("Dimensions must be at least 128px");
|
||||
}
|
||||
if (width > 4096 || height > 4096) {
|
||||
throw new Error("Dimensions cannot exceed 4096px");
|
||||
}
|
||||
if (width % 2 !== 0 || height % 2 !== 0) {
|
||||
throw new Error("Dimensions must be even numbers");
|
||||
}
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
## Resolution vs. Credit Cost
|
||||
|
||||
Higher resolutions may consume more credits:
|
||||
|
||||
| Resolution | Relative Cost |
|
||||
|------------|---------------|
|
||||
| 720p | Base rate |
|
||||
| 1080p | ~1.5x base rate |
|
||||
|
||||
Consider using 720p for drafts and testing, then 1080p for final output.
|
||||
|
||||
## Background Considerations
|
||||
|
||||
Match background image/video dimensions to your video dimensions:
|
||||
|
||||
```typescript
|
||||
// For 1080p landscape video
|
||||
const config = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {...},
|
||||
voice: {...},
|
||||
background: {
|
||||
type: "image",
|
||||
url: "https://example.com/1920x1080-background.jpg" // Match video dimensions
|
||||
}
|
||||
}
|
||||
],
|
||||
dimension: { width: 1920, height: 1080 }
|
||||
};
|
||||
```
|
||||
|
||||
## Creating a Video Config Factory
|
||||
|
||||
```typescript
|
||||
interface VideoConfigOptions {
|
||||
script: string;
|
||||
avatarId: string;
|
||||
voiceId: string;
|
||||
platform: "youtube" | "tiktok" | "instagram_feed" | "instagram_story" | "linkedin";
|
||||
quality?: "720p" | "1080p";
|
||||
}
|
||||
|
||||
function createVideoConfig(options: VideoConfigOptions) {
|
||||
const platformDimensions: Record<string, Dimensions> = {
|
||||
youtube: { width: 1920, height: 1080 },
|
||||
tiktok: { width: 1080, height: 1920 },
|
||||
instagram_feed: { width: 1080, height: 1080 },
|
||||
instagram_story: { width: 1080, height: 1920 },
|
||||
linkedin: { width: 1920, height: 1080 },
|
||||
};
|
||||
|
||||
const dimension = platformDimensions[options.platform];
|
||||
|
||||
// Scale down for 720p if requested
|
||||
if (options.quality === "720p") {
|
||||
dimension.width = Math.round((dimension.width * 720) / 1080);
|
||||
dimension.height = Math.round((dimension.height * 720) / 1080);
|
||||
}
|
||||
|
||||
return {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: options.avatarId,
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: options.script,
|
||||
voice_id: options.voiceId,
|
||||
},
|
||||
},
|
||||
],
|
||||
dimension,
|
||||
};
|
||||
}
|
||||
|
||||
// Usage
|
||||
const tiktokVideo = createVideoConfig({
|
||||
script: "Hey everyone! Check this out!",
|
||||
avatarId: "josh_lite3_20230714",
|
||||
voiceId: "1bd001e7e50f421d891986aad5158bc8",
|
||||
platform: "tiktok",
|
||||
quality: "1080p",
|
||||
});
|
||||
```
|
||||
@@ -0,0 +1,853 @@
|
||||
---
|
||||
name: photo-avatars
|
||||
description: Creating avatars from photos (talking photos) for HeyGen
|
||||
---
|
||||
|
||||
# Photo Avatars (Talking Photos)
|
||||
|
||||
Photo avatars allow you to animate a static photo and make it speak. This is useful for creating personalized video content from portraits, headshots, or any suitable image.
|
||||
|
||||
## Creating a Photo Avatar from an Uploaded Image
|
||||
|
||||
The workflow is: **Upload Image → Create Avatar Group → Use in Video**
|
||||
|
||||
### Step 1: Upload the Image
|
||||
|
||||
Upload a portrait photo using the asset upload endpoint. The response includes an `image_key` which you'll use in the next step.
|
||||
|
||||
```bash
|
||||
curl -X POST "https://upload.heygen.com/v1/asset" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: image/jpeg" \
|
||||
--data-binary '@./portrait.jpg'
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"code": 100,
|
||||
"data": {
|
||||
"id": "741299e941764988b432ed3a6757878f",
|
||||
"name": "741299e941764988b432ed3a6757878f",
|
||||
"file_type": "image",
|
||||
"url": "https://resource2.heygen.ai/image/.../original.jpg",
|
||||
"image_key": "image/741299e941764988b432ed3a6757878f/original.jpg"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> **Important:** Save the `image_key` field (not the `id`). The `image_key` is the S3 path used to create the photo avatar.
|
||||
|
||||
See [assets.md](assets.md) for full upload details.
|
||||
|
||||
### Step 2: Create Photo Avatar Group
|
||||
|
||||
Use the `image_key` from the upload response to create a photo avatar group. This processes the image and creates a usable photo avatar.
|
||||
|
||||
**Endpoint:** `POST https://api.heygen.com/v2/photo_avatar/avatar_group/create`
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v2/photo_avatar/avatar_group/create" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"image_key": "image/741299e941764988b432ed3a6757878f/original.jpg",
|
||||
"name": "My Photo Avatar"
|
||||
}'
|
||||
```
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `image_key` | string | ✓ | S3 image key from upload response |
|
||||
| `name` | string | ✓ | Display name for the avatar |
|
||||
| `generation_id` | string | | If using AI-generated photo (see below) |
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"id": "045c260bc0364727b2cbe50442c3a5bf",
|
||||
"image_url": "https://files2.heygen.ai/...",
|
||||
"created_at": 1771798135.777256,
|
||||
"name": "My Photo Avatar",
|
||||
"status": "pending",
|
||||
"group_id": "045c260bc0364727b2cbe50442c3a5bf",
|
||||
"is_motion": false,
|
||||
"business_type": "uploaded"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `id` (same as `group_id`) is your `talking_photo_id` for video generation.
|
||||
|
||||
### Step 3: Wait for Processing
|
||||
|
||||
The photo avatar starts with `status: "pending"` and transitions to `"completed"` within seconds. Poll the status endpoint:
|
||||
|
||||
**Endpoint:** `GET https://api.heygen.com/v2/photo_avatar/{id}`
|
||||
|
||||
```bash
|
||||
curl "https://api.heygen.com/v2/photo_avatar/045c260bc0364727b2cbe50442c3a5bf" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
Wait until `status` is `"completed"` before using in video generation.
|
||||
|
||||
### Step 4: Use in Video Generation
|
||||
|
||||
Use the photo avatar `id` as `talking_photo_id`:
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "talking_photo",
|
||||
talking_photo_id: "045c260bc0364727b2cbe50442c3a5bf",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello! This is my photo avatar speaking.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
},
|
||||
],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
};
|
||||
```
|
||||
|
||||
## TypeScript: Complete Workflow
|
||||
|
||||
```typescript
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
|
||||
interface AssetUploadResponse {
|
||||
code: number;
|
||||
data: {
|
||||
id: string;
|
||||
image_key: string;
|
||||
url: string;
|
||||
};
|
||||
}
|
||||
|
||||
interface PhotoAvatarResponse {
|
||||
error: string | null;
|
||||
data: {
|
||||
id: string;
|
||||
group_id: string;
|
||||
image_url: string;
|
||||
name: string;
|
||||
status: string;
|
||||
is_motion: boolean;
|
||||
business_type: string;
|
||||
};
|
||||
}
|
||||
|
||||
async function createPhotoAvatar(
|
||||
imagePath: string,
|
||||
name: string
|
||||
): Promise<string> {
|
||||
// 1. Upload image
|
||||
const resolvedPath = path.resolve(imagePath);
|
||||
const fileBuffer = fs.readFileSync(resolvedPath);
|
||||
const uploadResponse = await fetch("https://upload.heygen.com/v1/asset", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "image/jpeg",
|
||||
},
|
||||
body: fileBuffer,
|
||||
});
|
||||
|
||||
const uploadJson: AssetUploadResponse = await uploadResponse.json();
|
||||
if (uploadJson.code !== 100) {
|
||||
throw new Error("Upload failed");
|
||||
}
|
||||
|
||||
const imageKey = uploadJson.data.image_key;
|
||||
|
||||
// 2. Create avatar group
|
||||
const createResponse = await fetch(
|
||||
"https://api.heygen.com/v2/photo_avatar/avatar_group/create",
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({ image_key: imageKey, name }),
|
||||
}
|
||||
);
|
||||
|
||||
const createJson: PhotoAvatarResponse = await createResponse.json();
|
||||
if (createJson.error) {
|
||||
throw new Error(createJson.error);
|
||||
}
|
||||
|
||||
const photoAvatarId = createJson.data.id;
|
||||
|
||||
// 3. Wait for processing
|
||||
await waitForPhotoAvatar(photoAvatarId);
|
||||
|
||||
return photoAvatarId;
|
||||
}
|
||||
|
||||
async function waitForPhotoAvatar(id: string): Promise<void> {
|
||||
for (let i = 0; i < 30; i++) {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/photo_avatar/${id}`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const json: PhotoAvatarResponse = await response.json();
|
||||
|
||||
if (json.data.status === "completed") return;
|
||||
if (json.data.status === "failed") {
|
||||
throw new Error("Photo avatar processing failed");
|
||||
}
|
||||
|
||||
await new Promise((r) => setTimeout(r, 2000));
|
||||
}
|
||||
|
||||
throw new Error("Photo avatar processing timed out");
|
||||
}
|
||||
|
||||
async function createVideoFromPhoto(
|
||||
photoPath: string,
|
||||
script: string,
|
||||
voiceId: string
|
||||
): Promise<string> {
|
||||
// 1. Create photo avatar
|
||||
const talkingPhotoId = await createPhotoAvatar(photoPath, "Video Avatar");
|
||||
|
||||
// 2. Generate video
|
||||
const response = await fetch("https://api.heygen.com/v2/video/generate", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "talking_photo",
|
||||
talking_photo_id: talkingPhotoId,
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: voiceId,
|
||||
},
|
||||
},
|
||||
],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
}),
|
||||
});
|
||||
|
||||
const { data } = await response.json();
|
||||
return data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
## Python: Complete Workflow
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
import time
|
||||
|
||||
def create_photo_avatar(image_path: str, name: str) -> str:
|
||||
api_key = os.environ["HEYGEN_API_KEY"]
|
||||
|
||||
# 1. Upload image
|
||||
with open(image_path, "rb") as f:
|
||||
upload_resp = requests.post(
|
||||
"https://upload.heygen.com/v1/asset",
|
||||
headers={
|
||||
"X-Api-Key": api_key,
|
||||
"Content-Type": "image/jpeg",
|
||||
},
|
||||
data=f,
|
||||
)
|
||||
|
||||
upload_data = upload_resp.json()
|
||||
if upload_data.get("code") != 100:
|
||||
raise Exception("Upload failed")
|
||||
|
||||
image_key = upload_data["data"]["image_key"]
|
||||
|
||||
# 2. Create avatar group
|
||||
create_resp = requests.post(
|
||||
"https://api.heygen.com/v2/photo_avatar/avatar_group/create",
|
||||
headers={
|
||||
"X-Api-Key": api_key,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json={"image_key": image_key, "name": name},
|
||||
)
|
||||
|
||||
create_data = create_resp.json()
|
||||
if create_data.get("error"):
|
||||
raise Exception(create_data["error"])
|
||||
|
||||
photo_avatar_id = create_data["data"]["id"]
|
||||
|
||||
# 3. Wait for processing
|
||||
for _ in range(30):
|
||||
status_resp = requests.get(
|
||||
f"https://api.heygen.com/v2/photo_avatar/{photo_avatar_id}",
|
||||
headers={"X-Api-Key": api_key},
|
||||
)
|
||||
status = status_resp.json()["data"]["status"]
|
||||
if status == "completed":
|
||||
return photo_avatar_id
|
||||
if status == "failed":
|
||||
raise Exception("Photo avatar processing failed")
|
||||
time.sleep(2)
|
||||
|
||||
raise Exception("Photo avatar processing timed out")
|
||||
```
|
||||
|
||||
## Listing Existing Talking Photos
|
||||
|
||||
Retrieve all talking photos in your account:
|
||||
|
||||
**Endpoint:** `GET https://api.heygen.com/v1/talking_photo.list`
|
||||
|
||||
```bash
|
||||
curl "https://api.heygen.com/v1/talking_photo.list" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"code": 100,
|
||||
"data": [
|
||||
{
|
||||
"id": "ef0ed70f72c6497793e5e36e434d2aea",
|
||||
"image_url": "https://files2.heygen.ai/talking_photo/.../image.WEBP",
|
||||
"circle_image": ""
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Each `id` can be used as `talking_photo_id` in video generation.
|
||||
|
||||
## Adding Photos to an Existing Group
|
||||
|
||||
Add additional photo looks to an existing avatar group:
|
||||
|
||||
**Endpoint:** `POST https://api.heygen.com/v2/photo_avatar/avatar_group/add`
|
||||
|
||||
```typescript
|
||||
async function addPhotosToGroup(
|
||||
groupId: string,
|
||||
imageKeys: string[],
|
||||
name: string
|
||||
): Promise<void> {
|
||||
const response = await fetch(
|
||||
"https://api.heygen.com/v2/photo_avatar/avatar_group/add",
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
group_id: groupId,
|
||||
image_keys: imageKeys,
|
||||
name,
|
||||
}),
|
||||
}
|
||||
);
|
||||
|
||||
const json = await response.json();
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Training a Photo Avatar Group
|
||||
|
||||
Train the avatar group for improved animation quality:
|
||||
|
||||
**Endpoint:** `POST https://api.heygen.com/v2/photo_avatar/train`
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v2/photo_avatar/train" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"group_id": "045c260bc0364727b2cbe50442c3a5bf"}'
|
||||
```
|
||||
|
||||
Check training status:
|
||||
|
||||
**Endpoint:** `GET https://api.heygen.com/v2/photo_avatar/train/status/{group_id}`
|
||||
|
||||
## Avatar IV Video Generation
|
||||
|
||||
Avatar IV is HeyGen's latest photo avatar technology with improved quality and natural motion. It generates a video directly from an uploaded image, bypassing the avatar group creation step.
|
||||
|
||||
**Endpoint:** `POST https://api.heygen.com/v2/video/av4/generate`
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v2/video/av4/generate" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"image_key": "image/741299e941764988b432ed3a6757878f/original.jpg",
|
||||
"script": "Hello! This is Avatar IV with enhanced quality.",
|
||||
"voice_id": "1bd001e7e50f421d891986aad5158bc8",
|
||||
"video_orientation": "landscape",
|
||||
"video_title": "My Avatar IV Video"
|
||||
}'
|
||||
```
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `image_key` | string | ✓ | S3 image key from asset upload |
|
||||
| `script` | string | ✓ | Text for the avatar to speak |
|
||||
| `voice_id` | string | ✓ | Voice to use |
|
||||
| `video_orientation` | string | | `"portrait"`, `"landscape"`, or `"square"` |
|
||||
| `video_title` | string | | Title for the video |
|
||||
| `fit` | string | | `"cover"` or `"contain"` |
|
||||
| `custom_motion_prompt` | string | | Motion/expression description |
|
||||
| `enhance_custom_motion_prompt` | boolean | | Enhance the motion prompt with AI |
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface AvatarIVRequest {
|
||||
image_key: string;
|
||||
script: string;
|
||||
voice_id: string;
|
||||
video_orientation?: "portrait" | "landscape" | "square";
|
||||
video_title?: string;
|
||||
fit?: "cover" | "contain";
|
||||
custom_motion_prompt?: string;
|
||||
enhance_custom_motion_prompt?: boolean;
|
||||
}
|
||||
|
||||
interface AvatarIVResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
video_id: string;
|
||||
};
|
||||
}
|
||||
|
||||
async function generateAvatarIVVideo(
|
||||
config: AvatarIVRequest
|
||||
): Promise<string> {
|
||||
const response = await fetch(
|
||||
"https://api.heygen.com/v2/video/av4/generate",
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(config),
|
||||
}
|
||||
);
|
||||
|
||||
const json: AvatarIVResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
### Avatar IV Options
|
||||
|
||||
| Orientation | Dimensions | Use Case |
|
||||
|-------------|------------|----------|
|
||||
| `portrait` | 720x1280 | TikTok, Stories |
|
||||
| `landscape` | 1280x720 | YouTube, Web |
|
||||
| `square` | 720x720 | Instagram Feed |
|
||||
|
||||
| Fit | Description |
|
||||
|-----|-------------|
|
||||
| `cover` | Fill the frame, may crop edges |
|
||||
| `contain` | Fit entire image, may show background |
|
||||
|
||||
### Custom Motion Prompts
|
||||
|
||||
```typescript
|
||||
const videoId = await generateAvatarIVVideo({
|
||||
image_key: "image/.../original.jpg",
|
||||
script: "Let me tell you about our product.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
custom_motion_prompt: "nodding head and smiling",
|
||||
enhance_custom_motion_prompt: true,
|
||||
});
|
||||
```
|
||||
|
||||
## Generating AI Photo Avatars
|
||||
|
||||
Generate synthetic photo avatars from text descriptions instead of uploading a photo.
|
||||
|
||||
**Endpoint:** `POST https://api.heygen.com/v2/photo_avatar/photo/generate`
|
||||
|
||||
> **IMPORTANT: All 8 fields are REQUIRED.** The API will reject requests missing any field.
|
||||
> When a user asks to "generate an AI avatar of a professional man", you need to ask for or select values for ALL fields below.
|
||||
|
||||
### Required Fields (ALL must be provided)
|
||||
|
||||
| Field | Type | Allowed Values |
|
||||
|-------|------|----------------|
|
||||
| `name` | string | Name for the generated avatar |
|
||||
| `age` | enum | `"Young Adult"`, `"Early Middle Age"`, `"Late Middle Age"`, `"Senior"`, `"Unspecified"` |
|
||||
| `gender` | enum | `"Woman"`, `"Man"`, `"Unspecified"` |
|
||||
| `ethnicity` | enum | `"White"`, `"Black"`, `"Asian American"`, `"East Asian"`, `"South East Asian"`, `"South Asian"`, `"Middle Eastern"`, `"Pacific"`, `"Hispanic"`, `"Unspecified"` |
|
||||
| `orientation` | enum | `"square"`, `"horizontal"`, `"vertical"` |
|
||||
| `pose` | enum | `"half_body"`, `"close_up"`, `"full_body"` |
|
||||
| `style` | enum | `"Realistic"`, `"Pixar"`, `"Cinematic"`, `"Vintage"`, `"Noir"`, `"Cyberpunk"`, `"Unspecified"` |
|
||||
| `appearance` | string | Text prompt describing appearance (clothing, mood, lighting, etc). Max 1000 chars |
|
||||
|
||||
### curl Example
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v2/photo_avatar/photo/generate" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "Sarah Product Demo",
|
||||
"age": "Young Adult",
|
||||
"gender": "Woman",
|
||||
"ethnicity": "White",
|
||||
"orientation": "horizontal",
|
||||
"pose": "half_body",
|
||||
"style": "Realistic",
|
||||
"appearance": "Professional woman with a friendly smile, wearing a navy blue blazer over a white blouse, soft studio lighting, clean neutral background"
|
||||
}'
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"generation_id": "6a7f7f2795de4599bec7cf1e06babe30"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Check Generation Status
|
||||
|
||||
**Endpoint:** `GET https://api.heygen.com/v2/photo_avatar/generation/{generation_id}`
|
||||
|
||||
The response includes multiple generated images to choose from:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"id": "6a7f7f2795de4599bec7cf1e06babe30",
|
||||
"status": "success",
|
||||
"image_url_list": [
|
||||
"https://resource2.heygen.ai/photo_generation/.../image1.jpg",
|
||||
"https://resource2.heygen.ai/photo_generation/.../image2.jpg",
|
||||
"https://resource2.heygen.ai/photo_generation/.../image3.jpg",
|
||||
"https://resource2.heygen.ai/photo_generation/.../image4.jpg"
|
||||
],
|
||||
"image_key_list": [
|
||||
"photo_generation/.../image1.jpg",
|
||||
"photo_generation/.../image2.jpg",
|
||||
"photo_generation/.../image3.jpg",
|
||||
"photo_generation/.../image4.jpg"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface GeneratePhotoAvatarRequest {
|
||||
name: string;
|
||||
age: "Young Adult" | "Early Middle Age" | "Late Middle Age" | "Senior" | "Unspecified";
|
||||
gender: "Woman" | "Man" | "Unspecified";
|
||||
ethnicity: "White" | "Black" | "Asian American" | "East Asian" | "South East Asian" | "South Asian" | "Middle Eastern" | "Pacific" | "Hispanic" | "Unspecified";
|
||||
orientation: "square" | "horizontal" | "vertical";
|
||||
pose: "half_body" | "close_up" | "full_body";
|
||||
style: "Realistic" | "Pixar" | "Cinematic" | "Vintage" | "Noir" | "Cyberpunk" | "Unspecified";
|
||||
appearance: string;
|
||||
}
|
||||
|
||||
interface GeneratePhotoAvatarResponse {
|
||||
error: string | null;
|
||||
data: {
|
||||
generation_id: string;
|
||||
};
|
||||
}
|
||||
|
||||
interface PhotoGenerationStatus {
|
||||
error: string | null;
|
||||
data: {
|
||||
id: string;
|
||||
status: "pending" | "processing" | "success" | "failed";
|
||||
msg: string | null;
|
||||
image_url_list?: string[];
|
||||
image_key_list?: string[];
|
||||
};
|
||||
}
|
||||
|
||||
async function generatePhotoAvatar(
|
||||
config: GeneratePhotoAvatarRequest
|
||||
): Promise<string> {
|
||||
const response = await fetch(
|
||||
"https://api.heygen.com/v2/photo_avatar/photo/generate",
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(config),
|
||||
}
|
||||
);
|
||||
|
||||
const json: GeneratePhotoAvatarResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(`Photo avatar generation failed: ${json.error}`);
|
||||
}
|
||||
|
||||
return json.data.generation_id;
|
||||
}
|
||||
|
||||
async function waitForPhotoGeneration(
|
||||
generationId: string
|
||||
): Promise<string[]> {
|
||||
for (let i = 0; i < 60; i++) {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/photo_avatar/generation/${generationId}`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const json: PhotoGenerationStatus = await response.json();
|
||||
|
||||
if (json.error) throw new Error(json.error);
|
||||
|
||||
if (json.data.status === "success") {
|
||||
return json.data.image_key_list!;
|
||||
}
|
||||
|
||||
if (json.data.status === "failed") {
|
||||
throw new Error(json.data.msg ?? "Photo generation failed");
|
||||
}
|
||||
|
||||
await new Promise((r) => setTimeout(r, 5000));
|
||||
}
|
||||
|
||||
throw new Error("Photo generation timed out");
|
||||
}
|
||||
```
|
||||
|
||||
### AI Photo → Avatar Group → Video
|
||||
|
||||
Use a generated AI photo to create an avatar group, then generate a video:
|
||||
|
||||
```typescript
|
||||
// 1. Generate AI photo
|
||||
const generationId = await generatePhotoAvatar({
|
||||
name: "Product Demo Host",
|
||||
age: "Young Adult",
|
||||
gender: "Woman",
|
||||
ethnicity: "Unspecified",
|
||||
orientation: "horizontal",
|
||||
pose: "half_body",
|
||||
style: "Realistic",
|
||||
appearance: "Professional woman, navy blazer, friendly smile, soft lighting",
|
||||
});
|
||||
|
||||
// 2. Wait for generation and pick first result
|
||||
const imageKeys = await waitForPhotoGeneration(generationId);
|
||||
const selectedImageKey = imageKeys[0];
|
||||
|
||||
// 3. Create avatar group from the AI photo
|
||||
const createResponse = await fetch(
|
||||
"https://api.heygen.com/v2/photo_avatar/avatar_group/create",
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
image_key: selectedImageKey,
|
||||
name: "Product Demo Host",
|
||||
generation_id: generationId,
|
||||
}),
|
||||
}
|
||||
);
|
||||
|
||||
const { data } = await createResponse.json();
|
||||
const talkingPhotoId = data.id;
|
||||
|
||||
// 4. Generate video (after status is "completed")
|
||||
const videoId = await generateVideo({
|
||||
video_inputs: [{
|
||||
character: {
|
||||
type: "talking_photo",
|
||||
talking_photo_id: talkingPhotoId,
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Welcome to our product demo!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
}],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
});
|
||||
```
|
||||
|
||||
### Pre-Generation Checklist
|
||||
|
||||
Before calling the AI generation API, ensure you have values for ALL fields:
|
||||
|
||||
| # | Field | Question to Ask / Default |
|
||||
|---|-------|---------------------------|
|
||||
| 1 | `name` | What should we call this avatar? |
|
||||
| 2 | `age` | Young Adult / Early Middle Age / Late Middle Age / Senior? |
|
||||
| 3 | `gender` | Woman / Man? |
|
||||
| 4 | `ethnicity` | Which ethnicity? (see enum values above) |
|
||||
| 5 | `orientation` | horizontal (landscape) / vertical (portrait) / square? |
|
||||
| 6 | `pose` | half_body (recommended) / close_up / full_body? |
|
||||
| 7 | `style` | Realistic (recommended) / Cinematic / other? |
|
||||
| 8 | `appearance` | Describe clothing, expression, lighting, background |
|
||||
|
||||
**If the user only provides a vague request** like "create a professional looking man", ask them to specify the missing fields OR make reasonable defaults (e.g., "Early Middle Age", "Realistic" style, "half_body" pose, "horizontal" orientation).
|
||||
|
||||
### Appearance Prompt Tips
|
||||
|
||||
The `appearance` field is a text prompt - be descriptive:
|
||||
|
||||
**Good prompts:**
|
||||
- "Professional woman with shoulder-length brown hair, wearing a light blue button-down shirt, warm friendly smile, soft studio lighting, clean white background"
|
||||
- "Young man with short black hair, casual tech startup style, wearing a dark hoodie, confident expression, modern office background with plants"
|
||||
|
||||
**Avoid:**
|
||||
- Vague descriptions: "a nice person"
|
||||
- Conflicting attributes
|
||||
- Requesting specific real people
|
||||
|
||||
## Managing Photo Avatars
|
||||
|
||||
### Get Photo Avatar Details
|
||||
|
||||
**Endpoint:** `GET https://api.heygen.com/v2/photo_avatar/{id}`
|
||||
|
||||
```typescript
|
||||
async function getPhotoAvatar(id: string): Promise<PhotoAvatarResponse> {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/photo_avatar/${id}`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
return response.json();
|
||||
}
|
||||
```
|
||||
|
||||
### Delete Photo Avatar
|
||||
|
||||
**Endpoint:** `DELETE https://api.heygen.com/v2/photo_avatar/{id}`
|
||||
|
||||
```typescript
|
||||
async function deletePhotoAvatar(id: string): Promise<void> {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/photo_avatar/${id}`,
|
||||
{
|
||||
method: "DELETE",
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
}
|
||||
);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error("Failed to delete photo avatar");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Delete Photo Avatar Group
|
||||
|
||||
**Endpoint:** `DELETE https://api.heygen.com/v2/photo_avatar_group/{group_id}`
|
||||
|
||||
```typescript
|
||||
async function deletePhotoAvatarGroup(groupId: string): Promise<void> {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/photo_avatar_group/${groupId}`,
|
||||
{
|
||||
method: "DELETE",
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
}
|
||||
);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error("Failed to delete photo avatar group");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `upload.heygen.com/v1/asset` | POST | Upload image (returns `image_key`) |
|
||||
| `/v2/photo_avatar/avatar_group/create` | POST | Create photo avatar from `image_key` |
|
||||
| `/v2/photo_avatar/avatar_group/add` | POST | Add photos to existing group |
|
||||
| `/v2/photo_avatar/train` | POST | Train avatar group |
|
||||
| `/v2/photo_avatar/train/status/{group_id}` | GET | Check training status |
|
||||
| `/v2/photo_avatar/{id}` | GET | Get photo avatar details/status |
|
||||
| `/v2/photo_avatar/{id}` | DELETE | Delete photo avatar |
|
||||
| `/v2/photo_avatar_group/{id}` | DELETE | Delete avatar group |
|
||||
| `/v2/photo_avatar/photo/generate` | POST | Generate AI photo from text |
|
||||
| `/v2/photo_avatar/generation/{id}` | GET | Check AI generation status |
|
||||
| `/v2/video/av4/generate` | POST | Avatar IV video from `image_key` |
|
||||
| `/v1/talking_photo.list` | GET | List all existing talking photos |
|
||||
| `/v2/video/generate` | POST | Generate video with `talking_photo_id` |
|
||||
|
||||
## Photo Requirements
|
||||
|
||||
### Technical Requirements
|
||||
|
||||
| Aspect | Requirement |
|
||||
|--------|-------------|
|
||||
| Format | JPEG, PNG |
|
||||
| Resolution | Minimum 512x512px |
|
||||
| File size | Under 10MB |
|
||||
| Face visibility | Clear, front-facing |
|
||||
|
||||
### Quality Guidelines
|
||||
|
||||
1. **Lighting** - Even, natural lighting on face
|
||||
2. **Expression** - Neutral or slight smile
|
||||
3. **Background** - Simple, uncluttered
|
||||
4. **Face position** - Centered, not cut off
|
||||
5. **Clarity** - Sharp, in focus
|
||||
6. **Angle** - Straight-on or slight angle
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use high-quality photos** - Better input = better output
|
||||
2. **Front-facing portraits** - Work best for animation
|
||||
3. **Neutral expressions** - Allow for more natural animation
|
||||
4. **Use Avatar IV for best quality** - Latest generation technology
|
||||
5. **Train avatar groups** - Improves animation quality
|
||||
6. **Reuse photo avatar IDs** - Once created, use the same `talking_photo_id` across multiple videos
|
||||
|
||||
## Limitations
|
||||
|
||||
- Photo quality significantly affects output
|
||||
- Side-profile photos have limited support
|
||||
- Full-body photos may not animate properly
|
||||
- Some expressions may look unnatural
|
||||
- Processing time varies by complexity
|
||||
@@ -0,0 +1,206 @@
|
||||
---
|
||||
name: prompt-examples
|
||||
description: Full production prompt examples and ready-to-use templates for Video Agent
|
||||
---
|
||||
|
||||
# Video Agent Prompt Examples
|
||||
|
||||
## Full Example: Brief to Production Prompt
|
||||
|
||||
### Input Brief
|
||||
|
||||
```
|
||||
Topic: Monthly company report for a SaaS startup
|
||||
Key data: $141M ARR (up from $54M), 1.85M signups (+28%), 3M paid videos/month
|
||||
Customer story: Creator built AI character, 2.5M followers, 20 min/video
|
||||
Challenge: Organic traffic volatile, -16% last week
|
||||
Duration: ~90 seconds
|
||||
Tone: Confident CEO, data-backed
|
||||
```
|
||||
|
||||
### Output Prompt
|
||||
|
||||
```
|
||||
FORMAT: Bloomberg-style company report. 90 seconds. Fast-paced, data-dense.
|
||||
Record-breaking month. Proud but analytical.
|
||||
|
||||
TONE: Confident, direct, data-backed. Highlights hit hard with numbers.
|
||||
Customer stories are the emotional core. Challenges are honest — no spin.
|
||||
|
||||
AVATAR: Man in simple black crew-neck tee, standing in a modern glass-walled
|
||||
office at golden hour. Behind him, a wall-mounted display shows the company logo
|
||||
in soft blue glow. Monitor to his right shows a dashboard with upward-trending
|
||||
charts. Desk beside him: laptop, half-empty flat white, scattered sticky notes.
|
||||
Warm afternoon light through floor-to-ceiling windows, long shadows on polished
|
||||
concrete. Minimal, focused startup HQ.
|
||||
|
||||
STYLE — SWISS PULSE (Müller-Brockmann): Grid-locked compositions. Black (#1a1a1a),
|
||||
white, electric blue (#0066FF), warm amber (#FF9500) for records. Helvetica Bold
|
||||
headlines, Regular labels. Numbers LARGE. Animated counters count up from 0.
|
||||
Diagonal compositions on accent moments. Grid wipe transitions. No dissolves.
|
||||
|
||||
CRITICAL ON-SCREEN TEXT (display literally):
|
||||
- "1.85M SIGNUPS — +28% MoM"
|
||||
- "$2.12M NEW SUBSCRIPTION REVENUE"
|
||||
- "$54M → $141M ARR"
|
||||
- "2.5M FOLLOWERS" and "20 MIN / VIDEO"
|
||||
- Quote: "Use technology to serve the message, not distract from it."
|
||||
- "ORGANIC: 65% OF SUBS — VOLATILE"
|
||||
|
||||
MUSIC: Upbeat electronic with a driving beat. Tycho meets Bloomberg opening theme.
|
||||
Builds through highlights, warms for customer story, softens for challenges, peaks
|
||||
on close.
|
||||
|
||||
---
|
||||
|
||||
SCENE 1 — A-ROLL (8s)
|
||||
[Avatar center-frame, energetic, leaning slightly forward]
|
||||
VOICEOVER: "January was a record month. New highs across acquisition, revenue,
|
||||
and product velocity. Here's the full picture."
|
||||
Lower-third SLIDES in: "COMPANY NAME | JANUARY 2026" white on blue bar.
|
||||
Grid wipe.
|
||||
|
||||
SCENE 2 — FULL SCREEN B-ROLL (12s)
|
||||
[NO AVATAR — motion graphic only]
|
||||
VOICEOVER: "One-point-eight-five million signups — twenty-eight percent month
|
||||
over month. Two-point-one-two million in new subscription revenue. Both all-time
|
||||
highs."
|
||||
LAYER 1: Dark #1a1a1a background with thin grid lines pulsing at 8% opacity.
|
||||
LAYER 2: "1.85M" SLAMS in from left, white Bold 140pt. "SIGNUPS" types on
|
||||
in electric blue 32pt uppercase. "+28% MoM" appears in amber.
|
||||
LAYER 3: Three stat cards CASCADE from top-right, staggered 0.3s:
|
||||
"$2.12M New Revenue" — "$3.4M Business ARR" — "$3M Pro ARR."
|
||||
Each number COUNTS UP from 0.
|
||||
LAYER 4: Bottom ticker scrolls: "Non-brand search +36% • Brand impressions 9.2M
|
||||
• Weekly subs +20.5%"
|
||||
LAYER 5: Grid lines RIPPLE outward on "1.85M" slam. Diagonal amber bar behind
|
||||
stat cards.
|
||||
Hard cut.
|
||||
|
||||
SCENE 3 — FULL SCREEN B-ROLL (12s)
|
||||
[NO AVATAR — motion graphic only]
|
||||
VOICEOVER: "Zoom out. Twelve months ago — fifty-four million ARR. Today —
|
||||
one hundred forty-one million. Nearly three X in a single year."
|
||||
LAYER 1: Dark background, subtle grid scrolling upward.
|
||||
LAYER 2: Animated line chart DRAWS ITSELF left to right. Y-axis: $50M to $150M.
|
||||
Final point "$140.84M" glows amber and pulses.
|
||||
LAYER 3: Milestone annotations float in at key data points.
|
||||
LAYER 4: Second smaller chart below — "Paid Videos" 0.91M to 2.97M, same style.
|
||||
LAYER 5: Thin grid lines converge toward final data point. Scan line sweeps.
|
||||
Grid wipe.
|
||||
|
||||
SCENE 4 — A-ROLL (8s)
|
||||
[Avatar center-frame, warm tone, genuine smile]
|
||||
VOICEOVER: "But the numbers only tell half the story. The other half is the
|
||||
people building on the platform."
|
||||
Lower-third: "Customer Spotlight"
|
||||
|
||||
SCENE 5 — FULL SCREEN B-ROLL (12s)
|
||||
[NO AVATAR — warm palette]
|
||||
VOICEOVER: "An AI character built entirely on the platform. Twenty minutes
|
||||
per video. Two-point-five million Instagram followers. The creator's principle:
|
||||
use technology to serve the message, not distract from it."
|
||||
LAYER 1: Dark background with warm amber grid lines at low opacity.
|
||||
LAYER 2: "CHARACTER NAME" in large white, center-top, 80pt.
|
||||
LAYER 3: Stats cascade from right: "2.5M Followers" COUNTS UP in amber —
|
||||
"20 min/video" — "7x Faster." Each a glowing node.
|
||||
LAYER 4: Quote card SLIDES UP: "Use technology to serve the message, not
|
||||
distract from it." Types on word by word.
|
||||
LAYER 5: Warm light bloom. Grid lines soften into curved arcs.
|
||||
Grid wipe.
|
||||
|
||||
SCENE 6 — A-ROLL (10s)
|
||||
[Avatar center-frame, serious/candid]
|
||||
VOICEOVER: "Now the honest part. Organic drives sixty-five percent of
|
||||
subscriptions and it's volatile. Non-brand traffic dropped sixteen percent
|
||||
last week. We've rebuilt attribution and we're investing in SEO."
|
||||
Lower-third: "Challenges"
|
||||
|
||||
SCENE 7 — A-ROLL (7s)
|
||||
[Avatar center-frame, energy lifts, direct eye contact]
|
||||
VOICEOVER: "Fifty-four million to one-forty-one in twelve months. Three million
|
||||
paid videos a month. January set the bar — now we raise it."
|
||||
End card: Logo centered, blue glow fade-in. Grid lines converge. Music peaks.
|
||||
|
||||
---
|
||||
|
||||
NARRATION STYLE: CEO energy — conviction backed by data. Fast on highlights.
|
||||
Warm on customer stories. Candid on challenges. Close with forward momentum.
|
||||
```
|
||||
|
||||
## Ready-to-Use Templates
|
||||
|
||||
### Tech News Briefing
|
||||
```
|
||||
FORMAT: 75-second high-energy tech briefing. Think: Bloomberg meets Vice.
|
||||
|
||||
AVATAR: [Presenter in tech-casual at a multi-monitor station.
|
||||
Describe clothing, monitor content, desk items, lighting.]
|
||||
|
||||
STYLE — DECONSTRUCTED (Brody): Dark grey #1a1a1a, rust orange #D4501E.
|
||||
Type at angles, overlapping. Gritty textures. Smash cut transitions.
|
||||
|
||||
CRITICAL ON-SCREEN TEXT:
|
||||
- [List every stat, quote, handle that must appear]
|
||||
|
||||
SCENE 1 — A-ROLL (8s): Hook with energy. State what's happening.
|
||||
SCENE 2 — B-ROLL (12s): First story with layered visuals (L1-L5).
|
||||
SCENE 3 — A-ROLL + OVERLAY (10s): Second story, split frame.
|
||||
SCENE 4 — B-ROLL (10s): Third story or dramatic data point.
|
||||
SCENE 5 — A-ROLL (8s): Wrap-up and forward look.
|
||||
```
|
||||
|
||||
### Product Comparison
|
||||
```
|
||||
FORMAT: 60-second comparison. [Product A] vs [Product B]. Data-driven.
|
||||
|
||||
AVATAR: [Presenter in review studio. Desk with both products visible.]
|
||||
|
||||
STYLE — DIGITAL GRID (Crouwel): Dark #0a0a0a, cyan #00D4FF and amber #FFB800.
|
||||
Two-color coding: cyan = Product A, amber = Product B. Monospaced type.
|
||||
|
||||
CRITICAL ON-SCREEN TEXT:
|
||||
- [Key stats for each product]
|
||||
- [Pricing, features, differentiators]
|
||||
|
||||
Use SPLIT FRAME B-roll: Product A left, Product B right.
|
||||
```
|
||||
|
||||
### Strategy Presentation
|
||||
```
|
||||
FORMAT: 90-second strategy briefing. Bloomberg meets board meeting.
|
||||
|
||||
AVATAR: [Executive in blazer over tee. Conference room with whiteboard frameworks.]
|
||||
|
||||
STYLE — SWISS PULSE (Müller-Brockmann): Black/white + blue #0066FF.
|
||||
Grid-locked. Helvetica. Animated counters. Grid wipe transitions.
|
||||
|
||||
CRITICAL ON-SCREEN TEXT:
|
||||
- [Framework labels, quadrant labels, key quotes]
|
||||
|
||||
Build frameworks visually: draw axes, plot positions, animate labels.
|
||||
```
|
||||
|
||||
### Social Ad (30 seconds)
|
||||
```
|
||||
FORMAT: 30-second social ad. Maximum energy. Portrait 9:16.
|
||||
|
||||
AVATAR: [Creator-style presenter. Ring light, colorful background.]
|
||||
|
||||
STYLE — CARNIVAL SURGE (Lins): Hot pink, yellow, teal. Collage layering.
|
||||
Text MASSIVE at angles. Confetti. Smash cuts.
|
||||
|
||||
Three scenes: Hook (8s) → Value prop (12s) → CTA (10s).
|
||||
Text fills 50-80% of every frame. Numbers SLAM.
|
||||
```
|
||||
|
||||
### Premium Report
|
||||
```
|
||||
FORMAT: 120-second investor-grade report. Understated authority.
|
||||
|
||||
AVATAR: [Tailored merino sweater. Architectural room, diffused natural light.]
|
||||
|
||||
STYLE — VELVET STANDARD (Vignelli): Black, white, gold #c9a84c.
|
||||
Thin ALL CAPS, wide spacing. Generous negative space.
|
||||
Slow cross-dissolves. Numbers fade in with weight.
|
||||
```
|
||||
@@ -0,0 +1,289 @@
|
||||
---
|
||||
name: prompt-optimizer
|
||||
description: Write production-quality prompts for HeyGen Video Agent — from basic ideas to fully art-directed scene-by-scene scripts
|
||||
---
|
||||
|
||||
# Video Agent Prompt Optimizer
|
||||
|
||||
Write effective prompts for the HeyGen Video Agent API. Based on patterns from 40+ produced videos.
|
||||
|
||||
**The core insight: Video Agent is an HTML interpreter.** It renders layouts, typography, and structured content natively. Describe B-roll as layered text motion graphics with action verbs ("slams in," "types on," "counts up") — not layout specs ("upper-left, 48pt").
|
||||
|
||||
## Reference Files
|
||||
|
||||
| File | Load when... |
|
||||
|------|-------------|
|
||||
| [visual-styles.md](visual-styles.md) | Choosing a visual style (20 styles with full specs) |
|
||||
| [prompt-examples.md](prompt-examples.md) | Writing a prompt from scratch (full production example + templates) |
|
||||
|
||||
## Workflow: Brief to Prompt
|
||||
|
||||
1. **Pull data** — Research the topic: web search, APIs, internal docs. Gather real quotes, stats, handles
|
||||
2. **Synthesize a thesis** — Not a list. A story. *"X is happening because Y — here's the proof."* Group into 3-5 themes with a narrative arc
|
||||
3. **Choose a style** — Match mood first, content second. Ask: *"What should the viewer FEEL?"* See [visual-styles.md](visual-styles.md)
|
||||
4. **Write the avatar** — Thematic wardrobe matching content's emotional context. Brand logos and content-specific props in the set (see Avatar Guide below)
|
||||
5. **Extract critical text** — List every number, quote, handle, and label that must appear literally
|
||||
6. **Break into scenes** — One concept per scene. Rotate scene types. Never 3+ of same type in a row. At least 2 pure B-roll scenes
|
||||
7. **Write voiceover** — Spell out numbers in VO ("one-point-eight-five million"), use figures on screen ("1.85M"). Narration on EVERY scene including B-roll
|
||||
8. **Layer each B-roll scene** — L1 background, L2 hero, L3 supporting, L4 info bar, L5 effects. Every element must MOVE
|
||||
9. **Add music direction** — Reference artists, describe energy arc
|
||||
10. **Add narration style** — How to deliver: fast/slow, where to pause, emotional register per section
|
||||
|
||||
## Prompt Anatomy
|
||||
|
||||
Every production-quality prompt follows this structure:
|
||||
|
||||
```
|
||||
FORMAT: What kind of video, how long, what energy
|
||||
TONE: Emotional register, references
|
||||
AVATAR: Detailed physical + environment description (60-100 words)
|
||||
STYLE: Named aesthetic with colors, typography, motion rules, transitions
|
||||
CRITICAL ON-SCREEN TEXT: Exact strings that must appear
|
||||
SCENE-BY-SCENE: Individual scene breakdowns with VO and layered visuals
|
||||
MUSIC: Genre, reference artists, energy arc
|
||||
NARRATION STYLE: How to deliver the voiceover
|
||||
```
|
||||
|
||||
### FORMAT
|
||||
|
||||
```
|
||||
FORMAT: 75-second high-energy tech daily briefing. Think: a creator who just got amazing news.
|
||||
FORMAT: Bloomberg-style strategy briefing. 100-120 seconds. CEO-delivered.
|
||||
```
|
||||
|
||||
### TONE
|
||||
|
||||
```
|
||||
TONE: Confident, direct, data-backed. Highlights hit hard. Lowlights are honest — no spin.
|
||||
TONE: Edgy, punk tech commentary. Vice News meets The Face magazine — raw, confrontational.
|
||||
```
|
||||
|
||||
### CRITICAL ON-SCREEN TEXT
|
||||
|
||||
List every exact string that must appear on screen. Without this, the agent may summarize, round numbers, or rephrase quotes.
|
||||
|
||||
```
|
||||
CRITICAL ON-SCREEN TEXT (display literally):
|
||||
- "$141M ARR — All-Time High"
|
||||
- "1.85M Signups — +28% MoM"
|
||||
- Quote: "Use technology to serve the message, not distract from it." — Shalev Hani
|
||||
- "@username" — exact social handle
|
||||
```
|
||||
|
||||
### MUSIC & NARRATION
|
||||
|
||||
```
|
||||
MUSIC: Driving electronic, heavy bass drops on key numbers. Run the Jewels meets
|
||||
a tech keynote. Builds relentlessly, only softens for customer stories.
|
||||
|
||||
NARRATION STYLE: High energy throughout. Let numbers PUNCH — pause before big ones,
|
||||
then deliver hard. Customer stories get warmth. The close should feel like a mic drop.
|
||||
```
|
||||
|
||||
## Avatar Description Guide
|
||||
|
||||
**The avatar is NOT a fixed headshot** — design it for each video like a movie character. Think costume designer + set designer.
|
||||
|
||||
### Thematic Wardrobe Rule
|
||||
|
||||
The avatar's outfit and environment MUST match the content's emotional/cultural context:
|
||||
|
||||
| Content Type | Avatar Design | NOT This |
|
||||
|---|---|---|
|
||||
| Chinese New Year | Red qipao with gold embroidery, lantern-lit courtyard | "Reporter in a blazer" |
|
||||
| Breaking tech news | Field reporter, windswept hair, earpiece, city skyline | "Anchor at a desk" |
|
||||
| Sleep science | Oversized cream knit, cross-legged on bed, warm lamp | "Analyst in a lab" |
|
||||
| Reddit community | Messy desk, Reddit alien on monitors, upvote arrows on wall | "Researcher in a studio" |
|
||||
|
||||
### What to Specify
|
||||
|
||||
| Element | Weak | Strong |
|
||||
|---------|------|--------|
|
||||
| Clothing | "Business casual" | "Black ribbed merino turtleneck, high collar framing jaw" |
|
||||
| Environment | "An office" | "Glass-walled conference room. Whiteboard with hand-drawn tier pyramid" |
|
||||
| Monitor content | "Computer screens" | "Monitor shows scrolling green terminal text and red security alerts" |
|
||||
| Lighting | "Well lit" | "Cool blue monitor glow from left, warm amber desk lamp from right" |
|
||||
|
||||
### Template
|
||||
|
||||
```
|
||||
AVATAR: [Clothing — fabric, color, fit, accessories, posture].
|
||||
[Setting — specific props, brand logos, what's on the walls].
|
||||
[Monitors/desk — content visible on screens, items on desk].
|
||||
[Lighting — direction, color temperature]. [Mood of the space].
|
||||
60-100 words. 3+ content-specific props. Brand elements visible.
|
||||
```
|
||||
|
||||
## Scene Types
|
||||
|
||||
| Type | Format | When to Use |
|
||||
|------|--------|-------------|
|
||||
| **A-ROLL** | Avatar speaking to camera | Intros, key insights, CTAs, emotional beats |
|
||||
| **FULL SCREEN B-ROLL** | No avatar — motion graphics only | Data visualization, information-dense content |
|
||||
| **A-ROLL + OVERLAY** | Split frame: avatar + content | Presenting data while maintaining human connection |
|
||||
|
||||
**Rotation is mandatory.** Never 3+ of the same type in a row. Every prompt needs at least 2 pure B-roll scenes.
|
||||
|
||||
**Voiceover on EVERY scene.** Every B-roll scene MUST include a `VOICEOVER:` line. Silent B-roll = broken video.
|
||||
|
||||
### Scene Anatomy
|
||||
|
||||
**A-ROLL:**
|
||||
```
|
||||
SCENE 1 — A-ROLL (10s)
|
||||
[Avatar center-frame, excited, hands gesturing]
|
||||
VOICEOVER: "The exact script for this scene."
|
||||
Lower-third: "TITLE TEXT" white on blue bar.
|
||||
```
|
||||
|
||||
**B-ROLL with layers:**
|
||||
```
|
||||
SCENE 2 — FULL SCREEN B-ROLL (12s)
|
||||
[NO AVATAR — motion graphic only]
|
||||
VOICEOVER: "The exact script for this scene."
|
||||
LAYER 1: Dark #1a1a1a background with subtle grid lines pulsing.
|
||||
LAYER 2: "HEADLINE" SLAMS in from left in white Bold 100pt at -5 degrees.
|
||||
LAYER 3: Three data cards CASCADE from right, staggered 0.3s.
|
||||
LAYER 4: Bottom ticker SLIDES in: "supporting text scrolling continuously."
|
||||
LAYER 5: Grid lines RIPPLE outward from impact point.
|
||||
Hard cut.
|
||||
```
|
||||
|
||||
**A-ROLL + OVERLAY:**
|
||||
```
|
||||
SCENE 3 — A-ROLL + OVERLAY (10s)
|
||||
[SPLIT — Avatar LEFT 35%. Content RIGHT 65%. NO overlap.]
|
||||
Avatar gestures toward content side.
|
||||
VOICEOVER: "The exact script for this scene."
|
||||
RIGHT SIDE: "HEADLINE" in cyan 60pt. Three stats COUNT UP below.
|
||||
```
|
||||
|
||||
Alternate which side the avatar appears on between overlay scenes.
|
||||
|
||||
## The Visual Layer System
|
||||
|
||||
Break B-roll into 5 stacked layers. This is the most powerful technique for motion graphics scenes.
|
||||
|
||||
| Layer | Purpose | Examples |
|
||||
|-------|---------|---------|
|
||||
| **L1** | Background | Textured surface, grid, gradient, color field |
|
||||
| **L2** | Hero content | Main headline/number that dominates the frame |
|
||||
| **L3** | Supporting data | Cards, stats, bullet points, secondary information |
|
||||
| **L4** | Information bar | Tickers, labels, source attributions, quotes |
|
||||
| **L5** | Effects | Particles, glitches, grid animations, ambient motion |
|
||||
|
||||
Every B-roll: 4+ layers. Every overlay content side: 3+ layers. **Every element must MOVE.**
|
||||
|
||||
## Motion Vocabulary
|
||||
|
||||
### High Energy
|
||||
| Verb | Example |
|
||||
|------|---------|
|
||||
| **SLAMS** | `"$95M" SLAMS in from left at -5 degrees` |
|
||||
| **CRASHES** | `Title CRASHES in from right, screen-shake on impact` |
|
||||
| **PUNCHES** | `Quote card PUNCHES up from bottom` |
|
||||
| **STAMPS** | `Data blocks STAMP in staggered 0.4s` |
|
||||
| **SHATTERS** | `Text SHATTERS after 1.5s, revealing number underneath` |
|
||||
|
||||
### Medium Energy
|
||||
| Verb | Example |
|
||||
|------|---------|
|
||||
| **CASCADE** | `Three cards CASCADE from top, staggered 0.3s` |
|
||||
| **SLIDES** | `Ticker SLIDES in from right — continuous scroll` |
|
||||
| **DROPS** | `"TIER 1" DROPS in with white flash` |
|
||||
| **FILLS** | `Progress bar FILLS 0 to 90% in orange` |
|
||||
| **DRAWS** | `Chart line DRAWS itself left to right` |
|
||||
|
||||
### Low Energy
|
||||
| Verb | Example |
|
||||
|------|---------|
|
||||
| **types on** | `Quote types on word by word in italic white` |
|
||||
| **fades in** | `Logo fades in at center, held for 3 seconds` |
|
||||
| **FLOATS** | `Bokeh orbs FLOAT across frame at different speeds` |
|
||||
| **morphs** | `Number morphs from 17 to 18.9` |
|
||||
| **COUNTS UP** | `"1.85M" COUNTS UP from 0 in amber 96pt` |
|
||||
|
||||
## Transition Types
|
||||
|
||||
| Transition | Energy | Styles It Fits |
|
||||
|------------|--------|---------------|
|
||||
| Smash cut | Aggressive | Deconstructed, Maximalist, Carnival Surge |
|
||||
| White flash frame | Punchy | Deconstructed, Maximalist |
|
||||
| Grid wipe | Systematic | Swiss Pulse, Digital Grid |
|
||||
| Hard cut | Clean | Swiss Pulse, Shadow Cut |
|
||||
| Liquid dissolve | Elegant | Data Drift, Dream State |
|
||||
| Slow cross-dissolve | Refined | Velvet Standard |
|
||||
| Pop cut / bounce | Fun | Play Mode, Carnival Surge |
|
||||
| Snap cut | Urgent | Red Wire, Contact Sheet |
|
||||
| Soft dissolve | Warm | Soft Signal, Warm Grain, Quiet Drama |
|
||||
| Iris wipe | Nostalgic | Heritage Reel |
|
||||
|
||||
## Timing Guidelines
|
||||
|
||||
| Content Type | Duration |
|
||||
|--------------|----------|
|
||||
| Hook/Intro (A-roll) | 6-10 seconds |
|
||||
| Data-heavy B-roll | 10-15 seconds (NEVER ≤5s — causes black frames) |
|
||||
| A-roll + Overlay | 8-12 seconds |
|
||||
| CTA / Close (A-roll) | 6-8 seconds |
|
||||
|
||||
**Common video lengths:** Social clip: 30-45s (5-7 scenes) | Briefing: 60-75s (7-9 scenes) | Deep dive: 90-120s (10-13 scenes)
|
||||
|
||||
**Speaking pace:** ~150 words/minute. Calculate: `words / 150 * 60 = seconds`
|
||||
|
||||
## What Doesn't Work
|
||||
|
||||
Patterns that consistently produce poor results:
|
||||
|
||||
**Layout language** — Screen coordinates cause empty/black B-roll:
|
||||
```
|
||||
❌ "UPPER-LEFT: headline in 48pt Helvetica"
|
||||
❌ "CENTER-SCREEN: display at coordinates (400, 300)"
|
||||
✅ "135K" SLAMS in from left, white Impact 120pt, fills 40% of frame.
|
||||
```
|
||||
|
||||
**Named artists without specs** — "Ikko Tanaka style" means nothing to Video Agent. Translate to concrete rules:
|
||||
```
|
||||
❌ "Use an Ikko Tanaka style"
|
||||
✅ "Flat color blocks, maximum 3 colors per frame, 60% negative space, typography as primary element"
|
||||
```
|
||||
|
||||
**Style examples injected into prompts** — Full example scenes from a style library confuse the agent. Use the style's **rules**, not example scenes.
|
||||
|
||||
**Forced short B-roll (≤5 seconds)** — Too short for rendering. Every tested video with 5s B-roll had empty/black screens. Use 10-15s.
|
||||
|
||||
**Content as a list, not a story** — "Here are 5 tweets" produces flat videos. Always synthesize: *"X is happening because Y — here's the proof."*
|
||||
|
||||
## Production Insights
|
||||
|
||||
### Style Performance (from 40+ videos)
|
||||
|
||||
| Rank | Style | Strength |
|
||||
|------|-------|----------|
|
||||
| 1 | Deconstructed (Brody) | Most reliable across all topics |
|
||||
| 2 | Swiss Pulse (Müller-Brockmann) | Best for data-heavy content |
|
||||
| 3 | Digital Grid (Crouwel) | Strong for tech topics |
|
||||
| 4 | Geometric Bold (Tanaka) | Elegant and versatile |
|
||||
| 5 | Maximalist Type (Scher) | High energy, use sparingly |
|
||||
|
||||
### Duration by Approach
|
||||
|
||||
| Approach | Avg Duration | Quality |
|
||||
|----------|-------------|---------|
|
||||
| Natural storyboard + custom avatar | ~106s | Best |
|
||||
| Natural storyboard, no custom avatar | ~69s | Good |
|
||||
| Forced short scenes + custom avatar | ~71s | Mixed |
|
||||
| Layout language prompts | ~48s | Poor |
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
- [ ] Thesis-driven — story, not bullet points
|
||||
- [ ] Style named with colors, typography, motion, transitions (see [visual-styles.md](visual-styles.md))
|
||||
- [ ] Avatar has thematic wardrobe + branded environment (60-100 words)
|
||||
- [ ] Critical text listed — every stat, quote, label
|
||||
- [ ] Scenes rotate types — never 3+ same type. At least 2 B-roll scenes
|
||||
- [ ] Every scene has VOICEOVER — including B-roll
|
||||
- [ ] B-roll scenes have 4+ layers, every element has motion verbs
|
||||
- [ ] B-roll scenes are 10-15 seconds (never ≤5s)
|
||||
- [ ] Brand logos appear when discussing companies
|
||||
- [ ] Every element moves — no static frames
|
||||
@@ -0,0 +1,195 @@
|
||||
---
|
||||
name: quota
|
||||
description: Credit system, usage limits, and checking remaining quota for HeyGen
|
||||
---
|
||||
|
||||
# HeyGen Quota and Credits
|
||||
|
||||
HeyGen uses a credit-based system for video generation. Understanding quota management helps prevent failed video generation requests.
|
||||
|
||||
## Checking Remaining Quota
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/user/remaining_quota" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface QuotaResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
remaining_quota: number;
|
||||
used_quota: number;
|
||||
};
|
||||
}
|
||||
|
||||
const response = await fetch("https://api.heygen.com/v2/user/remaining_quota", {
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
});
|
||||
|
||||
const { data }: QuotaResponse = await response.json();
|
||||
console.log(`Remaining credits: ${data.remaining_quota}`);
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
|
||||
response = requests.get(
|
||||
"https://api.heygen.com/v2/user/remaining_quota",
|
||||
headers={"X-Api-Key": os.environ["HEYGEN_API_KEY"]}
|
||||
)
|
||||
|
||||
data = response.json()["data"]
|
||||
print(f"Remaining credits: {data['remaining_quota']}")
|
||||
```
|
||||
|
||||
## Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"remaining_quota": 450,
|
||||
"used_quota": 50
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Credit Consumption
|
||||
|
||||
Different operations consume different amounts of credits:
|
||||
|
||||
| Operation | Credit Cost | Notes |
|
||||
|-----------|-------------|-------|
|
||||
| Standard video (1 min) | ~1 credit per minute | Varies by resolution |
|
||||
| 720p video | Base rate | Standard quality |
|
||||
| 1080p video | ~1.5x base rate | Higher quality |
|
||||
| Video translation | Varies | Depends on video length |
|
||||
| Streaming avatar | Per session | Real-time usage |
|
||||
|
||||
## Pre-Generation Quota Check
|
||||
|
||||
Always verify sufficient quota before generating videos:
|
||||
|
||||
```typescript
|
||||
async function generateVideoWithQuotaCheck(videoConfig: VideoConfig) {
|
||||
// Check quota first
|
||||
const quotaResponse = await fetch(
|
||||
"https://api.heygen.com/v2/user/remaining_quota",
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const { data: quota } = await quotaResponse.json();
|
||||
|
||||
// Estimate required credits (rough estimate: 1 credit per minute)
|
||||
const estimatedMinutes = videoConfig.estimatedDuration / 60;
|
||||
const requiredCredits = Math.ceil(estimatedMinutes);
|
||||
|
||||
if (quota.remaining_quota < requiredCredits) {
|
||||
throw new Error(
|
||||
`Insufficient credits. Need ${requiredCredits}, have ${quota.remaining_quota}`
|
||||
);
|
||||
}
|
||||
|
||||
// Proceed with video generation
|
||||
return generateVideo(videoConfig);
|
||||
}
|
||||
```
|
||||
|
||||
## Quota Management Best Practices
|
||||
|
||||
### 1. Monitor Usage Regularly
|
||||
|
||||
```typescript
|
||||
async function logQuotaUsage() {
|
||||
const response = await fetch(
|
||||
"https://api.heygen.com/v2/user/remaining_quota",
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const { data } = await response.json();
|
||||
|
||||
console.log({
|
||||
remaining: data.remaining_quota,
|
||||
used: data.used_quota,
|
||||
percentUsed: (
|
||||
(data.used_quota / (data.remaining_quota + data.used_quota)) *
|
||||
100
|
||||
).toFixed(1),
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Set Up Alerts
|
||||
|
||||
```typescript
|
||||
const QUOTA_WARNING_THRESHOLD = 50;
|
||||
|
||||
async function checkQuotaWithAlert() {
|
||||
const response = await fetch(
|
||||
"https://api.heygen.com/v2/user/remaining_quota",
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const { data } = await response.json();
|
||||
|
||||
if (data.remaining_quota < QUOTA_WARNING_THRESHOLD) {
|
||||
// Send alert (email, Slack, etc.)
|
||||
await sendAlert(`Low HeyGen quota: ${data.remaining_quota} credits remaining`);
|
||||
}
|
||||
|
||||
return data;
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Use Test Mode for Development
|
||||
|
||||
When available, use test mode to avoid consuming credits during development:
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
test: true, // Use test mode during development
|
||||
video_inputs: [...],
|
||||
};
|
||||
|
||||
// Test videos may have watermarks but don't consume credits
|
||||
```
|
||||
|
||||
## Subscription Tiers
|
||||
|
||||
Different subscription tiers have different quota allocations and features:
|
||||
|
||||
| Tier | Features |
|
||||
|------|----------|
|
||||
| Free | Limited credits, basic features |
|
||||
| Creator | More credits, standard avatars |
|
||||
| Team | Higher limits, team collaboration |
|
||||
| Enterprise | Custom limits, API access, priority support |
|
||||
|
||||
API access typically requires Enterprise tier or higher.
|
||||
|
||||
## Error Handling for Quota Issues
|
||||
|
||||
```typescript
|
||||
async function handleQuotaError(error: any) {
|
||||
if (error.message.includes("quota") || error.message.includes("credit")) {
|
||||
console.error("Quota exceeded. Consider:");
|
||||
console.error("1. Upgrading your subscription");
|
||||
console.error("2. Waiting for quota reset");
|
||||
console.error("3. Purchasing additional credits");
|
||||
|
||||
// Check current quota
|
||||
const quota = await getQuota();
|
||||
console.error(`Current remaining: ${quota.remaining_quota}`);
|
||||
}
|
||||
|
||||
throw error;
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,705 @@
|
||||
---
|
||||
name: remotion-integration
|
||||
description: Using HeyGen avatar videos in Remotion compositions
|
||||
---
|
||||
|
||||
# HeyGen + Remotion Integration
|
||||
|
||||
This guide covers workflows for generating HeyGen avatar videos and using them in Remotion compositions.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```typescript
|
||||
// 1. Get avatar with default voice
|
||||
const avatar = await getAvatarDetails(avatarId);
|
||||
|
||||
// 2. Generate video (MP4 with background - most common)
|
||||
const videoId = await generateVideo({
|
||||
video_inputs: [{
|
||||
character: { type: "avatar", avatar_id: avatar.id, avatar_style: "normal" },
|
||||
voice: { type: "text", input_text: script, voice_id: avatar.default_voice_id },
|
||||
background: { type: "color", value: "#1a1a2e" },
|
||||
}],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
});
|
||||
|
||||
// 3. Poll for completion (10-15+ min)
|
||||
// 4. Use in Remotion with motion graphics overlaid on top
|
||||
```
|
||||
|
||||
## Overview
|
||||
|
||||
A typical workflow:
|
||||
1. Generate avatar video with HeyGen
|
||||
2. Wait for completion and get video URL
|
||||
3. Download or use URL directly in Remotion
|
||||
4. Compose with other elements (backgrounds, overlays, animations)
|
||||
|
||||
## Choosing the Right Output Format
|
||||
|
||||
| Your Composition | Recommended | Why |
|
||||
|------------------|-------------|-----|
|
||||
| Avatar as presenter with overlays | MP4 + background | Simpler, overlays go on top |
|
||||
| Loom-style (avatar over screen recording) | WebM + `closeUp`, mask in Remotion | Need transparency, apply circle mask in CSS |
|
||||
| Avatar overlaid ON other video/content | WebM (transparent) | Need to see through to content behind |
|
||||
| Full-screen avatar | MP4 + background | Standard approach |
|
||||
|
||||
**Use MP4 with background for most cases.** Use WebM when you need to see content *behind* the avatar.
|
||||
|
||||
**Note:** WebM only supports `normal` and `closeUp` styles. For circular framing, use CSS `border-radius: 50%` in Remotion.
|
||||
|
||||
## Recommended: Parallel Development Workflow
|
||||
|
||||
HeyGen video generation takes **10-15+ minutes**. Don't wait - work in parallel:
|
||||
|
||||
1. **Start HeyGen generation** - save `video_id` to a file, exit immediately
|
||||
2. **Build Remotion composition** - use a placeholder or the avatar's `preview_video_url` (a short loop)
|
||||
3. **Check HeyGen status** periodically or when done building
|
||||
4. **Swap placeholder** for real video URL once ready
|
||||
|
||||
**Estimate duration from script**: ~150 words/minute speech rate, so `wordCount / 150 * 60 * fps` gives approximate frames.
|
||||
|
||||
**Composition tip**: Design components to work with or without the avatar video, so motion graphics can be tested independently.
|
||||
|
||||
## Dimension Alignment
|
||||
|
||||
**Critical**: Match HeyGen output dimensions to your Remotion composition.
|
||||
|
||||
### Common Dimension Presets
|
||||
|
||||
```typescript
|
||||
// Shared dimension constants for both HeyGen and Remotion
|
||||
const DIMENSIONS = {
|
||||
landscape_1080p: { width: 1920, height: 1080 },
|
||||
landscape_720p: { width: 1280, height: 720 },
|
||||
portrait_1080p: { width: 1080, height: 1920 },
|
||||
portrait_720p: { width: 720, height: 1280 },
|
||||
square_1080p: { width: 1080, height: 1080 },
|
||||
square_720p: { width: 720, height: 720 },
|
||||
} as const;
|
||||
|
||||
type DimensionPreset = keyof typeof DIMENSIONS;
|
||||
```
|
||||
|
||||
### HeyGen Video Generation
|
||||
|
||||
```typescript
|
||||
// Generate HeyGen video with specific dimensions
|
||||
async function generateHeyGenVideo(
|
||||
script: string,
|
||||
avatarId: string,
|
||||
voiceId: string,
|
||||
preset: DimensionPreset
|
||||
): Promise<string> {
|
||||
const dimension = DIMENSIONS[preset];
|
||||
|
||||
const response = await fetch("https://api.heygen.com/v2/video/generate", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: avatarId,
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: voiceId,
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#00FF00", // Green screen for compositing
|
||||
},
|
||||
},
|
||||
],
|
||||
dimension,
|
||||
}),
|
||||
});
|
||||
|
||||
const { data } = await response.json();
|
||||
return data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
### Remotion Composition Setup
|
||||
|
||||
```tsx
|
||||
// remotion/src/Root.tsx
|
||||
import { Composition } from "remotion";
|
||||
import { AvatarComposition } from "./AvatarComposition";
|
||||
|
||||
const DIMENSIONS = {
|
||||
landscape_1080p: { width: 1920, height: 1080 },
|
||||
// ... same as above
|
||||
};
|
||||
|
||||
export const RemotionRoot: React.FC = () => {
|
||||
return (
|
||||
<>
|
||||
<Composition
|
||||
id="AvatarVideo"
|
||||
component={AvatarComposition}
|
||||
durationInFrames={300} // Will be set dynamically
|
||||
fps={30}
|
||||
width={DIMENSIONS.landscape_1080p.width}
|
||||
height={DIMENSIONS.landscape_1080p.height}
|
||||
defaultProps={{
|
||||
avatarVideoUrl: "",
|
||||
}}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
## Generating Avatar Video for Remotion
|
||||
|
||||
### Standard: MP4 with Background
|
||||
|
||||
Most Remotion compositions work best with MP4 + background. Overlays and motion graphics go on top:
|
||||
|
||||
```typescript
|
||||
async function generateAvatarForRemotion(
|
||||
script: string,
|
||||
avatarId: string,
|
||||
voiceId: string,
|
||||
options: {
|
||||
style?: "normal" | "closeUp" | "circle";
|
||||
backgroundColor?: string;
|
||||
} = {}
|
||||
): Promise<string> {
|
||||
const { style = "normal", backgroundColor = "#1a1a2e" } = options;
|
||||
|
||||
const response = await fetch("https://api.heygen.com/v2/video/generate", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
video_inputs: [{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: avatarId,
|
||||
avatar_style: style,
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: voiceId,
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: backgroundColor,
|
||||
},
|
||||
}],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
}),
|
||||
});
|
||||
|
||||
const { data } = await response.json();
|
||||
return data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
### Transparent Background (WebM)
|
||||
|
||||
Only use when you need to see content *behind* the avatar (e.g., avatar overlaid on screen recording):
|
||||
|
||||
```typescript
|
||||
// Use /v1/video.webm endpoint for transparent background
|
||||
// Note: Different structure than /v2/video/generate
|
||||
const response = await fetch("https://api.heygen.com/v1/video.webm", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
avatar_pose_id: avatarPoseId, // Required: avatar pose ID
|
||||
avatar_style: "normal", // Required: "normal" or "closeUp" only
|
||||
input_text: script, // Required (with voice_id)
|
||||
voice_id: voiceId, // Required (with input_text)
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
}),
|
||||
});
|
||||
```
|
||||
|
||||
## Using HeyGen Video in Remotion
|
||||
|
||||
### Important: Use OffthreadVideo for Frame-Accurate Rendering
|
||||
|
||||
**Always use `OffthreadVideo` instead of `Video`** for HeyGen avatar videos. The basic `Video` component uses the browser's video decoder which isn't frame-accurate, causing jitter during rendering. `OffthreadVideo` extracts frames via FFmpeg for smooth, accurate playback.
|
||||
|
||||
`OffthreadVideo` is included in the core `remotion` package - no additional install needed.
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```tsx
|
||||
// remotion/src/AvatarComposition.tsx
|
||||
import { OffthreadVideo, useVideoConfig } from "remotion";
|
||||
|
||||
interface AvatarCompositionProps {
|
||||
avatarVideoUrl: string;
|
||||
}
|
||||
|
||||
export const AvatarComposition: React.FC<AvatarCompositionProps> = ({
|
||||
avatarVideoUrl,
|
||||
}) => {
|
||||
return (
|
||||
<div style={{ flex: 1, backgroundColor: "#1a1a2e" }}>
|
||||
<OffthreadVideo
|
||||
src={avatarVideoUrl}
|
||||
style={{
|
||||
width: "100%",
|
||||
height: "100%",
|
||||
objectFit: "contain",
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### WebM with Transparent Background (Recommended)
|
||||
|
||||
Using WebM from `/v1/video.webm` - no chroma keying needed:
|
||||
|
||||
```tsx
|
||||
import { OffthreadVideo, AbsoluteFill, Sequence } from "remotion";
|
||||
|
||||
export const AvatarWithMotionGraphics: React.FC<{
|
||||
avatarWebmUrl: string
|
||||
}> = ({ avatarWebmUrl }) => {
|
||||
return (
|
||||
<AbsoluteFill>
|
||||
{/* Layer 1: Your background/content */}
|
||||
<AbsoluteFill style={{ backgroundColor: "#1a1a2e" }}>
|
||||
<YourMotionGraphics />
|
||||
</AbsoluteFill>
|
||||
|
||||
{/* Layer 2: Avatar with transparent background - use OffthreadVideo for frame-accurate rendering */}
|
||||
<OffthreadVideo
|
||||
src={avatarWebmUrl}
|
||||
transparent
|
||||
style={{
|
||||
position: "absolute",
|
||||
bottom: 0,
|
||||
right: 0,
|
||||
width: "50%",
|
||||
height: "auto",
|
||||
}}
|
||||
/>
|
||||
|
||||
{/* Layer 3: Overlays on top of avatar */}
|
||||
<Sequence from={30}>
|
||||
<AnimatedTitle text="Welcome!" />
|
||||
</Sequence>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### Loom-Style: Circle Avatar Over Screen Recording
|
||||
|
||||
Use `closeUp` style + WebM, then apply circular mask in Remotion:
|
||||
|
||||
```tsx
|
||||
import { OffthreadVideo, AbsoluteFill } from "remotion";
|
||||
|
||||
export const LoomStyleComposition: React.FC<{
|
||||
screenRecordingUrl: string;
|
||||
avatarWebmUrl: string; // Generated with avatar_style: "closeUp" via /v1/video.webm
|
||||
}> = ({ screenRecordingUrl, avatarWebmUrl }) => {
|
||||
return (
|
||||
<AbsoluteFill>
|
||||
{/* Screen recording fills the frame */}
|
||||
<OffthreadVideo src={screenRecordingUrl} style={{ width: "100%", height: "100%" }} />
|
||||
|
||||
{/* Avatar with circular mask - transparent bg shows screen behind */}
|
||||
<OffthreadVideo
|
||||
src={avatarWebmUrl}
|
||||
transparent
|
||||
style={{
|
||||
position: "absolute",
|
||||
bottom: 40,
|
||||
left: 40,
|
||||
width: 180,
|
||||
height: 180,
|
||||
borderRadius: "50%", // Circular mask applied in CSS
|
||||
overflow: "hidden",
|
||||
objectFit: "cover",
|
||||
}}
|
||||
/>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
**Note:** WebM doesn't support `circle` style - use `normal` or `closeUp` and apply circular masking via CSS.
|
||||
|
||||
### Legacy: Green Screen with Chroma Key
|
||||
|
||||
If using MP4 with green background (not recommended - use WebM instead):
|
||||
|
||||
```tsx
|
||||
// Note: True chroma key requires WebGL or post-processing
|
||||
// WebM transparent background is much simpler
|
||||
<OffthreadVideo
|
||||
src={avatarVideoUrl}
|
||||
style={{
|
||||
mixBlendMode: "multiply", // Basic compositing only
|
||||
}}
|
||||
/>
|
||||
```
|
||||
|
||||
### Layered Composition
|
||||
|
||||
```tsx
|
||||
import { OffthreadVideo, Sequence, useVideoConfig, Img } from "remotion";
|
||||
|
||||
interface LayeredAvatarProps {
|
||||
avatarVideoUrl: string;
|
||||
backgroundUrl: string;
|
||||
logoUrl: string;
|
||||
title: string;
|
||||
}
|
||||
|
||||
export const LayeredAvatarComposition: React.FC<LayeredAvatarProps> = ({
|
||||
avatarVideoUrl,
|
||||
backgroundUrl,
|
||||
logoUrl,
|
||||
title,
|
||||
}) => {
|
||||
const { fps } = useVideoConfig();
|
||||
|
||||
return (
|
||||
<div style={{ position: "relative", width: "100%", height: "100%" }}>
|
||||
{/* Layer 1: Background */}
|
||||
<Img
|
||||
src={backgroundUrl}
|
||||
style={{
|
||||
position: "absolute",
|
||||
width: "100%",
|
||||
height: "100%",
|
||||
objectFit: "cover",
|
||||
}}
|
||||
/>
|
||||
|
||||
{/* Layer 2: Avatar video - use OffthreadVideo to prevent jitter */}
|
||||
<OffthreadVideo
|
||||
src={avatarVideoUrl}
|
||||
style={{
|
||||
position: "absolute",
|
||||
bottom: 0,
|
||||
right: 0,
|
||||
width: "40%",
|
||||
height: "auto",
|
||||
}}
|
||||
/>
|
||||
|
||||
{/* Layer 3: Title (appears after 1 second) */}
|
||||
<Sequence from={fps}>
|
||||
<div
|
||||
style={{
|
||||
position: "absolute",
|
||||
top: 50,
|
||||
left: 50,
|
||||
color: "white",
|
||||
fontSize: 48,
|
||||
fontWeight: "bold",
|
||||
}}
|
||||
>
|
||||
{title}
|
||||
</div>
|
||||
</Sequence>
|
||||
|
||||
{/* Layer 4: Logo */}
|
||||
<Img
|
||||
src={logoUrl}
|
||||
style={{
|
||||
position: "absolute",
|
||||
top: 20,
|
||||
right: 20,
|
||||
width: 100,
|
||||
height: "auto",
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
## Complete Workflow
|
||||
|
||||
### Generate and Compose
|
||||
|
||||
```typescript
|
||||
import { bundle } from "@remotion/bundler";
|
||||
import { renderMedia, selectComposition } from "@remotion/renderer";
|
||||
|
||||
async function generateAvatarVideoForRemotion(
|
||||
script: string,
|
||||
outputPath: string
|
||||
) {
|
||||
// 1. Generate HeyGen video
|
||||
console.log("Generating HeyGen avatar video...");
|
||||
const videoId = await generateHeyGenVideo(
|
||||
script,
|
||||
"josh_lite3_20230714",
|
||||
"1bd001e7e50f421d891986aad5158bc8",
|
||||
"landscape_1080p"
|
||||
);
|
||||
|
||||
// 2. Wait for completion
|
||||
console.log("Waiting for HeyGen video...");
|
||||
const avatarVideoUrl = await waitForVideo(videoId);
|
||||
console.log(`HeyGen video ready: ${avatarVideoUrl}`);
|
||||
|
||||
// 3. Get video duration for Remotion
|
||||
const avatarDuration = await getVideoDuration(avatarVideoUrl);
|
||||
const durationInFrames = Math.ceil(avatarDuration * 30); // 30 fps
|
||||
|
||||
// 4. Bundle Remotion project
|
||||
console.log("Bundling Remotion project...");
|
||||
const bundleLocation = await bundle({
|
||||
entryPoint: "./remotion/src/index.ts",
|
||||
});
|
||||
|
||||
// 5. Select composition
|
||||
const composition = await selectComposition({
|
||||
serveUrl: bundleLocation,
|
||||
id: "AvatarVideo",
|
||||
inputProps: {
|
||||
avatarVideoUrl,
|
||||
},
|
||||
});
|
||||
|
||||
// 6. Render final video
|
||||
console.log("Rendering final composition...");
|
||||
await renderMedia({
|
||||
composition: {
|
||||
...composition,
|
||||
durationInFrames,
|
||||
},
|
||||
serveUrl: bundleLocation,
|
||||
codec: "h264",
|
||||
outputLocation: outputPath,
|
||||
inputProps: {
|
||||
avatarVideoUrl,
|
||||
},
|
||||
});
|
||||
|
||||
console.log(`Final video rendered: ${outputPath}`);
|
||||
return outputPath;
|
||||
}
|
||||
```
|
||||
|
||||
### Dynamic Duration with calculateMetadata
|
||||
|
||||
```tsx
|
||||
// remotion/src/AvatarComposition.tsx
|
||||
import { CalculateMetadataFunction } from "remotion";
|
||||
|
||||
export const calculateAvatarMetadata: CalculateMetadataFunction<
|
||||
AvatarCompositionProps
|
||||
> = async ({ props }) => {
|
||||
// Fetch video duration from HeyGen video
|
||||
const duration = await getVideoDurationInSeconds(props.avatarVideoUrl);
|
||||
|
||||
return {
|
||||
durationInFrames: Math.ceil(duration * 30),
|
||||
fps: 30,
|
||||
width: 1920,
|
||||
height: 1080,
|
||||
};
|
||||
};
|
||||
|
||||
// In Root.tsx
|
||||
<Composition
|
||||
id="AvatarVideo"
|
||||
component={AvatarComposition}
|
||||
calculateMetadata={calculateAvatarMetadata}
|
||||
defaultProps={{
|
||||
avatarVideoUrl: "",
|
||||
}}
|
||||
/>
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Use Green Screen for Flexibility
|
||||
|
||||
Generate HeyGen videos with green screen background when you want to composite:
|
||||
|
||||
```typescript
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#00FF00", // Pure green for chroma key
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Match Frame Rates
|
||||
|
||||
HeyGen default is 25 fps. Consider this when setting Remotion fps:
|
||||
|
||||
```typescript
|
||||
// Option 1: Match HeyGen's 25 fps
|
||||
fps: 25
|
||||
|
||||
// Option 2: Use 30 fps with playback rate adjustment
|
||||
<OffthreadVideo
|
||||
src={avatarVideoUrl}
|
||||
playbackRate={25/30} // Slow down slightly to match
|
||||
/>
|
||||
```
|
||||
|
||||
### 3. URL vs Download: When to Use Each
|
||||
|
||||
**Use URL directly** when:
|
||||
- Previewing in Remotion Studio (`npm run dev`)
|
||||
- URL won't expire before render completes
|
||||
- You want faster iteration during development
|
||||
|
||||
```tsx
|
||||
// Direct URL usage - simpler, faster for dev
|
||||
<OffthreadVideo src={avatarVideoUrl} />
|
||||
```
|
||||
|
||||
**Download first** when:
|
||||
- URL has expiration (HeyGen URLs expire after ~24 hours)
|
||||
- Rendering will happen later or repeatedly
|
||||
- Network reliability is a concern
|
||||
- You need offline rendering
|
||||
|
||||
```typescript
|
||||
// Download with retry for reliability
|
||||
async function downloadVideoWithRetry(
|
||||
url: string,
|
||||
outputPath: string,
|
||||
maxRetries = 5
|
||||
): Promise<string> {
|
||||
for (let attempt = 0; attempt < maxRetries; attempt++) {
|
||||
try {
|
||||
const response = await fetch(url);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
|
||||
const buffer = await response.arrayBuffer();
|
||||
await fs.promises.writeFile(outputPath, Buffer.from(buffer));
|
||||
return outputPath;
|
||||
} catch (error) {
|
||||
const delay = 2000 * Math.pow(2, attempt);
|
||||
console.log(`Retry ${attempt + 1}/${maxRetries} in ${delay}ms...`);
|
||||
await new Promise((r) => setTimeout(r, delay));
|
||||
}
|
||||
}
|
||||
throw new Error("Download failed after retries");
|
||||
}
|
||||
|
||||
// Use local file in Remotion
|
||||
const localPath = await downloadVideoWithRetry(avatarVideoUrl, "./public/avatar.mp4");
|
||||
```
|
||||
|
||||
**Hybrid approach** (recommended for production):
|
||||
```typescript
|
||||
// Save both URL and local path in metadata
|
||||
const metadata = {
|
||||
videoUrl: result.video_url, // For quick preview
|
||||
localPath: "./public/avatar.mp4", // For reliable rendering
|
||||
expiresAt: Date.now() + 24 * 60 * 60 * 1000, // URL expiration
|
||||
};
|
||||
|
||||
// In Remotion component, prefer local if available
|
||||
const videoSrc = fs.existsSync(localPath) ? staticFile("avatar.mp4") : avatarVideoUrl;
|
||||
```
|
||||
|
||||
### 4. Handle Avatar Positioning
|
||||
|
||||
Common avatar positions in compositions:
|
||||
|
||||
```typescript
|
||||
const AVATAR_POSITIONS = {
|
||||
fullscreen: { width: "100%", height: "100%", position: "center" },
|
||||
bottomRight: { width: "40%", bottom: 0, right: 0 },
|
||||
bottomLeft: { width: "40%", bottom: 0, left: 0 },
|
||||
pictureInPicture: { width: "25%", bottom: 20, right: 20 },
|
||||
leftThird: { width: "33%", left: 0, height: "100%" },
|
||||
};
|
||||
```
|
||||
|
||||
## Output Formats
|
||||
|
||||
### HeyGen Output
|
||||
- Format: MP4 (H.264)
|
||||
- Audio: AAC
|
||||
- Resolution: As specified in request
|
||||
|
||||
### Remotion Output
|
||||
- Codec: H.264 (default), VP8, VP9, ProRes
|
||||
- Match or exceed HeyGen quality settings
|
||||
|
||||
```typescript
|
||||
await renderMedia({
|
||||
codec: "h264",
|
||||
crf: 18, // High quality
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Video Not Playing in Remotion
|
||||
|
||||
1. Check URL accessibility (CORS issues)
|
||||
2. Verify video format compatibility
|
||||
3. Try downloading locally first
|
||||
|
||||
### Dimension Mismatch
|
||||
|
||||
Ensure both HeyGen and Remotion use identical dimensions:
|
||||
|
||||
```typescript
|
||||
// Shared config
|
||||
const VIDEO_CONFIG = {
|
||||
width: 1920,
|
||||
height: 1080,
|
||||
fps: 30,
|
||||
};
|
||||
|
||||
// HeyGen
|
||||
dimension: { width: VIDEO_CONFIG.width, height: VIDEO_CONFIG.height }
|
||||
|
||||
// Remotion
|
||||
<Composition width={VIDEO_CONFIG.width} height={VIDEO_CONFIG.height} />
|
||||
```
|
||||
|
||||
### Video Jitter During Rendering
|
||||
|
||||
If avatar video appears jittery or stuttery in rendered output:
|
||||
|
||||
1. **Use `OffthreadVideo` instead of `Video`** - The basic `Video` component uses the browser's video decoder which isn't frame-accurate
|
||||
2. Update imports (no additional install needed - it's in core `remotion`):
|
||||
```tsx
|
||||
// Before (causes jitter)
|
||||
import { Video } from "remotion";
|
||||
|
||||
// After (frame-accurate)
|
||||
import { OffthreadVideo } from "remotion";
|
||||
```
|
||||
3. For WebM with transparency, add the `transparent` prop:
|
||||
```tsx
|
||||
<OffthreadVideo src={avatarWebmUrl} transparent />
|
||||
```
|
||||
|
||||
### Audio Sync Issues
|
||||
|
||||
If avatar audio drifts:
|
||||
- Verify source video frame rate
|
||||
- Check for encoding issues
|
||||
- Consider re-encoding with consistent settings
|
||||
@@ -0,0 +1,322 @@
|
||||
---
|
||||
name: scripts
|
||||
description: Writing effective scripts for HeyGen AI avatar videos
|
||||
---
|
||||
|
||||
# Writing Scripts for HeyGen Videos
|
||||
|
||||
Scripts for AI avatar videos have different requirements than scripts for human presenters. This guide covers best practices for writing scripts that sound natural and render well.
|
||||
|
||||
## Script Basics
|
||||
|
||||
### Speech Rate and Duration
|
||||
|
||||
Typical speech is approximately **150 words per minute** at normal speed (1.0x). Use this as a rough estimate for planning script length.
|
||||
|
||||
| Script Length | Approximate Duration |
|
||||
|---------------|---------------------|
|
||||
| 75 words | 30 seconds |
|
||||
| 150 words | 1 minute |
|
||||
| 300 words | 2 minutes |
|
||||
| 450 words | 3 minutes |
|
||||
| 750 words | 5 minutes |
|
||||
|
||||
```typescript
|
||||
// Estimate video duration from script
|
||||
function estimateDuration(script: string, speed: number = 1.0): number {
|
||||
const words = script.split(/\s+/).filter(w => w.length > 0).length;
|
||||
const wordsPerMinute = 150 * speed;
|
||||
return words / wordsPerMinute * 60; // seconds
|
||||
}
|
||||
|
||||
// Estimate frames for Remotion
|
||||
function estimateFrames(script: string, fps: number = 30, speed: number = 1.0): number {
|
||||
const durationSeconds = estimateDuration(script, speed);
|
||||
return Math.ceil(durationSeconds * fps);
|
||||
}
|
||||
```
|
||||
|
||||
### Sentence Structure
|
||||
|
||||
**Keep sentences short.** AI voices handle shorter sentences more naturally.
|
||||
|
||||
| Guideline | Example |
|
||||
|-----------|---------|
|
||||
| **Good**: 10-20 words per sentence | "Our platform helps teams collaborate. It syncs in real-time across all devices." |
|
||||
| **Avoid**: 30+ word run-on sentences | "Our platform helps teams collaborate more effectively by providing real-time synchronization across all devices while also offering offline support and automatic conflict resolution." |
|
||||
|
||||
### Punctuation Affects Delivery
|
||||
|
||||
| Punctuation | Effect |
|
||||
|-------------|--------|
|
||||
| Period `.` | Full stop, natural pause |
|
||||
| Comma `,` | Brief pause |
|
||||
| Question mark `?` | Rising intonation |
|
||||
| Exclamation `!` | Emphasis (use sparingly) |
|
||||
| Ellipsis `...` | Trailing off, slight pause |
|
||||
|
||||
## Adding Pauses with Break Tags
|
||||
|
||||
Use SSML-style `<break>` tags for precise pause control:
|
||||
|
||||
```
|
||||
<break time="Xs"/>
|
||||
```
|
||||
|
||||
Where `X` is seconds (e.g., `0.5s`, `1s`, `1.5s`, `2s`).
|
||||
|
||||
### Formatting Rules
|
||||
|
||||
| Rule | Correct | Incorrect |
|
||||
|------|---------|-----------|
|
||||
| Space before tag | `word <break time="1s"/>` | `word<break time="1s"/>` |
|
||||
| Space after tag | `<break time="1s"/> word` | `<break time="1s"/>word` |
|
||||
| Use seconds with "s" | `<break time="1.5s"/>` | `<break time="1500ms"/>` |
|
||||
| Self-closing tag | `<break time="1s"/>` | `<break time="1s"></break>` |
|
||||
|
||||
### When to Use Pauses
|
||||
|
||||
| Situation | Recommended Pause | Example |
|
||||
|-----------|-------------------|---------|
|
||||
| After greeting | 0.5-1s | `Hello! <break time="0.5s"/> Welcome to...` |
|
||||
| Between sections | 1-1.5s | `...that's feature one. <break time="1.5s"/> Now let's look at...` |
|
||||
| Before key point | 0.5s | `The most important thing is <break time="0.5s"/> consistency.` |
|
||||
| For dramatic effect | 1.5-2s | `And the winner is... <break time="2s"/> you!` |
|
||||
| After question | 1s | `Sound good? <break time="1s"/> Let's get started.` |
|
||||
| List items | 0.5s | `First, speed. <break time="0.5s"/> Second, reliability.` |
|
||||
|
||||
### Pause Duration Guide
|
||||
|
||||
| Duration | Feel | Use For |
|
||||
|----------|------|---------|
|
||||
| 0.3-0.5s | Brief breath | Between clauses, light emphasis |
|
||||
| 0.5-1s | Natural pause | Sentence breaks, transitions |
|
||||
| 1-1.5s | Deliberate pause | Section changes, setup for key points |
|
||||
| 1.5-2s | Dramatic | Reveals, important announcements |
|
||||
| 2s+ | Long pause | Use sparingly, can feel unnatural |
|
||||
|
||||
### Examples
|
||||
|
||||
```typescript
|
||||
// Section transitions
|
||||
const script = `
|
||||
Welcome to our product overview. <break time="1s"/>
|
||||
|
||||
Today I'll cover three key features. <break time="0.5s"/>
|
||||
First, let's look at the dashboard. <break time="1.5s"/>
|
||||
|
||||
As you can see, it's designed for simplicity. <break time="0.5s"/>
|
||||
Every action is just one click away.
|
||||
`;
|
||||
|
||||
// Building suspense
|
||||
const announcement = `
|
||||
We've been working on something special. <break time="1s"/>
|
||||
After months of development... <break time="1.5s"/>
|
||||
I'm excited to announce <break time="0.5s"/> our new AI assistant.
|
||||
`;
|
||||
|
||||
// List with rhythm
|
||||
const features = `
|
||||
Our platform offers three core benefits. <break time="0.5s"/>
|
||||
Speed. <break time="0.5s"/>
|
||||
Reliability. <break time="0.5s"/>
|
||||
And simplicity. <break time="1s"/>
|
||||
Let me show you each one.
|
||||
`;
|
||||
```
|
||||
|
||||
### Consecutive Breaks
|
||||
|
||||
Multiple consecutive breaks are combined:
|
||||
|
||||
```typescript
|
||||
// These two breaks:
|
||||
"Hello <break time=\"1s\"/> <break time=\"0.5s\"/> world"
|
||||
|
||||
// Are treated as a single 1.5s pause
|
||||
```
|
||||
|
||||
## Script Structure Templates
|
||||
|
||||
### Product Demo (60 seconds, ~150 words)
|
||||
|
||||
```typescript
|
||||
const productDemo = `
|
||||
Hi, I'm [Name], and I'm excited to show you [Product]. <break time="1s"/>
|
||||
|
||||
[Product] helps you [main benefit] in just [timeframe]. <break time="0.5s"/>
|
||||
|
||||
Here's how it works. <break time="1s"/>
|
||||
|
||||
First, [step 1]. <break time="0.5s"/>
|
||||
Then, [step 2]. <break time="0.5s"/>
|
||||
And finally, [step 3]. <break time="1s"/>
|
||||
|
||||
What used to take [old time] now takes [new time]. <break time="0.5s"/>
|
||||
|
||||
Ready to get started? <break time="0.5s"/>
|
||||
Visit [website] today.
|
||||
`;
|
||||
```
|
||||
|
||||
### Tutorial Introduction (90 seconds, ~225 words)
|
||||
|
||||
```typescript
|
||||
const tutorial = `
|
||||
Welcome to this tutorial on [topic]. <break time="0.5s"/>
|
||||
I'm [Name], and I'll guide you through everything you need to know. <break time="1s"/>
|
||||
|
||||
By the end of this video, you'll be able to [outcome 1], [outcome 2], and [outcome 3]. <break time="1s"/>
|
||||
|
||||
Let's start with the basics. <break time="1.5s"/>
|
||||
|
||||
[Section 1 content - 2-3 sentences] <break time="1s"/>
|
||||
|
||||
Now that you understand [concept], let's move on to [next topic]. <break time="1.5s"/>
|
||||
|
||||
[Section 2 content - 2-3 sentences] <break time="1s"/>
|
||||
|
||||
And finally, let's cover [last topic]. <break time="1.5s"/>
|
||||
|
||||
[Section 3 content - 2-3 sentences] <break time="1s"/>
|
||||
|
||||
That's everything you need to get started. <break time="0.5s"/>
|
||||
If you have questions, leave a comment below. <break time="0.5s"/>
|
||||
Thanks for watching!
|
||||
`;
|
||||
```
|
||||
|
||||
### Announcement (30 seconds, ~75 words)
|
||||
|
||||
```typescript
|
||||
const announcement = `
|
||||
Big news! <break time="0.5s"/>
|
||||
|
||||
We're thrilled to announce [announcement]. <break time="1s"/>
|
||||
|
||||
This means [benefit 1] and [benefit 2] for all our users. <break time="0.5s"/>
|
||||
|
||||
Starting [date], you'll be able to [new capability]. <break time="1s"/>
|
||||
|
||||
Head to [location] to learn more. <break time="0.5s"/>
|
||||
We can't wait to hear what you think!
|
||||
`;
|
||||
```
|
||||
|
||||
## Writing Tips for AI Voices
|
||||
|
||||
### Do
|
||||
|
||||
- **Write conversationally** - Read it aloud to check flow
|
||||
- **Use contractions** - "We're" not "We are", "It's" not "It is"
|
||||
- **Break up long sentences** - Split at natural pause points
|
||||
- **Spell out abbreviations** - "API" may sound like "a pee eye"
|
||||
- **Add pauses for emphasis** - Guide the listener's attention
|
||||
- **End sections clearly** - Don't trail off mid-thought
|
||||
|
||||
### Avoid
|
||||
|
||||
- **Jargon without context** - Explain technical terms
|
||||
- **Long parentheticals** - Move to separate sentences
|
||||
- **Ambiguous pronunciations** - "read" (present) vs "read" (past)
|
||||
- **Excessive exclamation marks** - One per script is usually enough
|
||||
- **Run-on sentences** - Break into digestible chunks
|
||||
- **Dense information** - Space out facts with pauses
|
||||
|
||||
### Pronunciation Hints
|
||||
|
||||
For words that might be mispronounced, spell phonetically or add hints:
|
||||
|
||||
```typescript
|
||||
// Technical terms
|
||||
const script1 = "Our API (A-P-I) handles authentication...";
|
||||
|
||||
// Ambiguous words
|
||||
const script2 = "I read (red) the documentation yesterday...";
|
||||
|
||||
// Brand names
|
||||
const script3 = "Welcome to HeyGen (hey-jen)...";
|
||||
```
|
||||
|
||||
## Multi-Scene Scripts
|
||||
|
||||
When splitting scripts across scenes (for different backgrounds or avatars):
|
||||
|
||||
```typescript
|
||||
const multiSceneVideo = {
|
||||
video_inputs: [
|
||||
{
|
||||
// Scene 1: Introduction
|
||||
character: { type: "avatar", avatar_id: "josh_lite3_20230714", avatar_style: "normal" },
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Welcome to our quarterly update. <break time=\"1s\"/> I'm Josh, and I'll walk you through the highlights.",
|
||||
voice_id: "voice_id_here",
|
||||
},
|
||||
background: { type: "color", value: "#1a1a2e" },
|
||||
},
|
||||
{
|
||||
// Scene 2: Main content (different background)
|
||||
character: { type: "avatar", avatar_id: "josh_lite3_20230714", avatar_style: "normal" },
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Let's start with revenue. <break time=\"0.5s\"/> We grew 25 percent quarter over quarter. <break time=\"1s\"/> Here's what drove that growth.",
|
||||
voice_id: "voice_id_here",
|
||||
},
|
||||
background: { type: "image", url: "https://..." },
|
||||
},
|
||||
// ... more scenes
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Scene Transition Tips
|
||||
|
||||
- End each scene with a complete thought
|
||||
- Start new scenes with brief context
|
||||
- Maintain consistent tone across scenes
|
||||
- Use pauses at scene starts to let visuals register
|
||||
|
||||
## Testing Your Script
|
||||
|
||||
Before generating the full video:
|
||||
|
||||
1. **Read aloud** - Time yourself, check for awkward phrasing
|
||||
2. **Count words** - Verify expected duration
|
||||
3. **Check break tags** - Ensure proper spacing and syntax
|
||||
4. **Preview with short clip** - Generate a 10-second test if unsure about pronunciation
|
||||
|
||||
```typescript
|
||||
// Test a small portion first
|
||||
const testScript = script.split('.').slice(0, 2).join('.') + '.';
|
||||
const testVideoId = await generateVideo({
|
||||
video_inputs: [{
|
||||
character: { type: "avatar", avatar_id: avatarId, avatar_style: "normal" },
|
||||
voice: { type: "text", input_text: testScript, voice_id: voiceId },
|
||||
}],
|
||||
dimension: { width: 1280, height: 720 }, // Lower res for test
|
||||
});
|
||||
```
|
||||
|
||||
## Voice Speed Adjustment
|
||||
|
||||
Adjust delivery speed in the voice configuration:
|
||||
|
||||
```typescript
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: "voice_id",
|
||||
speed: 1.1, // Slightly faster (range: 0.5 - 2.0)
|
||||
}
|
||||
```
|
||||
|
||||
| Speed | Effect | Use Case |
|
||||
|-------|--------|----------|
|
||||
| 0.8-0.9 | Slower, deliberate | Complex topics, older audiences |
|
||||
| 1.0 | Normal | General use |
|
||||
| 1.1-1.2 | Slightly faster | Energetic content, younger audiences |
|
||||
| 1.3+ | Fast | Use sparingly, may reduce clarity |
|
||||
|
||||
See [voices.md](voices.md) for full voice configuration options.
|
||||
@@ -0,0 +1,433 @@
|
||||
---
|
||||
name: templates
|
||||
description: Template listing and variable replacement for HeyGen videos
|
||||
---
|
||||
|
||||
# Video Templates
|
||||
|
||||
HeyGen templates allow you to create reusable video structures with variable placeholders, enabling personalized video generation at scale.
|
||||
|
||||
## Listing Templates
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/templates" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface Template {
|
||||
template_id: string;
|
||||
name: string;
|
||||
thumbnail_url: string;
|
||||
variables: TemplateVariable[];
|
||||
}
|
||||
|
||||
interface TemplateVariable {
|
||||
name: string;
|
||||
type: "text" | "image" | "audio";
|
||||
properties?: {
|
||||
max_length?: number;
|
||||
default_value?: string;
|
||||
};
|
||||
}
|
||||
|
||||
interface TemplatesResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
templates: Template[];
|
||||
};
|
||||
}
|
||||
|
||||
async function listTemplates(): Promise<Template[]> {
|
||||
const response = await fetch("https://api.heygen.com/v2/templates", {
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
});
|
||||
|
||||
const json: TemplatesResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data.templates;
|
||||
}
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
|
||||
def list_templates() -> list:
|
||||
response = requests.get(
|
||||
"https://api.heygen.com/v2/templates",
|
||||
headers={"X-Api-Key": os.environ["HEYGEN_API_KEY"]}
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
if data.get("error"):
|
||||
raise Exception(data["error"])
|
||||
|
||||
return data["data"]["templates"]
|
||||
```
|
||||
|
||||
## Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"templates": [
|
||||
{
|
||||
"template_id": "template_abc123",
|
||||
"name": "Product Announcement",
|
||||
"thumbnail_url": "https://files.heygen.ai/...",
|
||||
"variables": [
|
||||
{
|
||||
"name": "product_name",
|
||||
"type": "text",
|
||||
"properties": {
|
||||
"max_length": 50
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "presenter_script",
|
||||
"type": "text",
|
||||
"properties": {
|
||||
"max_length": 500
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "product_image",
|
||||
"type": "image"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Getting Template Details
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/template/{template_id}" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
async function getTemplate(templateId: string): Promise<Template> {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/template/${templateId}`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const json = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data;
|
||||
}
|
||||
```
|
||||
|
||||
## Generating Video from Template
|
||||
|
||||
### Request Fields
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `variables` | object | ✓ | Key-value pairs matching template variables |
|
||||
| `test` | boolean | | Test mode (watermarked, no credits) |
|
||||
| `title` | string | | Video name for organization |
|
||||
| `callback_id` | string | | Custom ID for webhook tracking |
|
||||
| `callback_url` | string | | URL for completion notification |
|
||||
|
||||
**Note:** The `variables` object keys must match the template's defined variable names. Check template details to see which variables are defined.
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v2/template/{template_id}/generate" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"test": false,
|
||||
"variables": {
|
||||
"product_name": "SuperWidget Pro",
|
||||
"presenter_script": "Introducing our latest innovation!",
|
||||
"product_image": "https://example.com/product.jpg"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface TemplateGenerateRequest {
|
||||
variables: Record<string, string>; // Required
|
||||
test?: boolean;
|
||||
title?: string;
|
||||
callback_id?: string;
|
||||
callback_url?: string;
|
||||
}
|
||||
|
||||
interface TemplateGenerateResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
video_id: string;
|
||||
};
|
||||
}
|
||||
|
||||
async function generateFromTemplate(
|
||||
templateId: string,
|
||||
variables: Record<string, string>,
|
||||
test: boolean = false
|
||||
): Promise<string> {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/template/${templateId}/generate`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({ test, variables }),
|
||||
}
|
||||
);
|
||||
|
||||
const json: TemplateGenerateResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
def generate_from_template(template_id: str, variables: dict, test: bool = False) -> str:
|
||||
response = requests.post(
|
||||
f"https://api.heygen.com/v2/template/{template_id}/generate",
|
||||
headers={
|
||||
"X-Api-Key": os.environ["HEYGEN_API_KEY"],
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
json={
|
||||
"test": test,
|
||||
"variables": variables
|
||||
}
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
if data.get("error"):
|
||||
raise Exception(data["error"])
|
||||
|
||||
return data["data"]["video_id"]
|
||||
```
|
||||
|
||||
## Variable Types
|
||||
|
||||
### Text Variables
|
||||
|
||||
For dynamic text content:
|
||||
|
||||
```typescript
|
||||
const variables = {
|
||||
customer_name: "John Smith",
|
||||
product_name: "SuperWidget Pro",
|
||||
price: "$99.99",
|
||||
cta_text: "Order Now!",
|
||||
};
|
||||
```
|
||||
|
||||
### Image Variables
|
||||
|
||||
For dynamic images (backgrounds, product shots):
|
||||
|
||||
```typescript
|
||||
const variables = {
|
||||
product_image: "https://example.com/product.jpg",
|
||||
logo: "https://example.com/logo.png",
|
||||
background: "https://example.com/bg.jpg",
|
||||
};
|
||||
```
|
||||
|
||||
### Audio Variables
|
||||
|
||||
For custom audio content:
|
||||
|
||||
```typescript
|
||||
const variables = {
|
||||
background_music: "https://example.com/music.mp3",
|
||||
custom_voiceover: "https://example.com/voiceover.mp3",
|
||||
};
|
||||
```
|
||||
|
||||
## Batch Video Generation
|
||||
|
||||
Generate multiple personalized videos from a template:
|
||||
|
||||
```typescript
|
||||
interface PersonalizationData {
|
||||
name: string;
|
||||
email: string;
|
||||
company: string;
|
||||
customMessage: string;
|
||||
}
|
||||
|
||||
async function batchGenerateVideos(
|
||||
templateId: string,
|
||||
recipients: PersonalizationData[]
|
||||
): Promise<string[]> {
|
||||
const videoIds: string[] = [];
|
||||
|
||||
for (const recipient of recipients) {
|
||||
const variables = {
|
||||
recipient_name: recipient.name,
|
||||
company_name: recipient.company,
|
||||
personalized_message: recipient.customMessage,
|
||||
};
|
||||
|
||||
const videoId = await generateFromTemplate(templateId, variables);
|
||||
videoIds.push(videoId);
|
||||
|
||||
// Rate limiting: add delay between requests
|
||||
await new Promise((r) => setTimeout(r, 1000));
|
||||
}
|
||||
|
||||
return videoIds;
|
||||
}
|
||||
|
||||
// Usage
|
||||
const recipients = [
|
||||
{
|
||||
name: "John Smith",
|
||||
email: "john@example.com",
|
||||
company: "Acme Inc",
|
||||
customMessage: "Thanks for your interest in our product!",
|
||||
},
|
||||
{
|
||||
name: "Jane Doe",
|
||||
email: "jane@example.com",
|
||||
company: "Tech Corp",
|
||||
customMessage: "We'd love to show you a demo!",
|
||||
},
|
||||
];
|
||||
|
||||
const videoIds = await batchGenerateVideos("template_abc123", recipients);
|
||||
```
|
||||
|
||||
## Template Validation
|
||||
|
||||
Validate variables before generating:
|
||||
|
||||
```typescript
|
||||
function validateTemplateVariables(
|
||||
template: Template,
|
||||
variables: Record<string, string>
|
||||
): { valid: boolean; errors: string[] } {
|
||||
const errors: string[] = [];
|
||||
|
||||
for (const templateVar of template.variables) {
|
||||
const value = variables[templateVar.name];
|
||||
|
||||
// Check if required variable is provided
|
||||
if (!value) {
|
||||
errors.push(`Missing required variable: ${templateVar.name}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Check text length limits
|
||||
if (templateVar.type === "text" && templateVar.properties?.max_length) {
|
||||
if (value.length > templateVar.properties.max_length) {
|
||||
errors.push(
|
||||
`Variable "${templateVar.name}" exceeds max length of ${templateVar.properties.max_length}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Validate image URLs
|
||||
if (templateVar.type === "image") {
|
||||
try {
|
||||
new URL(value);
|
||||
} catch {
|
||||
errors.push(`Variable "${templateVar.name}" is not a valid URL`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
valid: errors.length === 0,
|
||||
errors,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Complete Template Workflow
|
||||
|
||||
```typescript
|
||||
async function createPersonalizedVideo(
|
||||
templateId: string,
|
||||
personalization: Record<string, string>
|
||||
): Promise<string> {
|
||||
// 1. Get template details
|
||||
const template = await getTemplate(templateId);
|
||||
console.log(`Using template: ${template.name}`);
|
||||
|
||||
// 2. Validate variables
|
||||
const validation = validateTemplateVariables(template, personalization);
|
||||
if (!validation.valid) {
|
||||
throw new Error(`Validation errors: ${validation.errors.join(", ")}`);
|
||||
}
|
||||
|
||||
// 3. Generate video
|
||||
console.log("Generating video...");
|
||||
const videoId = await generateFromTemplate(templateId, personalization);
|
||||
console.log(`Video ID: ${videoId}`);
|
||||
|
||||
// 4. Wait for completion
|
||||
const videoUrl = await waitForVideo(videoId);
|
||||
console.log(`Video ready: ${videoUrl}`);
|
||||
|
||||
return videoUrl;
|
||||
}
|
||||
|
||||
// Usage
|
||||
const videoUrl = await createPersonalizedVideo("template_abc123", {
|
||||
customer_name: "John Smith",
|
||||
product_name: "SuperWidget Pro",
|
||||
offer_details: "Get 20% off your first order!",
|
||||
});
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Design for flexibility** - Create templates with generic placeholders
|
||||
2. **Set reasonable limits** - Define max lengths for text variables
|
||||
3. **Validate inputs** - Check variable values before generating
|
||||
4. **Use test mode** - Test with `test: true` to verify before production
|
||||
5. **Implement rate limiting** - Add delays for batch generation
|
||||
6. **Cache template data** - Reduce API calls by caching template details
|
||||
7. **Error handling** - Gracefully handle generation failures
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **Sales outreach** - Personalized prospect videos
|
||||
- **Customer onboarding** - Welcome videos with customer name
|
||||
- **Product updates** - Announcements with dynamic content
|
||||
- **Training** - Customized training modules
|
||||
- **Marketing campaigns** - Targeted promotional videos
|
||||
@@ -0,0 +1,316 @@
|
||||
---
|
||||
name: text-overlays
|
||||
description: Adding text overlays with fonts and positioning to HeyGen videos
|
||||
---
|
||||
|
||||
# Text Overlays
|
||||
|
||||
Add text overlays to your HeyGen videos for titles, captions, lower thirds, and other on-screen text elements.
|
||||
|
||||
## Basic Text Overlay
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Welcome to our presentation!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#1a1a2e",
|
||||
},
|
||||
},
|
||||
],
|
||||
// Text overlay configuration (if supported in your API tier)
|
||||
// Note: Availability varies by plan
|
||||
};
|
||||
```
|
||||
|
||||
## Text Overlay Configuration
|
||||
|
||||
Text overlays typically support these properties:
|
||||
|
||||
```typescript
|
||||
interface TextOverlay {
|
||||
text: string;
|
||||
x: number; // X position (pixels or percentage)
|
||||
y: number; // Y position (pixels or percentage)
|
||||
width?: number; // Text box width
|
||||
height?: number; // Text box height
|
||||
font_family?: string;
|
||||
font_size?: number;
|
||||
font_color?: string;
|
||||
background_color?: string;
|
||||
text_align?: "left" | "center" | "right";
|
||||
duration?: {
|
||||
start: number; // Start time in seconds
|
||||
end: number; // End time in seconds
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Positioning Text
|
||||
|
||||
### Coordinate System
|
||||
|
||||
- **Origin**: Top-left corner (0, 0)
|
||||
- **X-axis**: Increases to the right
|
||||
- **Y-axis**: Increases downward
|
||||
- **Units**: Typically pixels or percentage of video dimensions
|
||||
|
||||
### Common Positions
|
||||
|
||||
For a 1920x1080 video:
|
||||
|
||||
| Position | X | Y | Description |
|
||||
|----------|---|---|-------------|
|
||||
| Top-left | 50 | 50 | Upper left corner |
|
||||
| Top-center | 960 | 50 | Top center |
|
||||
| Top-right | 1870 | 50 | Upper right corner |
|
||||
| Center | 960 | 540 | Dead center |
|
||||
| Bottom-left | 50 | 1030 | Lower third left |
|
||||
| Bottom-center | 960 | 1030 | Lower third center |
|
||||
|
||||
### Position Helper Function
|
||||
|
||||
```typescript
|
||||
interface Position {
|
||||
x: number;
|
||||
y: number;
|
||||
}
|
||||
|
||||
function getTextPosition(
|
||||
location: "top-left" | "top-center" | "top-right" | "center" | "bottom-left" | "bottom-center" | "bottom-right",
|
||||
videoWidth: number,
|
||||
videoHeight: number,
|
||||
padding: number = 50
|
||||
): Position {
|
||||
const positions: Record<string, Position> = {
|
||||
"top-left": { x: padding, y: padding },
|
||||
"top-center": { x: videoWidth / 2, y: padding },
|
||||
"top-right": { x: videoWidth - padding, y: padding },
|
||||
"center": { x: videoWidth / 2, y: videoHeight / 2 },
|
||||
"bottom-left": { x: padding, y: videoHeight - padding },
|
||||
"bottom-center": { x: videoWidth / 2, y: videoHeight - padding },
|
||||
"bottom-right": { x: videoWidth - padding, y: videoHeight - padding },
|
||||
};
|
||||
|
||||
return positions[location];
|
||||
}
|
||||
```
|
||||
|
||||
## Font Styling
|
||||
|
||||
### Available Font Properties
|
||||
|
||||
```typescript
|
||||
const textStyle = {
|
||||
font_family: "Arial",
|
||||
font_size: 48,
|
||||
font_color: "#FFFFFF",
|
||||
font_weight: "bold",
|
||||
background_color: "rgba(0, 0, 0, 0.5)",
|
||||
text_align: "center",
|
||||
};
|
||||
```
|
||||
|
||||
### Common Font Families
|
||||
|
||||
| Font | Style | Use Case |
|
||||
|------|-------|----------|
|
||||
| Arial | Sans-serif | Clean, universal |
|
||||
| Helvetica | Sans-serif | Modern, professional |
|
||||
| Times New Roman | Serif | Traditional, formal |
|
||||
| Georgia | Serif | Elegant, readable |
|
||||
| Roboto | Sans-serif | Modern, digital |
|
||||
| Open Sans | Sans-serif | Friendly, accessible |
|
||||
|
||||
## Common Text Overlay Patterns
|
||||
|
||||
### Title Card
|
||||
|
||||
```typescript
|
||||
const titleOverlay = {
|
||||
text: "Product Demo",
|
||||
x: 960,
|
||||
y: 540,
|
||||
font_family: "Arial",
|
||||
font_size: 72,
|
||||
font_color: "#FFFFFF",
|
||||
text_align: "center",
|
||||
duration: {
|
||||
start: 0,
|
||||
end: 3,
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
### Lower Third (Name/Title)
|
||||
|
||||
```typescript
|
||||
const lowerThirdOverlay = {
|
||||
text: "John Smith\nCEO, Company Inc.",
|
||||
x: 100,
|
||||
y: 900,
|
||||
font_family: "Arial",
|
||||
font_size: 36,
|
||||
font_color: "#FFFFFF",
|
||||
background_color: "rgba(0, 102, 204, 0.9)",
|
||||
text_align: "left",
|
||||
duration: {
|
||||
start: 2,
|
||||
end: 8,
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
### Call to Action
|
||||
|
||||
```typescript
|
||||
const ctaOverlay = {
|
||||
text: "Visit example.com",
|
||||
x: 960,
|
||||
y: 1000,
|
||||
font_family: "Arial",
|
||||
font_size: 42,
|
||||
font_color: "#FFD700",
|
||||
text_align: "center",
|
||||
duration: {
|
||||
start: 25,
|
||||
end: 30,
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
## Creating Text Overlay Templates
|
||||
|
||||
```typescript
|
||||
interface TextOverlayTemplate {
|
||||
name: string;
|
||||
style: Partial<TextOverlay>;
|
||||
}
|
||||
|
||||
const templates: TextOverlayTemplate[] = [
|
||||
{
|
||||
name: "title",
|
||||
style: {
|
||||
font_family: "Arial",
|
||||
font_size: 72,
|
||||
font_color: "#FFFFFF",
|
||||
text_align: "center",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "subtitle",
|
||||
style: {
|
||||
font_family: "Arial",
|
||||
font_size: 42,
|
||||
font_color: "#CCCCCC",
|
||||
text_align: "center",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "lower-third",
|
||||
style: {
|
||||
font_family: "Arial",
|
||||
font_size: 36,
|
||||
font_color: "#FFFFFF",
|
||||
background_color: "rgba(0, 0, 0, 0.7)",
|
||||
text_align: "left",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "caption",
|
||||
style: {
|
||||
font_family: "Arial",
|
||||
font_size: 32,
|
||||
font_color: "#FFFFFF",
|
||||
background_color: "rgba(0, 0, 0, 0.5)",
|
||||
text_align: "center",
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
function createTextOverlay(
|
||||
text: string,
|
||||
templateName: string,
|
||||
position: Position,
|
||||
duration?: { start: number; end: number }
|
||||
): TextOverlay {
|
||||
const template = templates.find((t) => t.name === templateName);
|
||||
|
||||
if (!template) {
|
||||
throw new Error(`Template "${templateName}" not found`);
|
||||
}
|
||||
|
||||
return {
|
||||
text,
|
||||
x: position.x,
|
||||
y: position.y,
|
||||
...template.style,
|
||||
duration,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Timing Text Overlays
|
||||
|
||||
Coordinate text appearance with your script:
|
||||
|
||||
```typescript
|
||||
// Script with timing markers
|
||||
const script = `
|
||||
Hello and welcome. [0:00 - 0:03]
|
||||
Let me show you our features. [0:03 - 0:08]
|
||||
First, we have analytics. [0:08 - 0:15]
|
||||
Get started today! [0:15 - 0:20]
|
||||
`;
|
||||
|
||||
// Matching text overlays
|
||||
const overlays = [
|
||||
{
|
||||
text: "Welcome",
|
||||
duration: { start: 0, end: 3 },
|
||||
...titleStyle,
|
||||
},
|
||||
{
|
||||
text: "Feature Overview",
|
||||
duration: { start: 3, end: 8 },
|
||||
...subtitleStyle,
|
||||
},
|
||||
{
|
||||
text: "Analytics Dashboard",
|
||||
duration: { start: 8, end: 15 },
|
||||
...lowerThirdStyle,
|
||||
},
|
||||
{
|
||||
text: "www.example.com",
|
||||
duration: { start: 15, end: 20 },
|
||||
...ctaStyle,
|
||||
},
|
||||
];
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Readability** - Use sufficient contrast between text and background
|
||||
2. **Size** - Ensure text is large enough to read on mobile devices
|
||||
3. **Duration** - Give viewers enough time to read (rule of thumb: 3 seconds minimum)
|
||||
4. **Positioning** - Don't overlap with the avatar's face
|
||||
5. **Consistency** - Use consistent fonts and styles throughout
|
||||
6. **Accessibility** - Consider color-blind friendly palettes
|
||||
|
||||
## Limitations
|
||||
|
||||
- Text overlay support varies by subscription tier
|
||||
- Some advanced styling options may not be available via API
|
||||
- Complex animations may require post-production tools
|
||||
- For auto-generated captions, see [captions.md](captions.md)
|
||||
@@ -0,0 +1,347 @@
|
||||
---
|
||||
name: video-agent
|
||||
description: One-shot prompt video generation with HeyGen Video Agent API
|
||||
---
|
||||
|
||||
# Video Agent API
|
||||
|
||||
The Video Agent API generates complete videos from a single text prompt. Unlike the standard video generation API which requires detailed scene-by-scene configuration, Video Agent automatically handles script writing, avatar selection, visuals, voiceover, pacing, and captions.
|
||||
|
||||
## MCP Tool (Preferred)
|
||||
|
||||
If the HeyGen MCP server is connected, use `mcp__heygen__generate_video_agent` instead of direct API calls:
|
||||
|
||||
```
|
||||
Tool: mcp__heygen__generate_video_agent
|
||||
Parameters:
|
||||
prompt: "<optimized prompt from prompt-optimizer.md>"
|
||||
config:
|
||||
duration_sec: 90 # optional, 5-300
|
||||
avatar_id: "avatar_id" # optional, agent selects if omitted
|
||||
orientation: "landscape" # optional, "landscape" or "portrait"
|
||||
files: # optional
|
||||
- asset_id: "uploaded_asset_id"
|
||||
```
|
||||
|
||||
Then check status with `mcp__heygen__get_video` using the returned `video_id`.
|
||||
|
||||
The prompt quality is still the critical factor — always follow [prompt-optimizer.md](prompt-optimizer.md) regardless of whether you use MCP or direct API.
|
||||
|
||||
## When to Use Video Agent vs Standard API
|
||||
|
||||
| Use Case | Recommended API |
|
||||
|----------|-----------------|
|
||||
| Quick video from idea | Video Agent |
|
||||
| Precise control over scenes, avatars, timing | Standard v2/video/generate |
|
||||
| Automated content generation at scale | Video Agent |
|
||||
| Specific avatar with exact script | Standard v2/video/generate |
|
||||
| Prototype or draft video | Video Agent |
|
||||
| Brand-consistent production video | Standard v2/video/generate |
|
||||
|
||||
## Before You Call This API
|
||||
|
||||
**Required step:** Optimize your prompt using [prompt-optimizer.md](prompt-optimizer.md) before generating a video. The difference between mediocre and professional results depends entirely on prompt quality.
|
||||
|
||||
Quick checklist:
|
||||
1. Define visual style (colors, aesthetic) — see [visual-styles.md](visual-styles.md)
|
||||
2. Structure scenes with specific scene types
|
||||
3. Write VO script at ~150 words/minute
|
||||
4. Specify media types for each scene (Motion Graphics, Stock, AI-generated)
|
||||
|
||||
## Direct API Endpoint
|
||||
|
||||
```
|
||||
POST https://api.heygen.com/v1/video_agent/generate
|
||||
```
|
||||
|
||||
## Request Fields
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `prompt` | string | ✓ | Text prompt describing the video you want |
|
||||
| `config` | object | | Configuration options (see below) |
|
||||
| `files` | array | | Asset files to reference in generation |
|
||||
| `callback_id` | string | | Custom ID for tracking. **Requires `callback_url` to also be set** — omit both if you don't need webhooks |
|
||||
| `callback_url` | string | | Webhook URL for completion notification |
|
||||
|
||||
### Config Object
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `duration_sec` | integer | Approximate duration in seconds (5-300) |
|
||||
| `avatar_id` | string | Specific avatar to use (optional - agent selects if not provided) |
|
||||
| `orientation` | string | `"portrait"` or `"landscape"` |
|
||||
|
||||
### Files Array
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `asset_id` | string | Asset ID of uploaded file to reference |
|
||||
|
||||
## Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"video_id": "abc123"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## curl Example
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v1/video_agent/generate" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"prompt": "Create a 60-second product demo video for a new AI-powered calendar app. The tone should be professional but friendly, targeting busy professionals. Highlight the smart scheduling feature and time zone handling."
|
||||
}'
|
||||
```
|
||||
|
||||
## TypeScript
|
||||
|
||||
```typescript
|
||||
interface VideoAgentConfig {
|
||||
duration_sec?: number; // 5-300 seconds
|
||||
avatar_id?: string; // Optional: specific avatar
|
||||
orientation?: "portrait" | "landscape";
|
||||
}
|
||||
|
||||
interface VideoAgentFile {
|
||||
asset_id: string;
|
||||
}
|
||||
|
||||
interface VideoAgentRequest {
|
||||
prompt: string; // Required
|
||||
config?: VideoAgentConfig;
|
||||
files?: VideoAgentFile[];
|
||||
callback_id?: string; // Requires callback_url if set
|
||||
callback_url?: string;
|
||||
}
|
||||
|
||||
interface VideoAgentResponse {
|
||||
error: string | null;
|
||||
data: {
|
||||
video_id: string;
|
||||
};
|
||||
}
|
||||
|
||||
async function generateWithVideoAgent(
|
||||
prompt: string,
|
||||
config?: VideoAgentConfig
|
||||
): Promise<string> {
|
||||
const request: VideoAgentRequest = { prompt };
|
||||
|
||||
if (config) {
|
||||
request.config = config;
|
||||
}
|
||||
|
||||
const response = await fetch(
|
||||
"https://api.heygen.com/v1/video_agent/generate",
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(request),
|
||||
}
|
||||
);
|
||||
|
||||
const json: VideoAgentResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(`Video Agent failed: ${json.error}`);
|
||||
}
|
||||
|
||||
return json.data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
## Python
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
def generate_with_video_agent(
|
||||
prompt: str,
|
||||
duration_sec: Optional[int] = None,
|
||||
avatar_id: Optional[str] = None,
|
||||
orientation: Optional[str] = None
|
||||
) -> str:
|
||||
request_body = {"prompt": prompt}
|
||||
|
||||
config = {}
|
||||
if duration_sec:
|
||||
config["duration_sec"] = duration_sec
|
||||
if avatar_id:
|
||||
config["avatar_id"] = avatar_id
|
||||
if orientation:
|
||||
config["orientation"] = orientation
|
||||
|
||||
if config:
|
||||
request_body["config"] = config
|
||||
|
||||
response = requests.post(
|
||||
"https://api.heygen.com/v1/video_agent/generate",
|
||||
headers={
|
||||
"X-Api-Key": os.environ["HEYGEN_API_KEY"],
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
json=request_body
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
if data.get("error"):
|
||||
raise Exception(f"Video Agent failed: {data['error']}")
|
||||
|
||||
return data["data"]["video_id"]
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Basic: Prompt Only
|
||||
|
||||
```typescript
|
||||
const videoId = await generateWithVideoAgent(
|
||||
"Create a 30-second welcome video for new employees at a tech startup. Keep it energetic and modern."
|
||||
);
|
||||
```
|
||||
|
||||
### With Duration and Orientation
|
||||
|
||||
```typescript
|
||||
const videoId = await generateWithVideoAgent(
|
||||
"Explain the benefits of cloud computing for small businesses. Use simple language and real-world examples.",
|
||||
{
|
||||
duration_sec: 90,
|
||||
orientation: "landscape"
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
### With Specific Avatar
|
||||
|
||||
```typescript
|
||||
const videoId = await generateWithVideoAgent(
|
||||
"Present quarterly sales results. Professional tone, data-focused.",
|
||||
{
|
||||
duration_sec: 120,
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
orientation: "landscape"
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
### With Reference Files
|
||||
|
||||
Upload assets first, then reference them:
|
||||
|
||||
```typescript
|
||||
// 1. Upload reference materials (see assets.md)
|
||||
const logoAssetId = await uploadFile("./company-logo.png", "image/png");
|
||||
const productImageId = await uploadFile("./product-screenshot.png", "image/png");
|
||||
|
||||
// 2. Generate video with references
|
||||
const response = await fetch(
|
||||
"https://api.heygen.com/v1/video_agent/generate",
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
prompt: "Create a product demo video showcasing our new dashboard feature. Use the uploaded screenshots as visual references.",
|
||||
config: {
|
||||
duration_sec: 60,
|
||||
orientation: "landscape"
|
||||
},
|
||||
files: [
|
||||
{ asset_id: logoAssetId },
|
||||
{ asset_id: productImageId }
|
||||
]
|
||||
}),
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
## Writing Effective Prompts
|
||||
|
||||
See **[prompt-optimizer.md](prompt-optimizer.md)** for comprehensive prompt writing guidance.
|
||||
|
||||
The prompt optimizer covers:
|
||||
- Prompt complexity levels (basic → scene-by-scene)
|
||||
- Visual style taxonomy and color specification
|
||||
- Media type selection (Motion Graphics vs Stock vs AI-generated)
|
||||
- Scene structure and timing calculations
|
||||
- Ready-to-use templates for common video types
|
||||
|
||||
## Checking Video Status
|
||||
|
||||
Video Agent returns a `video_id` - use the standard status endpoint to check progress:
|
||||
|
||||
```typescript
|
||||
// Same polling as standard video generation
|
||||
const videoUrl = await waitForVideo(videoId);
|
||||
```
|
||||
|
||||
See [video-status.md](video-status.md) for polling implementation.
|
||||
|
||||
## Comparison: Video Agent vs Standard API
|
||||
|
||||
### Video Agent Request
|
||||
```typescript
|
||||
// Simple: describe what you want
|
||||
const videoId = await generateWithVideoAgent(
|
||||
"Create a 60-second tutorial on setting up two-factor authentication. Professional tone, step-by-step."
|
||||
);
|
||||
```
|
||||
|
||||
### Equivalent Standard API Request
|
||||
```typescript
|
||||
// Complex: specify every detail
|
||||
const videoId = await generateVideo({
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Welcome to this tutorial on two-factor authentication...",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#1a1a2e",
|
||||
},
|
||||
},
|
||||
// ... more scenes for each step
|
||||
],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
});
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- Less control over exact script wording
|
||||
- Avatar selection may vary if not specified
|
||||
- Scene composition is automated
|
||||
- May not match precise brand guidelines
|
||||
- Duration is approximate, not exact
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Be specific in prompts** - More detail = better results
|
||||
2. **Specify duration** - Use `config.duration_sec` for predictable length
|
||||
3. **Lock avatar if needed** - Use `config.avatar_id` for consistency
|
||||
4. **Upload reference files** - Help agent understand your brand/product
|
||||
5. **Iterate on prompts** - Refine based on results
|
||||
6. **Use for drafts** - Video Agent is great for quick iterations before final production
|
||||
@@ -0,0 +1,770 @@
|
||||
---
|
||||
name: video-generation
|
||||
description: POST /v2/video/generate workflow and multi-scene videos for HeyGen
|
||||
---
|
||||
|
||||
# Video Generation
|
||||
|
||||
## Table of Contents
|
||||
- [Video Output Formats](#video-output-formats)
|
||||
- [Basic Video Generation](#basic-video-generation)
|
||||
- [Request Fields](#request-fields)
|
||||
- [Video Configuration Options](#video-configuration-options)
|
||||
- [Multi-Scene Videos](#multi-scene-videos)
|
||||
- [Using Different Character Types](#using-different-character-types)
|
||||
- [Voice Input Types](#voice-input-types)
|
||||
- [Complete Workflow Example](#complete-workflow-example)
|
||||
- [Error Handling](#error-handling)
|
||||
- [Script Length Limits](#script-length-limits)
|
||||
- [Adding Pauses to Scripts](#adding-pauses-to-scripts)
|
||||
- [Test Mode](#test-mode)
|
||||
- [Production-Ready Workflow](#production-ready-workflow)
|
||||
- [Transparent Background Videos (WebM)](#transparent-background-videos-webm)
|
||||
- [Best Practices](#best-practices)
|
||||
|
||||
---
|
||||
|
||||
The `/v2/video/generate` endpoint is the primary way to create AI avatar videos with HeyGen.
|
||||
|
||||
## Video Output Formats
|
||||
|
||||
| Endpoint | Format | Use Case |
|
||||
|----------|--------|----------|
|
||||
| `/v2/video/generate` | MP4 | **Standard** - videos with background (most common) |
|
||||
| `/v1/video.webm` | WebM | Transparent background - only when needed |
|
||||
|
||||
Use MP4 with background for most cases. WebM is only needed when you want to see content *behind* the avatar (e.g., overlaying avatar on a screen recording).
|
||||
|
||||
## Basic Video Generation
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v2/video/generate" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"video_inputs": [
|
||||
{
|
||||
"character": {
|
||||
"type": "avatar",
|
||||
"avatar_id": "josh_lite3_20230714",
|
||||
"avatar_style": "normal"
|
||||
},
|
||||
"voice": {
|
||||
"type": "text",
|
||||
"input_text": "Hello! Welcome to HeyGen.",
|
||||
"voice_id": "1bd001e7e50f421d891986aad5158bc8"
|
||||
}
|
||||
}
|
||||
],
|
||||
"dimension": {
|
||||
"width": 1920,
|
||||
"height": 1080
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
## Request Fields
|
||||
|
||||
### Top-Level Fields
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `video_inputs` | array | ✓ | Array of 1-50 video input objects |
|
||||
| `dimension` | object | | Video dimensions `{width, height}` |
|
||||
| `title` | string | | Video name for organization |
|
||||
| `test` | boolean | | Test mode (watermarked, no credits) |
|
||||
| `caption` | boolean | | Enable auto-captions |
|
||||
| `callback_id` | string | | Custom ID for webhook tracking |
|
||||
| `callback_url` | string | | URL for completion notification |
|
||||
| `folder_id` | string | | Storage folder ID |
|
||||
|
||||
### video_inputs[].character Fields
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `type` | string | ✓ | `"avatar"` or `"talking_photo"` |
|
||||
| `avatar_id` | string | ✓* | Avatar ID (*required when type is "avatar") |
|
||||
| `talking_photo_id` | string | ✓* | Photo ID (*required when type is "talking_photo") |
|
||||
| `avatar_style` | string | | `"normal"`, `"closeUp"`, or `"circle"` |
|
||||
| `scale` | number | | Avatar scale factor |
|
||||
| `offset` | object | | Position offset `{x, y}` |
|
||||
|
||||
### video_inputs[].voice Fields
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `type` | string | ✓ | `"text"`, `"audio"`, or `"silence"` |
|
||||
| `voice_id` | string | ✓* | Voice ID (*required when type is "text") |
|
||||
| `input_text` | string | ✓* | Script text (*required when type is "text") |
|
||||
| `audio_url` | string | ✓* | Audio URL (*required when type is "audio") |
|
||||
| `duration` | number | ✓* | Duration in seconds (*required when type is "silence") |
|
||||
| `speed` | number | | Speech speed 0.5-2.0 (default 1.0) |
|
||||
| `pitch` | number | | Voice pitch -20 to 20 (default 0) |
|
||||
|
||||
### video_inputs[].background Fields
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `type` | string | | `"color"`, `"image"`, or `"video"` |
|
||||
| `value` | string | | Hex color (when type is "color") |
|
||||
| `url` | string | | Image/video URL (when type is "image"/"video") |
|
||||
| `fit` | string | | `"cover"` or `"contain"` |
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
// Required fields have no '?' - optional fields have '?'
|
||||
interface VideoInput {
|
||||
character: {
|
||||
type: "avatar" | "talking_photo"; // Required
|
||||
avatar_id?: string; // Required when type="avatar"
|
||||
talking_photo_id?: string; // Required when type="talking_photo"
|
||||
avatar_style?: "normal" | "closeUp" | "circle";
|
||||
scale?: number;
|
||||
offset?: { x: number; y: number };
|
||||
};
|
||||
voice: {
|
||||
type: "text" | "audio" | "silence"; // Required
|
||||
input_text?: string; // Required when type="text"
|
||||
voice_id?: string; // Required when type="text"
|
||||
audio_url?: string; // Required when type="audio"
|
||||
duration?: number; // Required when type="silence"
|
||||
speed?: number;
|
||||
pitch?: number;
|
||||
};
|
||||
background?: {
|
||||
type?: "color" | "image" | "video";
|
||||
value?: string;
|
||||
url?: string;
|
||||
fit?: "cover" | "contain";
|
||||
};
|
||||
}
|
||||
|
||||
interface VideoGenerateRequest {
|
||||
video_inputs: VideoInput[]; // Required
|
||||
dimension?: { width: number; height: number };
|
||||
test?: boolean;
|
||||
title?: string;
|
||||
caption?: boolean;
|
||||
callback_id?: string;
|
||||
callback_url?: string;
|
||||
folder_id?: string;
|
||||
}
|
||||
|
||||
interface VideoGenerateResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
video_id: string;
|
||||
};
|
||||
}
|
||||
|
||||
async function generateVideo(config: VideoGenerateRequest): Promise<string> {
|
||||
const response = await fetch("https://api.heygen.com/v2/video/generate", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(config),
|
||||
});
|
||||
|
||||
const json: VideoGenerateResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
|
||||
def generate_video(config: dict) -> str:
|
||||
response = requests.post(
|
||||
"https://api.heygen.com/v2/video/generate",
|
||||
headers={
|
||||
"X-Api-Key": os.environ["HEYGEN_API_KEY"],
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
json=config
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
if data.get("error"):
|
||||
raise Exception(data["error"])
|
||||
|
||||
return data["data"]["video_id"]
|
||||
```
|
||||
|
||||
## Video Configuration Options
|
||||
|
||||
### Full Configuration Example
|
||||
|
||||
```typescript
|
||||
const fullConfig: VideoGenerateRequest = {
|
||||
// Test mode (no credits consumed, watermarked output)
|
||||
test: false,
|
||||
|
||||
// Video title (for organization)
|
||||
title: "Product Demo Video",
|
||||
|
||||
// Video dimensions
|
||||
dimension: {
|
||||
width: 1920,
|
||||
height: 1080,
|
||||
},
|
||||
|
||||
// Video scenes/inputs
|
||||
video_inputs: [
|
||||
{
|
||||
// Avatar configuration
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
|
||||
// Voice configuration
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Welcome to our product demonstration!",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
speed: 1.0,
|
||||
pitch: 0,
|
||||
},
|
||||
|
||||
// Background configuration
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#FFFFFF",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
## Multi-Scene Videos
|
||||
|
||||
Create videos with multiple scenes:
|
||||
|
||||
```typescript
|
||||
const multiSceneConfig = {
|
||||
video_inputs: [
|
||||
// Scene 1: Introduction
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello! Today I'll show you three key features.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#1a1a2e",
|
||||
},
|
||||
},
|
||||
// Scene 2: Feature 1
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "closeUp",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "First, let's look at our dashboard.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "image",
|
||||
url: "https://example.com/dashboard-bg.jpg",
|
||||
},
|
||||
},
|
||||
// Scene 3: Conclusion
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Thanks for watching! Try it today.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#1a1a2e",
|
||||
},
|
||||
},
|
||||
],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
};
|
||||
```
|
||||
|
||||
## Using Different Character Types
|
||||
|
||||
### Avatar
|
||||
|
||||
```typescript
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Talking Photo
|
||||
|
||||
```typescript
|
||||
{
|
||||
character: {
|
||||
type: "talking_photo",
|
||||
talking_photo_id: "your_talking_photo_id"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Voice Input Types
|
||||
|
||||
### Text-to-Speech
|
||||
|
||||
```typescript
|
||||
{
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Your script here",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
speed: 1.0, // 0.5 - 2.0
|
||||
pitch: 0 // -20 to 20
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Audio
|
||||
|
||||
```typescript
|
||||
{
|
||||
voice: {
|
||||
type: "audio",
|
||||
audio_url: "https://example.com/your-audio.mp3"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Complete Workflow Example
|
||||
|
||||
```typescript
|
||||
async function createVideo(script: string, avatarId: string, voiceId: string) {
|
||||
// 1. Generate video
|
||||
console.log("Starting video generation...");
|
||||
const videoId = await generateVideo({
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: avatarId,
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: voiceId,
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#FFFFFF",
|
||||
},
|
||||
},
|
||||
],
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
});
|
||||
|
||||
console.log(`Video ID: ${videoId}`);
|
||||
|
||||
// 2. Poll for completion
|
||||
console.log("Waiting for video completion...");
|
||||
const videoUrl = await waitForVideo(videoId);
|
||||
|
||||
console.log(`Video ready: ${videoUrl}`);
|
||||
return videoUrl;
|
||||
}
|
||||
|
||||
// Helper function for polling
|
||||
async function waitForVideo(videoId: string): Promise<string> {
|
||||
const maxAttempts = 60;
|
||||
const pollInterval = 10000; // 10 seconds
|
||||
|
||||
for (let i = 0; i < maxAttempts; i++) {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/videos/${videoId}`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const { data } = await response.json();
|
||||
|
||||
if (data.status === "completed") {
|
||||
return data.video_url;
|
||||
} else if (data.status === "failed") {
|
||||
throw new Error(data.failure_message || "Video generation failed");
|
||||
}
|
||||
|
||||
await new Promise((r) => setTimeout(r, pollInterval));
|
||||
}
|
||||
|
||||
throw new Error("Video generation timed out");
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```typescript
|
||||
async function generateVideoSafe(config: VideoGenerateRequest) {
|
||||
try {
|
||||
const videoId = await generateVideo(config);
|
||||
return { success: true, videoId };
|
||||
} catch (error) {
|
||||
// Common errors
|
||||
if (error.message.includes("quota")) {
|
||||
console.error("Insufficient credits");
|
||||
} else if (error.message.includes("avatar")) {
|
||||
console.error("Invalid avatar ID");
|
||||
} else if (error.message.includes("voice")) {
|
||||
console.error("Invalid voice ID");
|
||||
} else if (error.message.includes("script")) {
|
||||
console.error("Script too long or invalid");
|
||||
}
|
||||
|
||||
return { success: false, error: error.message };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Script Length Limits
|
||||
|
||||
| Tier | Max Characters |
|
||||
|------|----------------|
|
||||
| Free | ~500 |
|
||||
| Creator | ~1,500 |
|
||||
| Team | ~3,000 |
|
||||
| Enterprise | ~5,000+ |
|
||||
|
||||
## Adding Pauses to Scripts
|
||||
|
||||
Use `<break>` tags to add pauses in your script:
|
||||
|
||||
```typescript
|
||||
const script = "Welcome to our demo. <break time=\"1s\"/> Let me show you the features.";
|
||||
```
|
||||
|
||||
**Format:** `<break time="Xs"/>` where X is seconds (e.g., `1s`, `1.5s`, `0.5s`)
|
||||
|
||||
**Important:** Break tags must have spaces before and after them.
|
||||
|
||||
See [voices.md](voices.md) for detailed break tag documentation.
|
||||
|
||||
## Test Mode
|
||||
|
||||
Use test mode during development:
|
||||
|
||||
```typescript
|
||||
const config = {
|
||||
test: true, // Watermarked output, no credits consumed
|
||||
video_inputs: [...],
|
||||
};
|
||||
```
|
||||
|
||||
## Production-Ready Workflow
|
||||
|
||||
Complete example using avatar's default voice (recommended), proper timeouts, and retry logic:
|
||||
|
||||
```typescript
|
||||
interface VideoGenerationResult {
|
||||
videoId: string;
|
||||
videoUrl: string;
|
||||
duration: number;
|
||||
avatarId: string;
|
||||
voiceId: string;
|
||||
avatarName: string;
|
||||
}
|
||||
|
||||
async function generateAvatarVideo(
|
||||
script: string,
|
||||
options: {
|
||||
avatarId?: string; // Specific avatar, or will pick first available
|
||||
width?: number;
|
||||
height?: number;
|
||||
} = {}
|
||||
): Promise<VideoGenerationResult> {
|
||||
const { width = 1920, height = 1080 } = options;
|
||||
let { avatarId } = options;
|
||||
|
||||
// 1. List avatars if no specific one provided
|
||||
if (!avatarId) {
|
||||
console.log("Listing available avatars...");
|
||||
const listResponse = await fetch("https://api.heygen.com/v2/avatars", {
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
});
|
||||
const listData = await listResponse.json();
|
||||
|
||||
if (!listData.data?.avatars?.length) {
|
||||
throw new Error("No avatars available");
|
||||
}
|
||||
avatarId = listData.data.avatars[0].avatar_id;
|
||||
}
|
||||
|
||||
// 2. Get avatar details including default_voice_id
|
||||
console.log(`Getting details for avatar: ${avatarId}`);
|
||||
const detailsResponse = await fetch(
|
||||
`https://api.heygen.com/v2/avatar/${avatarId}/details`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
const { data: avatar } = await detailsResponse.json();
|
||||
|
||||
if (!avatar.default_voice_id) {
|
||||
throw new Error(`Avatar ${avatar.name} has no default voice - select voice manually`);
|
||||
}
|
||||
|
||||
console.log(`Using avatar: ${avatar.name} with default voice: ${avatar.default_voice_id}`);
|
||||
|
||||
// 3. Generate video using avatar's default voice
|
||||
const videoId = await generateVideo({
|
||||
video_inputs: [{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: avatar.id, // from details response
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: avatar.default_voice_id, // pre-matched default voice
|
||||
speed: 1.0,
|
||||
},
|
||||
background: {
|
||||
type: "color",
|
||||
value: "#1a1a2e",
|
||||
},
|
||||
}],
|
||||
dimension: { width, height },
|
||||
});
|
||||
|
||||
console.log(`Video ID: ${videoId}`);
|
||||
|
||||
// 3. Wait for completion (20 minute timeout - generation can take 15+ min)
|
||||
console.log("Waiting for video generation (typically 5-15 minutes, can be longer)...");
|
||||
const result = await waitForVideo(
|
||||
videoId,
|
||||
process.env.HEYGEN_API_KEY!,
|
||||
(status, elapsed) => {
|
||||
console.log(` [${Math.round(elapsed / 1000)}s] ${status}`);
|
||||
},
|
||||
1200000 // 20 minute timeout for safety
|
||||
);
|
||||
|
||||
return {
|
||||
videoId,
|
||||
videoUrl: result.video_url!,
|
||||
duration: result.duration!,
|
||||
avatarId: avatar.id,
|
||||
voiceId: avatar.default_voice_id,
|
||||
avatarName: avatar.name,
|
||||
};
|
||||
}
|
||||
|
||||
// Usage - let it pick an avatar automatically
|
||||
const result = await generateAvatarVideo(
|
||||
"Hello! Welcome to our product demonstration."
|
||||
);
|
||||
console.log(`Video ready: ${result.videoUrl}`);
|
||||
|
||||
// Or specify a known avatar_id
|
||||
const result2 = await generateAvatarVideo(
|
||||
"Hello! Welcome to our product demonstration.",
|
||||
{ avatarId: "josh_lite3_20230714" }
|
||||
);
|
||||
```
|
||||
|
||||
## Transparent Background Videos (WebM)
|
||||
|
||||
Use WebM **only when you need transparency** - i.e., when the avatar should be overlaid on other video content and you need to see through to what's behind.
|
||||
|
||||
**Don't need WebM for:**
|
||||
- Avatar with motion graphics/text overlaid ON TOP of avatar
|
||||
- Picture-in-picture with solid background
|
||||
- Standard presenter videos
|
||||
|
||||
**Do need WebM for:**
|
||||
- Avatar overlaid on screen recording
|
||||
- Avatar floating over video background
|
||||
- True alpha-channel compositing
|
||||
|
||||
### WebM Request Fields
|
||||
|
||||
**Note:** The WebM endpoint (`/v1/video.webm`) uses a different structure than `/v2/video/generate`.
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `avatar_pose_id` | string | ✓ | Avatar pose ID (from avatar details) |
|
||||
| `avatar_style` | string | ✓ | `"normal"` or `"closeUp"` only (no circle) |
|
||||
| `input_text` | string | ✓* | Script text (*required if not using input_audio) |
|
||||
| `voice_id` | string | ✓* | Voice ID (*required with input_text) |
|
||||
| `input_audio` | string | ✓* | Audio URL (*required if not using input_text) |
|
||||
| `dimension` | object | | `{width, height}` (default: 1280x720) |
|
||||
|
||||
**Either** (`input_text` + `voice_id`) **OR** `input_audio` must be provided, but not both.
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v1/video.webm" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"avatar_pose_id": "josh_lite3_20230714",
|
||||
"avatar_style": "normal",
|
||||
"input_text": "Hello! This video has a transparent background.",
|
||||
"voice_id": "1bd001e7e50f421d891986aad5158bc8",
|
||||
"dimension": {
|
||||
"width": 1920,
|
||||
"height": 1080
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface WebMVideoRequest {
|
||||
avatar_pose_id: string; // Required
|
||||
avatar_style: "normal" | "closeUp"; // Required (no circle support)
|
||||
input_text?: string; // Required if not using input_audio
|
||||
voice_id?: string; // Required with input_text
|
||||
input_audio?: string; // Required if not using input_text
|
||||
dimension?: { width: number; height: number };
|
||||
}
|
||||
|
||||
async function generateTransparentVideo(
|
||||
script: string,
|
||||
avatarPoseId: string,
|
||||
voiceId: string
|
||||
): Promise<string> {
|
||||
const response = await fetch("https://api.heygen.com/v1/video.webm", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
avatar_pose_id: avatarPoseId, // Required
|
||||
avatar_style: "normal", // Required: "normal" or "closeUp"
|
||||
input_text: script, // Required (with voice_id)
|
||||
voice_id: voiceId, // Required (with input_text)
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
}),
|
||||
});
|
||||
|
||||
const { data } = await response.json();
|
||||
return data.video_id;
|
||||
}
|
||||
```
|
||||
|
||||
### When to Use WebM vs MP4
|
||||
|
||||
| Scenario | Format | Why |
|
||||
|----------|--------|-----|
|
||||
| Avatar with overlays on top | **MP4** | Overlays go on top, don't need transparency |
|
||||
| Standard presenter | **MP4** | Simpler, more compatible |
|
||||
| Loom-style (avatar over screen recording) | **WebM** + `normal`/`closeUp` | Need transparency, crop to circle in post |
|
||||
| Avatar floating over video content | **WebM** | Need to see content behind avatar |
|
||||
|
||||
**Note:** WebM only supports `normal` and `closeUp` styles. Circle style is not supported for WebM - apply circular masking in your video editor/Remotion instead.
|
||||
|
||||
### WebM Example: Loom-Style (Avatar Over Screen Recording)
|
||||
|
||||
Generate with `normal` or `closeUp` style (circle not supported for WebM):
|
||||
|
||||
```typescript
|
||||
// Generate avatar with transparent background
|
||||
const videoId = await fetch("https://api.heygen.com/v1/video.webm", {
|
||||
method: "POST",
|
||||
headers: { "X-Api-Key": apiKey, "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
avatar_pose_id: avatarPoseId, // Required
|
||||
avatar_style: "closeUp", // Required: "normal" or "closeUp" only
|
||||
input_text: script, // Required (with voice_id)
|
||||
voice_id: voiceId, // Required (with input_text)
|
||||
dimension: { width: 1920, height: 1080 },
|
||||
}),
|
||||
}).then(r => r.json()).then(d => d.data.video_id);
|
||||
```
|
||||
|
||||
Apply circular masking in Remotion:
|
||||
|
||||
```tsx
|
||||
import { Video, AbsoluteFill } from "remotion";
|
||||
|
||||
export const LoomStyleVideo: React.FC<{
|
||||
screenRecordingUrl: string;
|
||||
avatarWebmUrl: string;
|
||||
}> = ({ screenRecordingUrl, avatarWebmUrl }) => {
|
||||
return (
|
||||
<AbsoluteFill>
|
||||
{/* Screen recording as base layer */}
|
||||
<Video src={screenRecordingUrl} style={{ width: "100%", height: "100%" }} />
|
||||
|
||||
{/* Avatar with circular mask applied in CSS */}
|
||||
<Video
|
||||
src={avatarWebmUrl}
|
||||
style={{
|
||||
position: "absolute",
|
||||
bottom: 20,
|
||||
left: 20,
|
||||
width: 150,
|
||||
height: 150,
|
||||
borderRadius: "50%", // Circular mask
|
||||
overflow: "hidden",
|
||||
objectFit: "cover",
|
||||
}}
|
||||
/>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### Note on Status Polling
|
||||
|
||||
WebM videos use the same status endpoint as MP4:
|
||||
|
||||
```typescript
|
||||
// Same polling as regular videos
|
||||
const status = await getVideoStatus(videoId);
|
||||
// status.video_url will be a .webm file
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Preview avatars before generating** - Download `preview_image_url` so user can see what the avatar looks like before committing to a video (see [avatars.md](avatars.md))
|
||||
2. **Use avatar's default voice** - Most avatars have a `default_voice_id` that's pre-matched for natural results (see [avatars.md](avatars.md))
|
||||
2. **Fallback: match gender manually** - If no default voice, ensure avatar and voice genders match (see [voices.md](voices.md))
|
||||
3. **Validate inputs** - Check avatar and voice IDs before generating
|
||||
4. **Use test mode** - Test configurations without consuming credits
|
||||
5. **Set generous timeouts** - Use 15-20 minutes; generation often takes 10-15 min, sometimes longer
|
||||
6. **Consider async patterns** - For long videos, save video_id and check status later (see [video-status.md](video-status.md))
|
||||
7. **Handle errors gracefully** - Implement proper error handling
|
||||
8. **Monitor progress** - Implement polling with progress feedback
|
||||
9. **Optimize scripts** - Keep scripts concise and natural
|
||||
10. **Consider dimensions** - Match dimensions to your use case (see [dimensions.md](dimensions.md))
|
||||
@@ -0,0 +1,502 @@
|
||||
---
|
||||
name: video-status
|
||||
description: Polling patterns, status types, and retrieving download URLs for HeyGen videos
|
||||
---
|
||||
|
||||
# Video Status and Polling
|
||||
|
||||
After generating a video, you need to poll for status until the video is complete. HeyGen processes videos asynchronously.
|
||||
|
||||
## MCP Tool (Preferred)
|
||||
|
||||
If the HeyGen MCP server is connected, use `mcp__heygen__get_video` with the `videoId` parameter. It returns status, video_url, thumbnail_url, duration, title, gif_url, captioned_video_url, and other metadata in a single call.
|
||||
|
||||
## Checking Video Status (Direct API)
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/videos/YOUR_VIDEO_ID" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface VideoStatusResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
id: string;
|
||||
status: "pending" | "processing" | "completed" | "failed";
|
||||
video_url?: string;
|
||||
thumbnail_url?: string;
|
||||
duration?: number;
|
||||
title?: string;
|
||||
created_at?: string;
|
||||
completed_at?: string;
|
||||
gif_url?: string;
|
||||
captioned_video_url?: string;
|
||||
subtitle_url?: string;
|
||||
folder_id?: string;
|
||||
output_language?: string;
|
||||
failure_code?: string;
|
||||
failure_message?: string;
|
||||
};
|
||||
}
|
||||
|
||||
async function getVideoStatus(videoId: string): Promise<VideoStatusResponse["data"]> {
|
||||
const response = await fetch(
|
||||
`https://api.heygen.com/v2/videos/${videoId}`,
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
|
||||
const json: VideoStatusResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data;
|
||||
}
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
|
||||
def get_video_status(video_id: str) -> dict:
|
||||
response = requests.get(
|
||||
f"https://api.heygen.com/v2/videos/{video_id}",
|
||||
headers={"X-Api-Key": os.environ["HEYGEN_API_KEY"]}
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
if data.get("error"):
|
||||
raise Exception(data["error"])
|
||||
|
||||
return data["data"]
|
||||
```
|
||||
|
||||
## Video Status Types
|
||||
|
||||
| Status | Description |
|
||||
|--------|-------------|
|
||||
| `pending` | Video is queued for processing |
|
||||
| `processing` | Video is being generated |
|
||||
| `completed` | Video is ready for download |
|
||||
| `failed` | Video generation failed |
|
||||
|
||||
## Expected Generation Times
|
||||
|
||||
Video generation typically takes **5-15 minutes**, but can exceed 20 minutes during peak load or for longer scripts.
|
||||
|
||||
| Factor | Impact |
|
||||
|--------|--------|
|
||||
| Script length | Longer scripts = significantly longer processing |
|
||||
| Resolution | 1080p takes longer than 720p |
|
||||
| Avatar complexity | Some avatars render faster |
|
||||
| Queue load | Peak hours may cause 15-20+ minute waits |
|
||||
| Multiple scenes | Each scene adds processing time |
|
||||
|
||||
**Recommendations**:
|
||||
- Set timeout to **15-20 minutes** (900,000-1,200,000 ms) for safety
|
||||
- For scripts > 2 minutes of speech, expect 15+ minutes
|
||||
- Consider async patterns (save video_id, check later) for long videos
|
||||
|
||||
## Response Format
|
||||
|
||||
### Completed Video
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"id": "abc123",
|
||||
"status": "completed",
|
||||
"video_url": "https://files.heygen.ai/video/abc123.mp4",
|
||||
"thumbnail_url": "https://files.heygen.ai/thumbnail/abc123.jpg",
|
||||
"duration": 45.2,
|
||||
"title": "My Video",
|
||||
"created_at": "2024-01-15T10:30:00Z",
|
||||
"completed_at": "2024-01-15T10:38:00Z",
|
||||
"gif_url": "https://files.heygen.ai/gif/abc123.gif",
|
||||
"captioned_video_url": null,
|
||||
"subtitle_url": null,
|
||||
"folder_id": null,
|
||||
"output_language": "en"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Failed Video
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"id": "abc123",
|
||||
"status": "failed",
|
||||
"failure_code": "script_too_long",
|
||||
"failure_message": "Script too long for selected avatar"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Polling Implementation
|
||||
|
||||
### Basic Polling
|
||||
|
||||
```typescript
|
||||
async function waitForVideo(
|
||||
videoId: string,
|
||||
maxWaitMs = 600000, // 10 minutes
|
||||
pollIntervalMs = 5000 // 5 seconds
|
||||
): Promise<string> {
|
||||
const startTime = Date.now();
|
||||
|
||||
while (Date.now() - startTime < maxWaitMs) {
|
||||
const status = await getVideoStatus(videoId);
|
||||
|
||||
switch (status.status) {
|
||||
case "completed":
|
||||
return status.video_url!;
|
||||
case "failed":
|
||||
throw new Error(status.failure_message || "Video generation failed");
|
||||
case "pending":
|
||||
case "processing":
|
||||
await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error("Video generation timed out");
|
||||
}
|
||||
```
|
||||
|
||||
### Polling with Progress Callback
|
||||
|
||||
```typescript
|
||||
type ProgressCallback = (status: string, elapsed: number) => void;
|
||||
|
||||
async function waitForVideoWithProgress(
|
||||
videoId: string,
|
||||
onProgress?: ProgressCallback,
|
||||
maxWaitMs = 600000,
|
||||
pollIntervalMs = 5000
|
||||
): Promise<string> {
|
||||
const startTime = Date.now();
|
||||
|
||||
while (Date.now() - startTime < maxWaitMs) {
|
||||
const elapsed = Date.now() - startTime;
|
||||
const status = await getVideoStatus(videoId);
|
||||
|
||||
onProgress?.(status.status, elapsed);
|
||||
|
||||
switch (status.status) {
|
||||
case "completed":
|
||||
return status.video_url!;
|
||||
case "failed":
|
||||
throw new Error(status.failure_message || "Video generation failed");
|
||||
default:
|
||||
await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error("Video generation timed out");
|
||||
}
|
||||
|
||||
// Usage
|
||||
const videoUrl = await waitForVideoWithProgress(
|
||||
videoId,
|
||||
(status, elapsed) => {
|
||||
console.log(`Status: ${status}, Elapsed: ${Math.round(elapsed / 1000)}s`);
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
### Python Polling
|
||||
|
||||
```python
|
||||
import time
|
||||
from typing import Optional, Callable
|
||||
|
||||
def wait_for_video(
|
||||
video_id: str,
|
||||
max_wait_seconds: int = 600,
|
||||
poll_interval: int = 5,
|
||||
on_progress: Optional[Callable[[str, int], None]] = None
|
||||
) -> str:
|
||||
start_time = time.time()
|
||||
|
||||
while time.time() - start_time < max_wait_seconds:
|
||||
elapsed = int(time.time() - start_time)
|
||||
status_data = get_video_status(video_id)
|
||||
status = status_data["status"]
|
||||
|
||||
if on_progress:
|
||||
on_progress(status, elapsed)
|
||||
|
||||
if status == "completed":
|
||||
return status_data["video_url"]
|
||||
elif status == "failed":
|
||||
raise Exception(status_data.get("failure_message", "Video generation failed"))
|
||||
|
||||
time.sleep(poll_interval)
|
||||
|
||||
raise Exception("Video generation timed out")
|
||||
|
||||
|
||||
# Usage
|
||||
def progress_callback(status: str, elapsed: int):
|
||||
print(f"Status: {status}, Elapsed: {elapsed}s")
|
||||
|
||||
video_url = wait_for_video(video_id, on_progress=progress_callback)
|
||||
```
|
||||
|
||||
## Downloading the Video
|
||||
|
||||
Once the video is complete, download it. **Important**: The video URL may not be immediately available after status shows "completed". Use retry logic with backoff.
|
||||
|
||||
### TypeScript (with retry)
|
||||
|
||||
```typescript
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
|
||||
async function downloadVideoWithRetry(
|
||||
videoUrl: string,
|
||||
outputPath = "./output/video.mp4",
|
||||
maxRetries = 5,
|
||||
initialDelayMs = 2000
|
||||
): Promise<void> {
|
||||
let lastError: Error | null = null;
|
||||
|
||||
for (let attempt = 0; attempt < maxRetries; attempt++) {
|
||||
try {
|
||||
const response = await fetch(videoUrl);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
|
||||
const arrayBuffer = await response.arrayBuffer();
|
||||
fs.writeFileSync(path.resolve(outputPath), Buffer.from(arrayBuffer));
|
||||
console.log(`Video downloaded to ${outputPath}`);
|
||||
return;
|
||||
} catch (error) {
|
||||
lastError = error as Error;
|
||||
const delay = initialDelayMs * Math.pow(2, attempt); // Exponential backoff
|
||||
console.log(`Download attempt ${attempt + 1} failed, retrying in ${delay}ms...`);
|
||||
await new Promise((resolve) => setTimeout(resolve, delay));
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(`Failed to download after ${maxRetries} attempts: ${lastError?.message}`);
|
||||
}
|
||||
```
|
||||
|
||||
### Python (with retry)
|
||||
|
||||
```python
|
||||
import requests
|
||||
import time
|
||||
|
||||
def download_video_with_retry(
|
||||
video_url: str,
|
||||
output_path: str,
|
||||
max_retries: int = 5,
|
||||
initial_delay: float = 2.0
|
||||
) -> None:
|
||||
last_error = None
|
||||
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
response = requests.get(video_url, stream=True, timeout=60)
|
||||
response.raise_for_status()
|
||||
|
||||
with open(output_path, "wb") as f:
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
|
||||
print(f"Video downloaded to {output_path}")
|
||||
return
|
||||
except Exception as e:
|
||||
last_error = e
|
||||
delay = initial_delay * (2 ** attempt) # Exponential backoff
|
||||
print(f"Download attempt {attempt + 1} failed, retrying in {delay}s...")
|
||||
time.sleep(delay)
|
||||
|
||||
raise Exception(f"Failed to download after {max_retries} attempts: {last_error}")
|
||||
```
|
||||
|
||||
### Simple Download (no retry)
|
||||
|
||||
For quick scripts where you'll retry manually:
|
||||
|
||||
```typescript
|
||||
async function downloadVideo(videoUrl: string, outputPath = "./output/video.mp4") {
|
||||
const response = await fetch(videoUrl);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to download: ${response.status}`);
|
||||
}
|
||||
const arrayBuffer = await response.arrayBuffer();
|
||||
fs.writeFileSync(path.resolve(outputPath), Buffer.from(arrayBuffer));
|
||||
}
|
||||
```
|
||||
|
||||
## Complete Workflow Example
|
||||
|
||||
```typescript
|
||||
async function generateAndDownloadVideo(config: VideoConfig): Promise<string> {
|
||||
// 1. Generate video
|
||||
const generateResponse = await fetch(
|
||||
"https://api.heygen.com/v2/video/generate",
|
||||
{
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(config),
|
||||
}
|
||||
);
|
||||
|
||||
const { data: generateData } = await generateResponse.json();
|
||||
const videoId = generateData.video_id;
|
||||
console.log(`Video ID: ${videoId}`);
|
||||
|
||||
// 2. Poll for completion
|
||||
const videoUrl = await waitForVideoWithProgress(
|
||||
videoId,
|
||||
(status, elapsed) => {
|
||||
console.log(`[${Math.round(elapsed / 1000)}s] Status: ${status}`);
|
||||
}
|
||||
);
|
||||
|
||||
// 3. Download
|
||||
const outputPath = `./output/${videoId}.mp4`;
|
||||
await downloadVideo(videoUrl, outputPath);
|
||||
|
||||
return outputPath;
|
||||
}
|
||||
```
|
||||
|
||||
## Resumable Status Checking
|
||||
|
||||
For long-running generations, save the video_id and check status later rather than keeping a process waiting.
|
||||
|
||||
### Save State After Generation
|
||||
|
||||
```typescript
|
||||
interface PendingVideo {
|
||||
videoId: string;
|
||||
createdAt: string;
|
||||
script: string;
|
||||
avatarId: string;
|
||||
voiceId: string;
|
||||
}
|
||||
|
||||
async function startVideoGeneration(config: VideoGenerateRequest): Promise<PendingVideo> {
|
||||
const videoId = await generateVideo(config);
|
||||
|
||||
const pending: PendingVideo = {
|
||||
videoId,
|
||||
createdAt: new Date().toISOString(),
|
||||
script: config.video_inputs[0].voice.input_text!,
|
||||
avatarId: config.video_inputs[0].character.avatar_id!,
|
||||
voiceId: config.video_inputs[0].voice.voice_id!,
|
||||
};
|
||||
|
||||
// Save to file for later retrieval
|
||||
fs.writeFileSync("pending-video.json", JSON.stringify(pending, null, 2));
|
||||
console.log(`Video generation started. ID: ${videoId}`);
|
||||
console.log("Check status later with: checkVideoStatus()");
|
||||
|
||||
return pending;
|
||||
}
|
||||
```
|
||||
|
||||
### Check Status Later
|
||||
|
||||
```typescript
|
||||
async function checkVideoStatus(): Promise<void> {
|
||||
if (!fs.existsSync("pending-video.json")) {
|
||||
console.log("No pending video found");
|
||||
return;
|
||||
}
|
||||
|
||||
const pending: PendingVideo = JSON.parse(
|
||||
fs.readFileSync("pending-video.json", "utf-8")
|
||||
);
|
||||
|
||||
const elapsed = Date.now() - new Date(pending.createdAt).getTime();
|
||||
console.log(`Checking video ${pending.videoId} (started ${Math.round(elapsed / 60000)} min ago)...`);
|
||||
|
||||
const status = await getVideoStatus(pending.videoId);
|
||||
|
||||
switch (status.status) {
|
||||
case "completed":
|
||||
console.log(`Video ready: ${status.video_url}`);
|
||||
console.log(`Duration: ${status.duration}s`);
|
||||
// Clean up pending file
|
||||
fs.unlinkSync("pending-video.json");
|
||||
// Save result
|
||||
fs.writeFileSync("video-result.json", JSON.stringify({
|
||||
...pending,
|
||||
videoUrl: status.video_url,
|
||||
thumbnailUrl: status.thumbnail_url,
|
||||
duration: status.duration,
|
||||
title: status.title,
|
||||
createdAt: status.created_at,
|
||||
completedAt: status.completed_at,
|
||||
}, null, 2));
|
||||
break;
|
||||
case "failed":
|
||||
console.error(`Video failed: ${status.failure_message}`);
|
||||
fs.unlinkSync("pending-video.json");
|
||||
break;
|
||||
default:
|
||||
console.log(`Status: ${status.status} - check again in a few minutes`);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### CLI-Friendly Pattern
|
||||
|
||||
```typescript
|
||||
// generate-video.ts - Start generation and exit
|
||||
async function main() {
|
||||
const pending = await startVideoGeneration(config);
|
||||
console.log(`\nVideo ID saved. Run 'npx tsx check-status.ts' to check progress.`);
|
||||
process.exit(0); // Exit immediately, don't wait
|
||||
}
|
||||
|
||||
// check-status.ts - Check and optionally wait
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const shouldWait = args.includes("--wait");
|
||||
|
||||
if (shouldWait) {
|
||||
// Poll until complete (with 20 min timeout)
|
||||
const result = await waitForVideo(pending.videoId, apiKey, onProgress, 1200000);
|
||||
console.log(`Done: ${result.video_url}`);
|
||||
} else {
|
||||
// Just check once and report
|
||||
await checkVideoStatus();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Alternative: Using Webhooks
|
||||
|
||||
Instead of polling, you can use webhooks to receive notifications when videos complete. See [webhooks.md](webhooks.md) for details. Webhooks are ideal for production systems where you don't want to maintain polling connections.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use exponential backoff** - Increase poll intervals for long-running jobs
|
||||
2. **Set reasonable timeouts** - Most videos complete within 10 minutes
|
||||
3. **Handle failures gracefully** - Check error messages for actionable feedback
|
||||
4. **Consider webhooks** - For production systems, webhooks are more efficient than polling
|
||||
5. **Cache video URLs** - Downloaded video URLs are valid for a limited time
|
||||
@@ -0,0 +1,367 @@
|
||||
---
|
||||
name: visual-styles
|
||||
description: 20 named visual styles for Video Agent prompts — each with colors, typography, motion, and transitions
|
||||
---
|
||||
|
||||
# Visual Style Library — 20 Styles
|
||||
|
||||
Named visual styles for Video Agent prompts. Each is inspired by a real graphic designer. Ordered by mood intensity.
|
||||
|
||||
**Picking a style:** Match mood first, content second. Ask: *"What should the viewer FEEL?"*
|
||||
|
||||
**Using a style:** Copy the style block into your prompt's STYLE section. Use the visual language rules — don't inject the example B-roll scenes (they confuse the agent).
|
||||
|
||||
**Custom styles:** These are examples. Create your own by combining elements, referencing other designers, art movements, or cultural aesthetics. The pattern: **named style + designer reference + color palette + typography + motion rules + transitions.**
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| # | Style | Artist | Mood | Best For |
|
||||
|---|---|---|---|---|
|
||||
| 1 | Soft Signal | Sagmeister | Intimate, warm | Personal stories, wellness |
|
||||
| 2 | Warm Grain | Eksell | Organic, friendly | Environmental, sustainability |
|
||||
| 3 | Quiet Drama | Ray | Humanist, contemplative | Profiles, biographical |
|
||||
| 4 | Heritage Reel | Cassandre | Nostalgic, vintage | History, retrospectives |
|
||||
| 5 | Silk Route | Abedini | Flowing, mysterious | Global affairs, cross-cultural |
|
||||
| 6 | Swiss Pulse | Müller-Brockmann | Clinical, precise | Data-heavy, analytical |
|
||||
| 7 | Geometric Bold | Tanaka | Minimal, elegant | Lifestyle, visual essays |
|
||||
| 8 | Velvet Standard | Vignelli | Premium, timeless | Luxury, investor updates |
|
||||
| 9 | Digital Grid | Crouwel | Systematic, technical | Infrastructure, engineering |
|
||||
| 10 | Contact Sheet | Brodovitch | Editorial, investigative | Journalism, deep dives |
|
||||
| 11 | Folk Frequency | Terrazas | Cultural, vivid | Festivals, food, heritage |
|
||||
| 12 | Earth Pulse | Ghariokwu | Grounded, communal | Community, grassroots |
|
||||
| 13 | Dream State | Tomaszewski | Surreal, poetic | Op-eds, philosophy |
|
||||
| 14 | Play Mode | Ahn Sang-soo | Playful, irreverent | Entertainment, pop culture |
|
||||
| 15 | Carnival Surge | Lins | Euphoric, celebratory | Milestones, hype |
|
||||
| 16 | Shadow Cut | Hillmann | Dark, cinematic | Exposés, investigations |
|
||||
| 17 | Deconstructed | Brody | Industrial, raw | Tech news, punk energy |
|
||||
| 18 | Maximalist Type | Scher | Loud, kinetic | Big announcements, launches |
|
||||
| 19 | Data Drift | Anadol | Futuristic, immersive | AI/tech, innovation |
|
||||
| 20 | Red Wire | Tartakover | Urgent, immediate | Breaking news, crisis |
|
||||
|
||||
## Mood-to-Style Guide
|
||||
|
||||
| Content feels... | Use... |
|
||||
|---|---|
|
||||
| Personal, intimate | Soft Signal, Quiet Drama |
|
||||
| Natural, earthy | Warm Grain, Earth Pulse |
|
||||
| Nostalgic, historical | Heritage Reel |
|
||||
| Data-driven, analytical | Swiss Pulse, Digital Grid |
|
||||
| Elegant, premium | Velvet Standard, Geometric Bold |
|
||||
| Cultural, global | Silk Route, Folk Frequency |
|
||||
| Investigative, serious | Contact Sheet, Shadow Cut |
|
||||
| Fun, lighthearted | Play Mode, Carnival Surge |
|
||||
| Philosophical, abstract | Dream State |
|
||||
| Punk, grassroots, raw | Deconstructed |
|
||||
| Hype, loud, high-energy | Maximalist Type |
|
||||
| Tech-forward, futuristic | Data Drift |
|
||||
| Breaking, urgent | Red Wire |
|
||||
|
||||
---
|
||||
|
||||
## 1. Soft Signal — Stefan Sagmeister
|
||||
|
||||
**Mood:** Intimate, warm | **Best for:** Personal stories, wellness, reflections
|
||||
|
||||
- Warm amber and cream with dusty rose, sage green, honey gold accents
|
||||
- Handwritten-style text overlays — personal, lowercase, delicate
|
||||
- Close-up framing: hands, faces, textures. Macro lens feel
|
||||
- Slow drifts and floats, never snaps. Soft dissolves, warm light leaks
|
||||
|
||||
```
|
||||
STYLE — SOFT SIGNAL (Sagmeister): Warm amber/cream, dusty rose, sage green.
|
||||
Handwritten-style text. Close-up framing. Slow drifts and floats.
|
||||
Soft dissolves with warm light leaks.
|
||||
```
|
||||
|
||||
## 2. Warm Grain — Olle Eksell
|
||||
|
||||
**Mood:** Organic, friendly | **Best for:** Environmental, sustainability, community
|
||||
|
||||
- Earth tones: ochre, forest green, terracotta, cream, soft brown
|
||||
- Rounded sans-serif type. Organic rounded compositions — nothing angular
|
||||
- 16mm film grain, slightly desaturated. Natural textures: wood, linen, stone
|
||||
- Gentle wipes, soft cuts, unhurried
|
||||
|
||||
```
|
||||
STYLE — WARM GRAIN (Eksell): Earth tones — ochre, forest green, terracotta, cream.
|
||||
Organic rounded compositions. 16mm film grain. Rounded sans-serif.
|
||||
Gentle wipes and soft cuts.
|
||||
```
|
||||
|
||||
## 3. Quiet Drama — Satyajit Ray
|
||||
|
||||
**Mood:** Humanist, contemplative | **Best for:** Profiles, biographical, cultural
|
||||
|
||||
- Muted warm: sepia, deep brown, soft gold, off-white, charcoal
|
||||
- Clean serif type, positioned with care. Portrait framing
|
||||
- Strong single-source contrast: window light, single lamp
|
||||
- Deliberate pacing, longer holds. Slow fades to black
|
||||
|
||||
```
|
||||
STYLE — QUIET DRAMA (Ray): Muted warm — sepia, deep brown, soft gold.
|
||||
Portrait framing. Clean serif. Strong single-source contrast.
|
||||
Slow fades to black.
|
||||
```
|
||||
|
||||
## 4. Heritage Reel — Cassandre
|
||||
|
||||
**Mood:** Nostalgic, vintage | **Best for:** History, retrospectives, brand origins
|
||||
|
||||
- Faded gold, deep burgundy, navy, cream, sepia wash
|
||||
- Elegant centered serif like classic film title cards
|
||||
- Vignetting, softened edges. Film grain, light scratches, gentle jitter
|
||||
- Iris wipes, film reel flicker
|
||||
|
||||
```
|
||||
STYLE — HERITAGE REEL (Cassandre): Faded gold, burgundy, navy, sepia wash.
|
||||
Elegant centered serif. Vignetting and aged film grain.
|
||||
Iris wipe transitions.
|
||||
```
|
||||
|
||||
## 5. Silk Route — Reza Abedini
|
||||
|
||||
**Mood:** Flowing, mysterious | **Best for:** Global affairs, cross-cultural, art/design
|
||||
|
||||
- Rich jewel tones: deep teal, burgundy, gold, lapis blue, black
|
||||
- Elegant spaced type along natural visual lines
|
||||
- Layered compositions — foreground, midground, background all active
|
||||
- Flowing dissolves, smooth morphs
|
||||
|
||||
```
|
||||
STYLE — SILK ROUTE (Abedini): Jewel tones — deep teal, burgundy, gold, lapis blue.
|
||||
Layered compositions, all depths active. Elegant spaced type.
|
||||
Flowing dissolves and smooth morphs.
|
||||
```
|
||||
|
||||
## 6. Swiss Pulse — Josef Müller-Brockmann
|
||||
|
||||
**Mood:** Clinical, precise | **Best for:** Data-heavy, analytical, financial, metrics
|
||||
|
||||
- Black (#1a1a1a), white, ONE accent: electric blue (#0066FF)
|
||||
- Helvetica Bold headlines, Regular labels. Numbers LARGE (80-120pt)
|
||||
- Grid-locked compositions. Every element snaps to 12-column grid
|
||||
- Animated counters COUNT UP from 0. Diagonal compositions on key moments
|
||||
- Grid wipes, hard cuts. No dissolves
|
||||
|
||||
```
|
||||
STYLE — SWISS PULSE (Müller-Brockmann): Black/white + electric blue #0066FF.
|
||||
Grid-locked. Helvetica Bold. Animated counters. Diagonal accents.
|
||||
Grid wipe transitions.
|
||||
```
|
||||
|
||||
## 7. Geometric Bold — Ikko Tanaka
|
||||
|
||||
**Mood:** Minimal, elegant | **Best for:** Clean lifestyle, culture, visual essays, brand profiles
|
||||
|
||||
- Maximum 3 flat colors per frame — no gradients
|
||||
- Bold clean type as primary visual element
|
||||
- Asymmetric composition, 60% negative space minimum. Single focal point
|
||||
- Clean cuts on beat, no effects
|
||||
|
||||
```
|
||||
STYLE — GEOMETRIC BOLD (Tanaka): Max 3 flat colors per frame.
|
||||
60% negative space. Bold type as primary element.
|
||||
Single focal point. Clean cuts on beat.
|
||||
```
|
||||
|
||||
## 8. Velvet Standard — Massimo Vignelli
|
||||
|
||||
**Mood:** Premium, timeless | **Best for:** Luxury, investor updates, keynotes, product showcases
|
||||
|
||||
- Black, white, ONE rich accent: deep navy (#1a237e) or gold (#c9a84c)
|
||||
- Thin sans-serif, ALL CAPS, letter-spaced wide
|
||||
- Generous negative space. Symmetrical, centered, architectural precision
|
||||
- Slow, deliberate. Sequential reveals. Elegant cross-dissolves
|
||||
|
||||
```
|
||||
STYLE — VELVET STANDARD (Vignelli): Black, white, one accent: gold #c9a84c.
|
||||
Thin ALL CAPS, wide spacing. Generous negative space.
|
||||
Slow elegant cross-dissolves.
|
||||
```
|
||||
|
||||
## 9. Digital Grid — Wim Crouwel
|
||||
|
||||
**Mood:** Systematic, technical | **Best for:** Infrastructure, engineering, code, tech
|
||||
|
||||
- Dark (#0a0a0a) with cyan (#00E5FF), amber (#FFB300), green (#00FF88)
|
||||
- Monospaced type throughout. Code-terminal aesthetic
|
||||
- Pixel grid overlays visible. Everything snaps to system
|
||||
- Grid nodes light up sequentially. Scan-line effects, cursor blinks
|
||||
- Clean wipe transitions
|
||||
|
||||
```
|
||||
STYLE — DIGITAL GRID (Crouwel): Monospaced type. Dark #0a0a0a with cyan #00E5FF, amber #FFB300.
|
||||
Pixel grid overlays. Terminal aesthetic. Clean wipe transitions.
|
||||
```
|
||||
|
||||
## 10. Contact Sheet — Alexey Brodovitch
|
||||
|
||||
**Mood:** Editorial, investigative | **Best for:** Journalism, deep dives, research breakdowns
|
||||
|
||||
- High contrast B&W with occasional desaturated color accents
|
||||
- Bold sans-serif captions like editorial annotations
|
||||
- Photo-editorial framing — multiple images, contact-sheet energy
|
||||
- Raw grain, imperfect focus. Tight crops on faces and hands
|
||||
- Hard cuts on beat, snap-zooms
|
||||
|
||||
```
|
||||
STYLE — CONTACT SHEET (Brodovitch): High contrast B&W, desaturated accents.
|
||||
Photo-editorial framing. Bold sans-serif annotations. Raw grain.
|
||||
Hard cuts on beat. Snap-zooms.
|
||||
```
|
||||
|
||||
## 11. Folk Frequency — Eduardo Terrazas
|
||||
|
||||
**Mood:** Cultural, vivid | **Best for:** Cultural events, food, tradition, heritage
|
||||
|
||||
- Vivid folk: hot pink, bright orange, cobalt blue, sun yellow, emerald
|
||||
- Bold warm rounded type. Pattern and repetition — folk art rhythms
|
||||
- Rich textures: woven fabrics, painted surfaces, ceramic, handmade
|
||||
- Colorful wipes, quick cuts on festive rhythm
|
||||
|
||||
```
|
||||
STYLE — FOLK FREQUENCY (Terrazas): Vivid folk — hot pink, cobalt blue, sun yellow, emerald.
|
||||
Bold rounded type. Folk art rhythms. Rich handmade textures.
|
||||
Colorful wipes on festive rhythm.
|
||||
```
|
||||
|
||||
## 12. Earth Pulse — Lemi Ghariokwu
|
||||
|
||||
**Mood:** Grounded, communal | **Best for:** Community, music/culture, grassroots
|
||||
|
||||
- Warm saturated: burnt orange, deep green, rich yellow, terracotta
|
||||
- Bold expressive type, center-frame. Wide community framing
|
||||
- Rhythmic editing timed to musical beats
|
||||
- Rhythmic cuts on beat, freeze-frames for emphasis
|
||||
|
||||
```
|
||||
STYLE — EARTH PULSE (Ghariokwu): Warm saturated — burnt orange, deep green, rich yellow.
|
||||
Bold expressive type. Wide community framing.
|
||||
Rhythmic cuts on beat. Freeze-frames.
|
||||
```
|
||||
|
||||
## 13. Dream State — Henryk Tomaszewski
|
||||
|
||||
**Mood:** Surreal, poetic | **Best for:** Op-eds, philosophy, think pieces, speculative
|
||||
|
||||
- Muted palette with one surreal accent: dusty blues, grey-greens, then shock of red or gold
|
||||
- Sparse precise text — few words, maximum impact. Thin elegant floating type
|
||||
- Unusual juxtapositions. Dreamlike quality: soft edges, atmospheric haze
|
||||
- Slow morph dissolves. NEVER hard cuts
|
||||
|
||||
```
|
||||
STYLE — DREAM STATE (Tomaszewski): Muted palette + one surreal accent.
|
||||
Thin elegant floating type. Soft edges, atmospheric haze.
|
||||
Slow morph dissolves — NEVER hard cuts.
|
||||
```
|
||||
|
||||
## 14. Play Mode — Ahn Sang-soo
|
||||
|
||||
**Mood:** Playful, irreverent | **Best for:** Entertainment, pop culture, listicles, fun
|
||||
|
||||
- Bright candy: electric blue, hot pink, lime green, yellow, white
|
||||
- Bouncy oversized tilted text. Asymmetric off-kilter compositions
|
||||
- Quick cuts (1-3 seconds). Score cards, achievement popups, XP bars
|
||||
- Bouncy spring physics — text overshoots and settles, screen shakes
|
||||
- Pop cuts, whip pans, bounce effects
|
||||
|
||||
```
|
||||
STYLE — PLAY MODE (Ahn Sang-soo): Electric blue, hot pink, lime green.
|
||||
Bouncy spring physics. Oversized tilted text. Score cards, XP bars.
|
||||
Pop cuts, bounce effects.
|
||||
```
|
||||
|
||||
## 15. Carnival Surge — Rico Lins
|
||||
|
||||
**Mood:** Euphoric, celebratory | **Best for:** Big announcements, milestones, celebrations, hype
|
||||
|
||||
- Maximum color: hot pink (#FF1493), electric yellow (#FFE000), teal (#00CED1), orange, violet
|
||||
- MASSIVE bold text at ANGLES over footage. Collage-style overlapping
|
||||
- Rapid 1-2 second clips. Confetti, lights, constant energy
|
||||
- Smash cuts, flash frames, rapid-fire montage
|
||||
|
||||
```
|
||||
STYLE — CARNIVAL SURGE (Lins): Max color — hot pink #FF1493, yellow #FFE000, teal #00CED1.
|
||||
Collage layering. Text MASSIVE at ANGLES. Confetti bursts.
|
||||
Smash cuts, flash frames.
|
||||
```
|
||||
|
||||
## 16. Shadow Cut — Hans Hillmann
|
||||
|
||||
**Mood:** Dark, cinematic | **Best for:** Exposés, investigations, controversy, dark deep dives
|
||||
|
||||
- Near-monochrome: deep blacks, cold greys, stark white + blood red or toxic green
|
||||
- Sharp angular text like film noir title cards
|
||||
- Heavy shadow — faces half-lit, objects emerging from darkness
|
||||
- Slow creeping push-ins, slow reveals, tension
|
||||
- Iris to black, slow fade from darkness, hard cuts to silence
|
||||
|
||||
```
|
||||
STYLE — SHADOW CUT (Hillmann): Deep blacks, cold greys + blood red accent.
|
||||
Sharp angular text. Heavy shadow. Slow creeping push-ins.
|
||||
Hard cuts to black. Film noir tension.
|
||||
```
|
||||
|
||||
## 17. Deconstructed — Neville Brody
|
||||
|
||||
**Mood:** Industrial, raw | **Best for:** Tech news, security, punk energy, counter-culture
|
||||
|
||||
- Dark grey (#1a1a1a), black, rust orange (#D4501E), raw white (#f0f0f0)
|
||||
- Type at angles, overlapping edges, escaping frames. Bold industrial
|
||||
- High contrast, gritty textures: scratched metal, peeling paint, scan-line glitch
|
||||
- Text SLAMS, SHATTERS, PUNCHES. Letters scramble then snap
|
||||
- Smash cuts, glitch transitions, white flash frames
|
||||
|
||||
```
|
||||
STYLE — DECONSTRUCTED (Brody): Dark grey #1a1a1a, rust orange #D4501E.
|
||||
Type at angles, overlapping. Gritty textures, scan-line glitch.
|
||||
Smash cuts with flash frames.
|
||||
```
|
||||
|
||||
## 18. Maximalist Type — Paula Scher
|
||||
|
||||
**Mood:** Loud, kinetic | **Best for:** Big announcements, launches, high-energy recaps
|
||||
|
||||
- Bold saturated: red, yellow, black, white — maximum contrast
|
||||
- Text IS the visual. Overlapping layers at different scales and angles, 50-80% of frame
|
||||
- Kinetic energy: everything moving, slamming, sliding. 1-2 second rapid cuts
|
||||
- Text layered OVER footage — never empty backgrounds
|
||||
- Smash cuts, text slamming from edges, flash frames
|
||||
|
||||
```
|
||||
STYLE — MAXIMALIST TYPE (Scher): Red, yellow, black, white — max contrast.
|
||||
Text IS the visual. Overlapping at different scales, 50-80% of frame.
|
||||
Kinetic everything. Smash cuts, flash frames.
|
||||
```
|
||||
|
||||
## 19. Data Drift — Refik Anadol
|
||||
|
||||
**Mood:** Futuristic, immersive | **Best for:** AI/tech, speculative, cutting-edge science
|
||||
|
||||
- Iridescent: holographic silver, electric purple (#7c3aed), cyan (#06b6d4), deep black (#0a0a0a)
|
||||
- Thin futuristic sans-serif — minimal, floating, weightless
|
||||
- Fluid morphing compositions. Extreme scale shifts: microscopic to cosmic
|
||||
- Particles coalesce into numbers, light traces data paths
|
||||
- Liquid dissolves, particles dispersing and reforming
|
||||
|
||||
```
|
||||
STYLE — DATA DRIFT (Anadol): Iridescent — purple #7c3aed, cyan #06b6d4, deep black.
|
||||
Fluid morphing compositions. Thin futuristic type.
|
||||
Liquid dissolves. Particles coalesce into numbers.
|
||||
```
|
||||
|
||||
## 20. Red Wire — David Tartakover
|
||||
|
||||
**Mood:** Urgent, immediate | **Best for:** Breaking news, crisis updates, alerts
|
||||
|
||||
- High alert: red, black, white, emergency yellow — maximum contrast
|
||||
- Bold condensed all caps — every word screams urgency
|
||||
- Split screens, ticker-style text bars, timestamp overlays — max information density
|
||||
- Multiple text elements simultaneously. Handheld energy
|
||||
- Snap cuts, flash frames, zero breathing room
|
||||
|
||||
```
|
||||
STYLE — RED WIRE (Tartakover): Red, black, white, emergency yellow.
|
||||
Bold condensed all-caps. Split screens, tickers, timestamps.
|
||||
Snap cuts, flash frames. Zero breathing room.
|
||||
```
|
||||
@@ -0,0 +1,505 @@
|
||||
---
|
||||
name: voices
|
||||
description: Listing voices, locales, speed/pitch configuration for HeyGen
|
||||
---
|
||||
|
||||
# HeyGen Voices
|
||||
|
||||
HeyGen provides a wide variety of AI voices for different languages, accents, and styles. Voices convert your text script into natural-sounding speech.
|
||||
|
||||
## Listing Available Voices
|
||||
|
||||
### curl
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.heygen.com/v2/voices" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY"
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface Voice {
|
||||
voice_id: string;
|
||||
name: string;
|
||||
language: string;
|
||||
gender: "male" | "female";
|
||||
preview_audio: string;
|
||||
support_pause: boolean;
|
||||
emotion_support: boolean;
|
||||
}
|
||||
|
||||
interface VoicesResponse {
|
||||
error: null | string;
|
||||
data: {
|
||||
voices: Voice[];
|
||||
};
|
||||
}
|
||||
|
||||
async function listVoices(): Promise<Voice[]> {
|
||||
const response = await fetch("https://api.heygen.com/v2/voices", {
|
||||
headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! },
|
||||
});
|
||||
|
||||
const json: VoicesResponse = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
|
||||
return json.data.voices;
|
||||
}
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
|
||||
def list_voices() -> list:
|
||||
response = requests.get(
|
||||
"https://api.heygen.com/v2/voices",
|
||||
headers={"X-Api-Key": os.environ["HEYGEN_API_KEY"]}
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
if data.get("error"):
|
||||
raise Exception(data["error"])
|
||||
|
||||
return data["data"]["voices"]
|
||||
```
|
||||
|
||||
## Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"error": null,
|
||||
"data": {
|
||||
"voices": [
|
||||
{
|
||||
"voice_id": "1bd001e7e50f421d891986aad5158bc8",
|
||||
"name": "Sara",
|
||||
"language": "English",
|
||||
"gender": "female",
|
||||
"preview_audio": "https://files.heygen.ai/...",
|
||||
"support_pause": true,
|
||||
"emotion_support": true
|
||||
},
|
||||
{
|
||||
"voice_id": "de8b5d78f2e0485f88d1e9f5c8e7f9a6",
|
||||
"name": "Paul",
|
||||
"language": "English",
|
||||
"gender": "male",
|
||||
"preview_audio": "https://files.heygen.ai/...",
|
||||
"support_pause": true,
|
||||
"emotion_support": false
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Supported Languages
|
||||
|
||||
HeyGen supports many languages including:
|
||||
|
||||
| Language | Code | Notes |
|
||||
|----------|------|-------|
|
||||
| English (US) | en-US | Multiple voice options |
|
||||
| English (UK) | en-GB | British accent |
|
||||
| Spanish | es-ES | Spain Spanish |
|
||||
| Spanish (Latin) | es-MX | Mexican Spanish |
|
||||
| French | fr-FR | France French |
|
||||
| German | de-DE | Standard German |
|
||||
| Portuguese | pt-BR | Brazilian Portuguese |
|
||||
| Chinese (Mandarin) | zh-CN | Simplified Chinese |
|
||||
| Japanese | ja-JP | Standard Japanese |
|
||||
| Korean | ko-KR | Standard Korean |
|
||||
| Italian | it-IT | Standard Italian |
|
||||
| Dutch | nl-NL | Standard Dutch |
|
||||
| Polish | pl-PL | Standard Polish |
|
||||
| Arabic | ar-SA | Saudi Arabic |
|
||||
|
||||
## Using Voices in Video Generation
|
||||
|
||||
### Basic Voice Usage
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello! Welcome to our presentation.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Voice with Speed Adjustment
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "This is spoken at a faster pace.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
speed: 1.2, // 1.0 is normal, range: 0.5 - 2.0
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Voice with Pitch Adjustment
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "This has a higher pitch.",
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
pitch: 10, // Range: -20 to 20
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
## Adding Pauses with Break Tags
|
||||
|
||||
HeyGen supports SSML-style `<break>` tags to add pauses in scripts.
|
||||
|
||||
### Break Tag Format
|
||||
|
||||
```
|
||||
<break time="Xs"/>
|
||||
```
|
||||
|
||||
Where `X` is the duration in seconds (e.g., `1s`, `1.5s`, `0.5s`).
|
||||
|
||||
### Requirements
|
||||
|
||||
| Rule | Example |
|
||||
|------|---------|
|
||||
| Use seconds with "s" suffix | `<break time="1.5s"/>` ✓ |
|
||||
| Must have space before tag | `word <break time="1s"/>` ✓ |
|
||||
| Must have space after tag | `<break time="1s"/> word` ✓ |
|
||||
| Self-closing tag | `<break time="1s"/>` ✓ |
|
||||
|
||||
**Incorrect:** `word<break time="1s"/>word` (no spaces)
|
||||
**Correct:** `word <break time="1s"/> word`
|
||||
|
||||
### Examples
|
||||
|
||||
```typescript
|
||||
// Single pause
|
||||
const script1 = "Hello and welcome. <break time=\"1s\"/> Let me introduce our product.";
|
||||
|
||||
// Multiple pauses
|
||||
const script2 = "First point. <break time=\"1.5s\"/> Second point. <break time=\"1s\"/> Third point.";
|
||||
|
||||
// Pause at start (dramatic opening)
|
||||
const script3 = "<break time=\"0.5s\"/> Welcome to our presentation.";
|
||||
|
||||
// Longer pause for emphasis
|
||||
const script4 = "And the winner is... <break time=\"2s\"/> You!";
|
||||
```
|
||||
|
||||
### Full Example
|
||||
|
||||
```typescript
|
||||
const scriptWithPauses = `
|
||||
Welcome to our product demo. <break time="1s"/>
|
||||
Today I'll show you three key features. <break time="0.5s"/>
|
||||
First, let's look at the dashboard. <break time="1.5s"/>
|
||||
As you can see, it's incredibly intuitive.
|
||||
`;
|
||||
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: scriptWithPauses,
|
||||
voice_id: "1bd001e7e50f421d891986aad5158bc8",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### Consecutive Breaks
|
||||
|
||||
Multiple consecutive break tags are automatically combined:
|
||||
|
||||
```typescript
|
||||
// These two breaks:
|
||||
"Hello <break time=\"1s\"/> <break time=\"0.5s\"/> world"
|
||||
|
||||
// Are treated as a single 1.5s pause
|
||||
```
|
||||
|
||||
### Best Practices
|
||||
|
||||
1. **Use for emphasis** - Add pauses before important points
|
||||
2. **Keep pauses reasonable** - 0.5s to 2s is typical; longer feels unnatural
|
||||
3. **Match natural speech** - Add pauses where a human would breathe or pause
|
||||
4. **Test the output** - Listen to generated audio to verify timing feels right
|
||||
|
||||
## Using Custom Audio Instead of TTS
|
||||
|
||||
Instead of text-to-speech, you can provide your own audio:
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "audio",
|
||||
audio_url: "https://example.com/my-audio.mp3",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
## Filtering Voices
|
||||
|
||||
### By Language
|
||||
|
||||
```typescript
|
||||
function filterByLanguage(voices: Voice[], language: string): Voice[] {
|
||||
return voices.filter((v) =>
|
||||
v.language.toLowerCase().includes(language.toLowerCase())
|
||||
);
|
||||
}
|
||||
|
||||
const englishVoices = filterByLanguage(voices, "english");
|
||||
const spanishVoices = filterByLanguage(voices, "spanish");
|
||||
```
|
||||
|
||||
### By Gender
|
||||
|
||||
```typescript
|
||||
function filterByGender(voices: Voice[], gender: "male" | "female"): Voice[] {
|
||||
return voices.filter((v) => v.gender === gender);
|
||||
}
|
||||
|
||||
const femaleVoices = filterByGender(voices, "female");
|
||||
```
|
||||
|
||||
### By Features
|
||||
|
||||
```typescript
|
||||
function filterByFeatures(
|
||||
voices: Voice[],
|
||||
options: { supportPause?: boolean; emotionSupport?: boolean }
|
||||
): Voice[] {
|
||||
return voices.filter((v) => {
|
||||
if (options.supportPause !== undefined && v.support_pause !== options.supportPause) {
|
||||
return false;
|
||||
}
|
||||
if (options.emotionSupport !== undefined && v.emotion_support !== options.emotionSupport) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
const expressiveVoices = filterByFeatures(voices, { emotionSupport: true });
|
||||
```
|
||||
|
||||
## Voice Selection Helper
|
||||
|
||||
```typescript
|
||||
interface VoiceSelectionCriteria {
|
||||
language?: string;
|
||||
gender?: "male" | "female";
|
||||
supportPause?: boolean;
|
||||
emotionSupport?: boolean;
|
||||
}
|
||||
|
||||
async function findVoice(criteria: VoiceSelectionCriteria): Promise<Voice | null> {
|
||||
const voices = await listVoices();
|
||||
|
||||
const filtered = voices.filter((v) => {
|
||||
if (criteria.language && !v.language.toLowerCase().includes(criteria.language.toLowerCase())) {
|
||||
return false;
|
||||
}
|
||||
if (criteria.gender && v.gender !== criteria.gender) {
|
||||
return false;
|
||||
}
|
||||
if (criteria.supportPause !== undefined && v.support_pause !== criteria.supportPause) {
|
||||
return false;
|
||||
}
|
||||
if (criteria.emotionSupport !== undefined && v.emotion_support !== criteria.emotionSupport) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
|
||||
return filtered[0] || null;
|
||||
}
|
||||
|
||||
// Usage
|
||||
const voice = await findVoice({
|
||||
language: "english",
|
||||
gender: "female",
|
||||
emotionSupport: true,
|
||||
});
|
||||
```
|
||||
|
||||
## Multi-Language Videos
|
||||
|
||||
Create videos with different languages per scene:
|
||||
|
||||
```typescript
|
||||
const multiLanguageConfig = {
|
||||
video_inputs: [
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hello! Welcome to our global product launch.",
|
||||
voice_id: "english_voice_id",
|
||||
},
|
||||
},
|
||||
{
|
||||
character: {
|
||||
type: "avatar",
|
||||
avatar_id: "josh_lite3_20230714",
|
||||
avatar_style: "normal",
|
||||
},
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: "Hola! Bienvenidos al lanzamiento global de nuestro producto.",
|
||||
voice_id: "spanish_voice_id",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
## Matching Voice to Avatar
|
||||
|
||||
### Recommended: Use Avatar's Default Voice
|
||||
|
||||
Many avatars have a `default_voice_id` that's pre-matched. **This is the best approach.**
|
||||
|
||||
```typescript
|
||||
// Using v2 API to get avatar with default voice
|
||||
const response = await fetch(
|
||||
"https://api.heygen.com/v2/avatar_group.list?include_public=true",
|
||||
{ headers: { "X-Api-Key": process.env.HEYGEN_API_KEY! } }
|
||||
);
|
||||
const { data } = await response.json();
|
||||
|
||||
// Find avatar with a default voice
|
||||
const avatar = data.avatar_group_list.find((a: any) => a.default_voice_id);
|
||||
|
||||
if (avatar) {
|
||||
const videoConfig = {
|
||||
video_inputs: [{
|
||||
character: { type: "avatar", avatar_id: avatar.id },
|
||||
voice: {
|
||||
type: "text",
|
||||
input_text: script,
|
||||
voice_id: avatar.default_voice_id, // Pre-matched voice
|
||||
},
|
||||
}],
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
See [avatars.md](avatars.md) for complete examples.
|
||||
|
||||
### Fallback: Match Gender Manually
|
||||
|
||||
If avatar has no default voice, match genders manually:
|
||||
|
||||
```typescript
|
||||
interface AvatarVoicePair {
|
||||
avatarId: string;
|
||||
voiceId: string;
|
||||
gender: "male" | "female";
|
||||
}
|
||||
|
||||
async function findMatchingAvatarAndVoice(
|
||||
preferredGender?: "male" | "female"
|
||||
): Promise<AvatarVoicePair> {
|
||||
const [avatars, voices] = await Promise.all([
|
||||
listAvatars(),
|
||||
listVoices(),
|
||||
]);
|
||||
|
||||
// Default to male if no preference
|
||||
const gender = preferredGender || "male";
|
||||
|
||||
// Find avatar with matching gender
|
||||
const avatar = avatars.find((a) => a.gender === gender);
|
||||
if (!avatar) {
|
||||
throw new Error(`No ${gender} avatar available`);
|
||||
}
|
||||
|
||||
// Find voice with matching gender AND language
|
||||
const voice = voices.find(
|
||||
(v) => v.gender === gender && v.language.toLowerCase().includes("english")
|
||||
);
|
||||
if (!voice) {
|
||||
throw new Error(`No ${gender} English voice available`);
|
||||
}
|
||||
|
||||
return {
|
||||
avatarId: avatar.avatar_id,
|
||||
voiceId: voice.voice_id,
|
||||
gender,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Match voice gender to avatar** - Always pair male voices with male avatars, female with female
|
||||
2. **Match voice to content** - Use professional voices for business content
|
||||
3. **Test voice previews** - Listen to preview audio before selecting
|
||||
4. **Consider locale** - Match voice accent to target audience
|
||||
5. **Use natural pacing** - Adjust speed for clarity, typically 0.9-1.1x
|
||||
6. **Add pauses** - Use SSML breaks for more natural speech flow
|
||||
7. **Validate availability** - Always verify voice_id exists before using
|
||||
@@ -0,0 +1,393 @@
|
||||
---
|
||||
name: webhooks
|
||||
description: Registering webhook endpoints and event types for HeyGen
|
||||
---
|
||||
|
||||
# Webhooks
|
||||
|
||||
Webhooks allow HeyGen to notify your application when events occur, such as video completion. This is more efficient than polling for status updates.
|
||||
|
||||
## Overview
|
||||
|
||||
Instead of repeatedly checking video status, webhooks push notifications to your server when:
|
||||
- Video generation completes
|
||||
- Video generation fails
|
||||
- Translation completes
|
||||
- Avatar training completes
|
||||
- Other async operations finish
|
||||
|
||||
## Setting Up a Webhook Endpoint
|
||||
|
||||
Your webhook endpoint should:
|
||||
1. Accept POST requests
|
||||
2. Return 200 status quickly
|
||||
3. Handle events asynchronously
|
||||
|
||||
### Express.js Example
|
||||
|
||||
```typescript
|
||||
import express from "express";
|
||||
import crypto from "crypto";
|
||||
|
||||
const app = express();
|
||||
app.use(express.json());
|
||||
|
||||
// Webhook endpoint
|
||||
app.post("/webhook/heygen", async (req, res) => {
|
||||
// Acknowledge receipt immediately
|
||||
res.status(200).send("OK");
|
||||
|
||||
// Process event asynchronously
|
||||
processWebhookEvent(req.body).catch(console.error);
|
||||
});
|
||||
|
||||
async function processWebhookEvent(event: HeyGenWebhookEvent) {
|
||||
console.log(`Received event: ${event.event_type}`);
|
||||
|
||||
switch (event.event_type) {
|
||||
case "avatar_video.success":
|
||||
await handleVideoSuccess(event);
|
||||
break;
|
||||
case "avatar_video.fail":
|
||||
await handleVideoFailure(event);
|
||||
break;
|
||||
case "video_translate.success":
|
||||
await handleTranslationSuccess(event);
|
||||
break;
|
||||
default:
|
||||
console.log(`Unknown event type: ${event.event_type}`);
|
||||
}
|
||||
}
|
||||
|
||||
app.listen(3000, () => {
|
||||
console.log("Webhook server running on port 3000");
|
||||
});
|
||||
```
|
||||
|
||||
### Python Flask Example
|
||||
|
||||
```python
|
||||
from flask import Flask, request, jsonify
|
||||
import threading
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
@app.route("/webhook/heygen", methods=["POST"])
|
||||
def heygen_webhook():
|
||||
event = request.json
|
||||
|
||||
# Acknowledge immediately
|
||||
response = jsonify({"status": "received"})
|
||||
|
||||
# Process asynchronously
|
||||
thread = threading.Thread(
|
||||
target=process_webhook_event,
|
||||
args=(event,)
|
||||
)
|
||||
thread.start()
|
||||
|
||||
return response, 200
|
||||
|
||||
def process_webhook_event(event):
|
||||
event_type = event.get("event_type")
|
||||
print(f"Received event: {event_type}")
|
||||
|
||||
if event_type == "avatar_video.success":
|
||||
handle_video_success(event)
|
||||
elif event_type == "avatar_video.fail":
|
||||
handle_video_failure(event)
|
||||
elif event_type == "video_translate.success":
|
||||
handle_translation_success(event)
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run(port=3000)
|
||||
```
|
||||
|
||||
## Webhook Event Types
|
||||
|
||||
| Event Type | Description |
|
||||
|------------|-------------|
|
||||
| `avatar_video.success` | Video generation completed |
|
||||
| `avatar_video.fail` | Video generation failed |
|
||||
| `video_translate.success` | Translation completed |
|
||||
| `video_translate.fail` | Translation failed |
|
||||
| `instant_avatar.success` | Instant avatar created |
|
||||
| `instant_avatar.fail` | Instant avatar creation failed |
|
||||
|
||||
## Event Payload Structure
|
||||
|
||||
### Video Success Event
|
||||
|
||||
```typescript
|
||||
interface VideoSuccessEvent {
|
||||
event_type: "avatar_video.success";
|
||||
event_data: {
|
||||
video_id: string;
|
||||
video_url: string;
|
||||
thumbnail_url: string;
|
||||
duration: number;
|
||||
callback_id?: string;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event_type": "avatar_video.success",
|
||||
"event_data": {
|
||||
"video_id": "abc123",
|
||||
"video_url": "https://files.heygen.ai/video/abc123.mp4",
|
||||
"thumbnail_url": "https://files.heygen.ai/thumbnail/abc123.jpg",
|
||||
"duration": 45.2,
|
||||
"callback_id": "your_custom_id"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Video Failure Event
|
||||
|
||||
```typescript
|
||||
interface VideoFailureEvent {
|
||||
event_type: "avatar_video.fail";
|
||||
event_data: {
|
||||
video_id: string;
|
||||
error: string;
|
||||
callback_id?: string;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"event_type": "avatar_video.fail",
|
||||
"event_data": {
|
||||
"video_id": "abc123",
|
||||
"error": "Script too long for selected avatar",
|
||||
"callback_id": "your_custom_id"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Registering a Webhook URL
|
||||
|
||||
Configure your webhook URL through the HeyGen dashboard or API:
|
||||
|
||||
### Request Fields
|
||||
|
||||
| Field | Type | Req | Description |
|
||||
|-------|------|:---:|-------------|
|
||||
| `url` | string | ✓ | Your webhook endpoint URL |
|
||||
| `events` | array | ✓ | Event types to subscribe to |
|
||||
| `secret` | string | | Shared secret for signature verification |
|
||||
|
||||
### Via API
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v1/webhook/endpoint.add" \
|
||||
-H "X-Api-Key: $HEYGEN_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"url": "https://your-domain.com/webhook/heygen",
|
||||
"events": ["avatar_video.success", "avatar_video.fail"]
|
||||
}'
|
||||
```
|
||||
|
||||
### TypeScript
|
||||
|
||||
```typescript
|
||||
interface WebhookConfig {
|
||||
url: string; // Required
|
||||
events: string[]; // Required
|
||||
secret?: string;
|
||||
}
|
||||
|
||||
async function registerWebhook(config: WebhookConfig): Promise<void> {
|
||||
const response = await fetch("https://api.heygen.com/v1/webhook/endpoint.add", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"X-Api-Key": process.env.HEYGEN_API_KEY!,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(config),
|
||||
});
|
||||
|
||||
const json = await response.json();
|
||||
|
||||
if (json.error) {
|
||||
throw new Error(json.error);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Using Callback IDs
|
||||
|
||||
Track which video triggered a webhook with callback IDs:
|
||||
|
||||
### Include Callback ID in Video Generation
|
||||
|
||||
```typescript
|
||||
const videoConfig = {
|
||||
video_inputs: [...],
|
||||
callback_id: "order_12345", // Your custom identifier
|
||||
};
|
||||
```
|
||||
|
||||
### Handle in Webhook
|
||||
|
||||
```typescript
|
||||
async function handleVideoSuccess(event: VideoSuccessEvent) {
|
||||
const { video_id, video_url, callback_id } = event.event_data;
|
||||
|
||||
if (callback_id) {
|
||||
// Look up your original request
|
||||
const order = await getOrderByCallbackId(callback_id);
|
||||
await updateOrderWithVideo(order.id, video_url);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Webhook Security
|
||||
|
||||
### Verify Webhook Signatures
|
||||
|
||||
If HeyGen provides signature verification:
|
||||
|
||||
```typescript
|
||||
import crypto from "crypto";
|
||||
|
||||
function verifyWebhookSignature(
|
||||
payload: string,
|
||||
signature: string,
|
||||
secret: string
|
||||
): boolean {
|
||||
const expectedSignature = crypto
|
||||
.createHmac("sha256", secret)
|
||||
.update(payload)
|
||||
.digest("hex");
|
||||
|
||||
return crypto.timingSafeEqual(
|
||||
Buffer.from(signature),
|
||||
Buffer.from(expectedSignature)
|
||||
);
|
||||
}
|
||||
|
||||
// In your webhook handler
|
||||
app.post("/webhook/heygen", (req, res) => {
|
||||
const signature = req.headers["x-heygen-signature"] as string;
|
||||
const payload = JSON.stringify(req.body);
|
||||
|
||||
if (!verifyWebhookSignature(payload, signature, WEBHOOK_SECRET)) {
|
||||
return res.status(401).send("Invalid signature");
|
||||
}
|
||||
|
||||
// Process event...
|
||||
});
|
||||
```
|
||||
|
||||
### Validate Event Origin
|
||||
|
||||
```typescript
|
||||
function isValidHeygenEvent(event: any): boolean {
|
||||
// Check required fields
|
||||
if (!event.event_type || !event.event_data) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Check event type is known
|
||||
const validEventTypes = [
|
||||
"avatar_video.success",
|
||||
"avatar_video.fail",
|
||||
"video_translate.success",
|
||||
"video_translate.fail",
|
||||
];
|
||||
|
||||
return validEventTypes.includes(event.event_type);
|
||||
}
|
||||
```
|
||||
|
||||
## Handling Webhook Failures
|
||||
|
||||
Implement retry logic and error handling:
|
||||
|
||||
```typescript
|
||||
async function processWebhookEvent(event: HeyGenWebhookEvent) {
|
||||
const maxRetries = 3;
|
||||
|
||||
for (let attempt = 1; attempt <= maxRetries; attempt++) {
|
||||
try {
|
||||
await handleEvent(event);
|
||||
return;
|
||||
} catch (error) {
|
||||
console.error(`Attempt ${attempt} failed:`, error);
|
||||
|
||||
if (attempt < maxRetries) {
|
||||
// Exponential backoff
|
||||
await new Promise((r) => setTimeout(r, Math.pow(2, attempt) * 1000));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Store failed event for manual review
|
||||
await storeFailedEvent(event);
|
||||
}
|
||||
```
|
||||
|
||||
## Webhook vs Polling Comparison
|
||||
|
||||
| Aspect | Webhook | Polling |
|
||||
|--------|---------|---------|
|
||||
| Latency | Immediate | Depends on interval |
|
||||
| Efficiency | High (push) | Low (repeated requests) |
|
||||
| Complexity | Requires endpoint | Simpler to implement |
|
||||
| Reliability | Needs retry handling | Guaranteed delivery |
|
||||
| Cost | Lower API usage | Higher API usage |
|
||||
|
||||
## Testing Webhooks
|
||||
|
||||
### Local Development with ngrok
|
||||
|
||||
```bash
|
||||
# Start ngrok tunnel
|
||||
ngrok http 3000
|
||||
|
||||
# Use ngrok URL as webhook endpoint
|
||||
# https://abc123.ngrok.io/webhook/heygen
|
||||
```
|
||||
|
||||
### Webhook Testing Tool
|
||||
|
||||
```typescript
|
||||
// Test webhook locally
|
||||
async function simulateWebhook(event: HeyGenWebhookEvent) {
|
||||
const response = await fetch("http://localhost:3000/webhook/heygen", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(event),
|
||||
});
|
||||
|
||||
console.log(`Response: ${response.status}`);
|
||||
}
|
||||
|
||||
// Simulate success event
|
||||
await simulateWebhook({
|
||||
event_type: "avatar_video.success",
|
||||
event_data: {
|
||||
video_id: "test_123",
|
||||
video_url: "https://example.com/test.mp4",
|
||||
thumbnail_url: "https://example.com/test.jpg",
|
||||
duration: 30,
|
||||
callback_id: "test_callback",
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Respond quickly** - Return 200 within 5 seconds, process async
|
||||
2. **Handle duplicates** - Same event may be sent multiple times
|
||||
3. **Implement retries** - Handle temporary processing failures
|
||||
4. **Log everything** - Store webhook payloads for debugging
|
||||
5. **Use callback IDs** - Track requests through the system
|
||||
6. **Secure endpoints** - Verify signatures, use HTTPS
|
||||
7. **Monitor health** - Track webhook success rates
|
||||
8. **Queue processing** - Use job queues for heavy processing
|
||||
Reference in New Issue
Block a user