Initial release — OpenMontage: the first open-source agentic video production system

11 production pipelines, 47 tools, 124 agent skills.
Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and
free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
calesthio
2026-03-29 08:25:17 -07:00
commit a3e735cc7a
1147 changed files with 240221 additions and 0 deletions
+302
View File
@@ -0,0 +1,302 @@
---
name: agents
description: Build voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversation experience.
license: MIT
compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
---
# ElevenLabs Agents Platform
Build voice AI agents with natural conversations, multiple LLM providers, custom tools, and easy web embedding.
> **Setup:** See [Installation Guide](references/installation.md) for CLI and SDK setup.
## Quick Start with CLI
The ElevenLabs CLI is the recommended way to create and manage agents:
```bash
# Install CLI and authenticate
npm install -g @elevenlabs/cli
elevenlabs auth login
# Initialize project and create an agent
elevenlabs agents init
elevenlabs agents add "My Assistant" --template complete
# Push to ElevenLabs platform
elevenlabs agents push
```
**Available templates:** `complete`, `minimal`, `voice-only`, `text-only`, `customer-service`, `assistant`
### Python
```python
from elevenlabs import ElevenLabs
client = ElevenLabs()
agent = client.conversational_ai.agents.create(
name="My Assistant",
enable_versioning=True,
conversation_config={
"agent": {
"first_message": "Hello! How can I help?",
"language": "en",
"prompt": {
"prompt": "You are a helpful assistant. Be concise and friendly.",
"llm": "gemini-2.0-flash",
"temperature": 0.7
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
}
)
```
### JavaScript
```javascript
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
const client = new ElevenLabsClient();
const agent = await client.conversationalAi.agents.create({
name: "My Assistant",
enableVersioning: true,
conversationConfig: {
agent: {
firstMessage: "Hello! How can I help?",
language: "en",
prompt: {
prompt: "You are a helpful assistant.",
llm: "gemini-2.0-flash",
temperature: 0.7
}
},
tts: { voiceId: "JBFqnCBsd6RMkjVDRZzb" }
}
});
```
### cURL
```bash
curl -X POST "https://api.elevenlabs.io/v1/convai/agents/create?enable_versioning=true" \
-H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
-d '{"name": "My Assistant", "conversation_config": {"agent": {"first_message": "Hello!", "language": "en", "prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}'
```
## Starting Conversations
**Server-side (Python):** Get signed URL for client connection:
```python
signed_url = client.conversational_ai.conversations.get_signed_url(
agent_id="your-agent-id",
environment="staging",
)
```
**Client-side (JavaScript):**
```javascript
import { Conversation } from "@elevenlabs/client";
const conversation = await Conversation.startSession({
agentId: "your-agent-id",
environment: "staging",
onMessage: (msg) => console.log("Agent:", msg.message),
onUserTranscript: (t) => console.log("User:", t.message),
onError: (e) => console.error(e)
});
```
**React Hook:**
```typescript
import { useConversation } from "@elevenlabs/react";
const conversation = useConversation({ onMessage: (msg) => console.log(msg) });
// Get a signed URL for the target environment from your backend, then:
await conversation.startSession({ signedUrl: token });
```
## Configuration
| Provider | Models |
|----------|--------|
| OpenAI | `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` |
| Anthropic | `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` |
| Google | `gemini-3.1-flash-lite-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` |
| ElevenLabs | `glm-45-air-fp8`, `qwen3-30b-a3b`, `gpt-oss-120b` |
| Custom | `custom-llm` (bring your own endpoint) |
Use `GET /v1/convai/llm/list` to inspect the current model catalog, including deprecation state, token/context limits, and capability flags such as image-input support.
**Popular voices:** `JBFqnCBsd6RMkjVDRZzb` (George), `EXAVITQu4vr4xnSDxMaL` (Sarah), `onwK4e9ZLuTAKqWW03F9` (Daniel), `XB0fDUnXU5powFXDhCwa` (Charlotte)
**Turn eagerness:** `patient` (waits longer for user to finish), `normal`, or `eager` (responds quickly)
See [Agent Configuration](references/agent-configuration.md) for all options.
## Tools
Extend agents with webhook, client, or built-in system tools. Tools are defined inside `conversation_config.agent.prompt`:
Workspace environment variables can resolve per-environment server tool URLs, headers, and auth connections, and runtime system variables such as `{{system__conversation_history}}` can pass full conversation context into tool calls when needed.
```python
"prompt": {
"prompt": "You are a helpful assistant that can check the weather.",
"llm": "gemini-2.0-flash",
"tools": [
# Webhook: server-side API call
{"type": "webhook", "name": "get_weather", "description": "Get weather",
"api_schema": {"url": "https://api.example.com/weather", "method": "POST",
"request_body_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}},
# Client: runs in the browser
{"type": "client", "name": "show_product", "description": "Display a product",
"parameters": {"type": "object", "properties": {"productId": {"type": "string"}}, "required": ["productId"]}}
],
"built_in_tools": {
"end_call": {},
"transfer_to_number": {"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]}
}
}
```
**Client tools** run in browser:
```javascript
clientTools: {
show_product: async ({ productId }) => {
document.getElementById("product").src = `/products/${productId}`;
return { success: true };
}
}
```
See [Client Tools Reference](references/client-tools.md) for complete documentation.
## Widget Embedding
```html
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
<script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>
```
Customize with attributes: `avatar-image-url`, `action-text`, `start-call-text`, `end-call-text`.
See [Widget Embedding Reference](references/widget-embedding.md) for all options.
## Outbound Calls
Make outbound phone calls using your agent via Twilio integration:
### Python
```python
response = client.conversational_ai.twilio.outbound_call(
agent_id="your-agent-id",
agent_phone_number_id="your-phone-number-id",
to_number="+1234567890",
call_recording_enabled=True
)
print(f"Call initiated: {response.conversation_id}")
```
### JavaScript
```javascript
const response = await client.conversationalAi.twilio.outboundCall({
agentId: "your-agent-id",
agentPhoneNumberId: "your-phone-number-id",
toNumber: "+1234567890",
callRecordingEnabled: true,
});
```
### cURL
```bash
curl -X POST "https://api.elevenlabs.io/v1/convai/twilio/outbound-call" \
-H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
-d '{"agent_id": "your-agent-id", "agent_phone_number_id": "your-phone-number-id", "to_number": "+1234567890", "call_recording_enabled": true}'
```
See [Outbound Calls Reference](references/outbound-calls.md) for configuration overrides and dynamic variables.
## Managing Agents
### Using CLI (Recommended)
```bash
# List agents and check status
elevenlabs agents list
elevenlabs agents status
# Import agents from platform to local config
elevenlabs agents pull # Import all agents
elevenlabs agents pull --agent <agent-id> # Import specific agent
# Push local changes to platform
elevenlabs agents push # Upload configurations
elevenlabs agents push --dry-run # Preview changes first
# Add tools
elevenlabs tools add-webhook "Weather API"
elevenlabs tools add-client "UI Tool"
```
### Project Structure
The CLI creates a project structure for managing agents:
```
your_project/
├── agents.json # Agent definitions
├── tools.json # Tool configurations
├── tests.json # Test configurations
├── agent_configs/ # Individual agent configs
├── tool_configs/ # Individual tool configs
└── test_configs/ # Individual test configs
```
### SDK Examples
```python
# List
agents = client.conversational_ai.agents.list()
# Get
agent = client.conversational_ai.agents.get(agent_id="your-agent-id")
# Update (partial - only include fields to change)
client.conversational_ai.agents.update(agent_id="your-agent-id", name="New Name")
client.conversational_ai.agents.update(agent_id="your-agent-id",
conversation_config={
"agent": {"prompt": {"prompt": "New instructions", "llm": "claude-sonnet-4"}}
})
# Delete
client.conversational_ai.agents.delete(agent_id="your-agent-id")
```
See [Agent Configuration](references/agent-configuration.md) for all configuration options and SDK examples.
## Error Handling
```python
try:
agent = client.conversational_ai.agents.create(...)
except Exception as e:
print(f"API error: {e}")
```
Common errors: **401** (invalid key), **404** (not found), **422** (invalid config), **429** (rate limit)
## References
- [Installation Guide](references/installation.md) - SDK setup and migration
- [Agent Configuration](references/agent-configuration.md) - All config options and CRUD examples
- [Client Tools](references/client-tools.md) - Webhook, client, and system tools
- [Widget Embedding](references/widget-embedding.md) - Website integration
- [Outbound Calls](references/outbound-calls.md) - Twilio phone call integration
@@ -0,0 +1,606 @@
# Agent Configuration
Complete reference for configuring conversational AI agents.
## Configuration Structure
```python
agent = client.conversational_ai.agents.create(
name="My Agent",
conversation_config={
"agent": {
"first_message": "Hello!",
"language": "en",
"prompt": { # LLM, system prompt, tools, and knowledge base
"prompt": "You are helpful.",
"llm": "gemini-2.0-flash",
"tools": [...],
"built_in_tools": {...}
}
},
"tts": {...}, # Voice and TTS model settings
"asr": {...}, # Speech recognition settings
"turn": {...}, # Turn-taking behavior
"conversation": {...}, # Duration, events, monitoring
"vad": {...}, # Voice activity detection config
"language_presets": {...} # Language-specific overrides
},
platform_settings={...} # Auth, call limits
)
```
## conversation_config
Controls the real-time conversation behavior.
### agent
```python
conversation_config={
"agent": {
"first_message": "Hello! How can I help you today?",
"language": "en",
"disable_first_message_interruptions": False,
"prompt": {
"prompt": "You are a helpful assistant.",
"llm": "gemini-2.0-flash",
"temperature": 0.7
}
}
}
```
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `first_message` | string | `""` | What the agent says when conversation starts |
| `language` | string | `"en"` | ISO 639-1 language code (en, es, fr, etc.) |
| `disable_first_message_interruptions` | bool | `false` | Prevent user from interrupting the first message |
| `hinglish_mode` | bool | `false` | When enabled and language is Hindi, agent responds in Hinglish |
| `dynamic_variables` | object | - | Config with `dynamic_variable_placeholders` containing key-value pairs |
| `prompt` | object | - | LLM configuration (see prompt section below) |
### tts (Text-to-Speech)
```python
conversation_config={
"tts": {
"voice_id": "JBFqnCBsd6RMkjVDRZzb",
"model_id": "eleven_flash_v2_5",
"stability": 0.5,
"similarity_boost": 0.8,
"speed": 1.0,
"optimize_streaming_latency": 3,
"expressive_mode": True
}
}
```
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `voice_id` | string | `"cjVigY5qzO86Huf0OWal"` | Voice to use |
| `model_id` | string | - | TTS model (see below) |
| `stability` | float | `0.5` | 0-1, lower = more expressive |
| `similarity_boost` | float | `0.8` | 0-1, higher = closer to original voice |
| `speed` | float | `1.0` | 0.7-1.2, speech speed multiplier |
| `optimize_streaming_latency` | int | - | 0-4, higher = faster but lower quality |
| `expressive_mode` | bool | `true` | Enable expressive voice generation |
| `agent_output_audio_format` | string | - | Output audio codec format |
| `pronunciation_dictionary_locators` | array | - | Pronunciation overrides |
**Available TTS models for agents:**
| Model ID | Languages | Latency |
|----------|-----------|---------|
| `eleven_flash_v2_5` | 32 | ~75ms (recommended) |
| `eleven_flash_v2` | English | ~75ms |
| `eleven_turbo_v2_5` | 32 | ~250-300ms |
| `eleven_turbo_v2` | English | ~250-300ms |
| `eleven_multilingual_v2` | 29 | Standard |
| `eleven_v3_conversational` | 70+ | Standard |
### asr (Automatic Speech Recognition)
```python
conversation_config={
"asr": {
"quality": "high",
"keywords": ["ElevenLabs", "TechCorp"],
"user_input_audio_format": "pcm_16000"
}
}
```
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `quality` | string | `"high"` | Transcription quality level |
| `provider` | string | `"elevenlabs"` | ASR provider (`elevenlabs` or `scribe_realtime`) |
| `keywords` | array | - | Words to boost recognition accuracy |
| `user_input_audio_format` | string | - | Input audio format (e.g., `pcm_16000`, `ulaw_8000`) |
### turn (Turn-Taking)
```python
conversation_config={
"turn": {
"turn_timeout": 7,
"turn_eagerness": "normal",
"silence_end_call_timeout": -1
}
}
```
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `turn_timeout` | number | `7` | Seconds to wait before re-engaging the user |
| `turn_eagerness` | string | `"normal"` | How quickly agent responds: `patient`, `normal`, or `eager` |
| `silence_end_call_timeout` | number | `-1` | Seconds of silence before ending call (-1 = disabled) |
| `initial_wait_time` | number | - | Seconds to wait for user to start speaking |
| `spelling_patience` | string | `"auto"` | Entity detection patience: `auto` or `off` |
| `speculative_turn` | bool | `false` | Enable speculative turn detection |
| `soft_timeout_config` | object | - | Configures a message if user is silent (see below) |
**soft_timeout_config:**
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `timeout_seconds` | number | `-1` | Seconds before soft timeout (-1 = disabled) |
| `message` | string | `"Hhmmmm...yeah."` | What agent says on timeout |
| `use_llm_generated_message` | bool | `false` | Let LLM generate the timeout message |
## prompt (nested in conversation_config.agent)
Configures the LLM behavior. This object lives at `conversation_config.agent.prompt`:
```python
conversation_config={
"agent": {
"prompt": {
"prompt": "You are a helpful customer service agent...",
"llm": "gemini-2.0-flash",
"temperature": 0.7,
"max_tokens": 500,
"tools": [...],
"built_in_tools": {...},
"knowledge_base": [...]
}
}
}
```
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `prompt` | string | `""` | System prompt defining agent behavior |
| `llm` | string | - | Model ID (see LLM providers below) |
| `temperature` | float | `0` | 0-1, higher = more creative |
| `max_tokens` | int | `-1` | Max tokens for LLM response (-1 = unlimited) |
| `reasoning_effort` | string | - | Reasoning depth: `none`, `minimal`, `low`, `medium`, `high` (model-dependent) |
| `thinking_budget` | int | - | Max thinking tokens for reasoning models |
| `tools` | array | - | Webhook and client tool definitions |
| `built_in_tools` | object | - | System tools (end_call, transfer, etc.) |
| `tool_ids` | array | - | References to pre-configured tools |
| `knowledge_base` | array | - | Documents for RAG |
| `custom_llm` | object | - | Custom LLM endpoint config |
| `timezone` | string | - | IANA timezone (e.g., `America/New_York`) |
| `backup_llm_config` | object | - | Fallback LLM configuration |
| `cascade_timeout_seconds` | number | `8` | Seconds before cascading to backup LLM (2-15) |
| `mcp_server_ids` | array | - | MCP server IDs to connect |
| `native_mcp_server_ids` | array | - | Native MCP server IDs |
| `ignore_default_personality` | bool | - | Skip default personality instructions |
Workspace environment variables let one agent configuration span multiple deployments. Use
`{{system_env__label}}` in server tool and MCP server URLs, `{ "env_var_label": "orders_api_key" }`
for secret-backed tool headers, and `{ "env_var_label": "orders_oauth" }` in `auth_connection`
to resolve per-environment auth connections at runtime.
### LLM Providers
| Provider | Model IDs |
|----------|-----------|
| OpenAI | `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` |
| Anthropic | `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` |
| Google | `gemini-3.1-flash-lite-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` |
| ElevenLabs | `glm-45-air-fp8`, `qwen3-30b-a3b`, `gpt-oss-120b` (hosted, ultra-low latency) |
| Custom | `custom-llm` (requires custom_llm config) |
Use `GET /v1/convai/llm/list` to inspect the current model catalog, including deprecation state, token/context limits, and capability flags such as image-input support.
### Custom LLM
The `custom_llm` field is nested inside `conversation_config.agent.prompt`:
```python
conversation_config={
"agent": {
"prompt": {
"prompt": "You are helpful.",
"llm": "custom-llm",
"custom_llm": {
"url": "https://your-llm-endpoint.com/v1/chat/completions",
"model_id": "your-model-id",
"api_key": {"secret_id": "your-secret-id"},
"api_type": "chat_completions" # or "responses"
}
}
}
}
```
## platform_settings
Platform-level configuration for security, limits, summaries, and widget behavior.
```python
platform_settings={
"summary_language": "en",
"widget": {
"show_agent_status": True,
"show_conversation_id": True
},
"auth": {
"enable_auth": True,
"allowlist": [{"hostname": "example.com"}]
},
"call_limits": {
"agent_concurrency_limit": 10,
"daily_limit": 100
}
}
```
### Top-Level Fields
| Field | Type | Description |
|-------|------|-------------|
| `summary_language` | string | Language for conversation analysis outputs such as summaries, titles, evaluation rationales, and data collection rationales. If omitted, ElevenLabs infers it from the conversation. |
| `widget` | object | Hosted widget and shareable page configuration. See the widget table below for selected options. |
| `auth` | object | Authentication and origin restrictions for agent access |
| `call_limits` | object | Concurrency and daily usage limits |
| `guardrails` | object | Built-in safety and policy controls for agent interactions |
| `privacy` | object | Recording, retention, and conversation history redaction settings |
### auth
| Field | Type | Description |
|-------|------|-------------|
| `enable_auth` | bool | Require signed URLs/tokens for connections |
| `allowlist` | array | Allowed origins for CORS |
| `shareable_token` | string | Public conversation token |
### call_limits
| Field | Type | Description |
|-------|------|-------------|
| `agent_concurrency_limit` | int | Max simultaneous conversations (default: -1, unlimited) |
| `daily_limit` | int | Max conversations per day (default: 100000) |
| `bursting_enabled` | bool | Allow exceeding limits at 2x cost (default: true) |
### guardrails
Use `platform_settings.guardrails` to configure built-in safety controls for user input and agent behavior. The fields below cover the current schema additions that are most relevant in agent configs.
| Field | Type | Description |
|-------|------|-------------|
| `version` | string | Guardrail config version. Use `"1"` for the current schema. |
| `focus` | object | Keeps the agent on-topic and aligned with the configured task. |
| `prompt_injection` | object | Detects prompt injection and instruction override attempts. |
| `custom` | object | Configures user-defined response validation guardrails. |
| `content` | object | Configures category-specific content moderation guardrails. |
**focus / prompt_injection:**
| Field | Type | Description |
|-------|------|-------------|
| `is_enabled` | bool | Enables the guardrail. |
**content:**
| Field | Type | Description |
|-------|------|-------------|
| `execution_mode` | string | Guardrail execution mode: `streaming` or `blocking`. |
| `config` | object | Category threshold settings for content moderation. |
**content.config:**
| Field | Type | Description |
|-------|------|-------------|
| `sexual` | object | Threshold settings for sexual content. |
| `violence` | object | Threshold settings for violent content. |
| `harassment` | object | Threshold settings for harassment. |
| `self_harm` | object | Threshold settings for self-harm content. |
| `profanity` | object | Threshold settings for profanity. |
| `religion_or_politics` | object | Threshold settings for religion or politics content. |
| `medical_and_legal_information` | object | Threshold settings for medical or legal information. |
**content.config.\<category\>:**
| Field | Type | Description |
|-------|------|-------------|
| `is_enabled` | bool | Enables moderation for the category. |
| `threshold` | number or string | Category threshold as a numeric score or one of `low`, `medium`, or `high`. |
Blocking content guardrails and custom guardrails support a `trigger_action` that either ends
the session immediately or retries the response. Retry removes the blocked reply, injects your
feedback as a system message, and re-generates up to 3 times before the platform falls back to
ending the session. Feedback templates can use `{{trigger_reason}}` and `{{agent_message}}`.
### privacy
Use `platform_settings.privacy` to control recording, retention, and redaction behavior. The redaction-specific field is:
| Field | Type | Description |
|-------|------|-------------|
| `conversation_history_redaction` | object | Redacts configured entity types from stored transcripts, audio, and analysis. |
**conversation_history_redaction:**
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `enabled` | bool | `false` | Whether conversation history redaction is enabled |
| `entities` | array | - | Entity types to redact. Use parent types such as `name` or specific values such as `name.name_given`, `email_address`, `contact_number`, `dob`, and `age`. |
### widget
Use `platform_settings.widget` to configure the hosted widget and shareable page defaults. For client-side embed attributes, see the widget embedding reference.
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `dismissible` | bool | `false` | Whether the widget can be dismissed by the user |
| `show_agent_status` | bool | `false` | Whether to show working, done, or error status while tools are running |
| `show_conversation_id` | bool | `true` | Whether to show the conversation ID after disconnection |
| `strip_audio_tags` | bool | `true` | Whether to strip audio markup from messages |
| `syntax_highlight_theme` | string | auto | Code block syntax highlighting theme (`light` or `dark`); omit it to let the widget auto-detect |
### conversation (inside conversation_config)
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `max_duration_seconds` | int | `600` | Max conversation duration |
| `text_only` | bool | `false` | Text-only mode (avoids audio pricing) |
| `monitoring_enabled` | bool | `false` | Enable real-time WebSocket monitoring |
## Additional Top-Level Fields
| Field | Type | Description |
|-------|------|-------------|
| `tags` | array | Classification labels for filtering (e.g., `["production"]`, `["test"]`) |
| `workflow` | object | Conversation flow definition and tool interaction sequences |
## Knowledge Base / RAG
Knowledge base is configured inside `conversation_config.agent.prompt`:
```python
agent = client.conversational_ai.agents.create(
name="Support Agent",
conversation_config={
"agent": {
"prompt": {
"prompt": "You are a support agent. Use the knowledge base to answer questions.",
"llm": "gemini-2.0-flash",
"knowledge_base": [
{"type": "file", "id": "doc-id", "name": "Product Guide", "usage_mode": "auto"}
],
"rag": {
"enabled": True,
"embedding_model": "qwen3_embedding_4b",
"max_documents_length": 50000,
"max_retrieved_rag_chunks_count": 20
}
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
}
)
```
`rag.embedding_model` supports `e5_mistral_7b_instruct`, `multilingual_e5_large_instruct`, and `qwen3_embedding_4b`.
## CRUD Operations
### Using CLI (Recommended)
```bash
# Initialize project
elevenlabs agents init
# Create agent from template
elevenlabs agents add "My Agent" --template complete
elevenlabs agents add "Support Bot" --template customer-service
# List agents
elevenlabs agents list
# Check status
elevenlabs agents status
# Push local changes to platform
elevenlabs agents push
elevenlabs agents push --dry-run # Preview changes first
# Import agents from platform
elevenlabs agents pull # Import all
elevenlabs agents pull --agent <agent-id> # Import specific agent
elevenlabs agents pull --update # Override local configs
# View available templates
elevenlabs agents templates list
elevenlabs agents templates show <template-name>
# Add tools
elevenlabs tools add-webhook "API Tool"
elevenlabs tools add-client "UI Tool"
# Generate widget code
elevenlabs agents widget <agent-id>
```
### SDK: List Agents
```python
agents = client.conversational_ai.agents.list()
for agent in agents.agents:
print(f"{agent.name}: {agent.agent_id}")
```
```javascript
const agents = await client.conversationalAi.agents.list();
```
```bash
curl -X GET "https://api.elevenlabs.io/v1/convai/agents" -H "xi-api-key: $ELEVENLABS_API_KEY"
```
### SDK: Get Agent
```python
agent = client.conversational_ai.agents.get(agent_id="your-agent-id")
```
```javascript
const agent = await client.conversationalAi.agents.get("your-agent-id");
```
```bash
curl -X GET "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" -H "xi-api-key: $ELEVENLABS_API_KEY"
```
### SDK: Update Agent
Only include fields you want to change. All other settings remain unchanged.
**Python:**
```python
# Update name
client.conversational_ai.agents.update(agent_id="id", name="New Name")
# Update TTS voice
client.conversational_ai.agents.update(agent_id="id", conversation_config={
"tts": {"voice_id": "EXAVITQu4vr4xnSDxMaL", "model_id": "eleven_flash_v2_5"}
})
# Update prompt/LLM (nested in agent)
client.conversational_ai.agents.update(agent_id="id", conversation_config={
"agent": {"prompt": {"prompt": "New instructions.", "llm": "claude-sonnet-4", "temperature": 0.8}}
})
# Update first message
client.conversational_ai.agents.update(agent_id="id", conversation_config={
"agent": {"first_message": "Welcome back!"}
})
# Update platform settings
client.conversational_ai.agents.update(agent_id="id", platform_settings={
"auth": {"enable_auth": True, "allowlist": [{"hostname": "myapp.com"}]}
})
```
**JavaScript:**
```javascript
await client.conversationalAi.agents.update("id", { name: "New Name" });
await client.conversationalAi.agents.update("id", {
conversationConfig: { tts: { voiceId: "EXAVITQu4vr4xnSDxMaL" } }
});
await client.conversationalAi.agents.update("id", {
conversationConfig: { agent: { prompt: { prompt: "New instructions.", llm: "claude-sonnet-4" } } }
});
```
**cURL:**
```bash
curl -X PATCH "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" \
-H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
-d '{"name": "New Name"}'
```
#### Updatable Fields
| Section | Fields |
|---------|--------|
| Root | `name`, `tags` |
| `conversation_config.agent` | `first_message`, `language`, `disable_first_message_interruptions`, `dynamic_variables` |
| `conversation_config.agent.prompt` | `prompt`, `llm`, `temperature`, `max_tokens`, `reasoning_effort`, `tools`, `built_in_tools`, `knowledge_base`, `custom_llm`, `timezone` |
| `conversation_config.tts` | `voice_id`, `model_id`, `stability`, `similarity_boost`, `speed`, `optimize_streaming_latency`, `expressive_mode` |
| `conversation_config.asr` | `quality`, `provider`, `keywords`, `user_input_audio_format` |
| `conversation_config.turn` | `turn_timeout`, `turn_eagerness`, `silence_end_call_timeout`, `soft_timeout_config` |
| `conversation_config.conversation` | `max_duration_seconds`, `text_only`, `monitoring_enabled` |
| `platform_settings` | `summary_language`, `guardrails`, `privacy` |
| `platform_settings.widget` | `dismissible`, `show_agent_status`, `show_conversation_id`, `strip_audio_tags`, `syntax_highlight_theme` |
| `platform_settings.auth` | `enable_auth`, `allowlist` |
| `platform_settings.call_limits` | `agent_concurrency_limit`, `daily_limit`, `bursting_enabled` |
### SDK: Delete Agent
```python
client.conversational_ai.agents.delete(agent_id="your-agent-id")
```
```javascript
await client.conversationalAi.agents.delete("your-agent-id");
```
```bash
curl -X DELETE "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" -H "xi-api-key: $ELEVENLABS_API_KEY"
```
## CI/CD Integration
Use the CLI in your deployment pipeline:
```bash
# Set API key as environment variable
export ELEVENLABS_API_KEY="your-api-key"
# Push changes (non-interactive)
elevenlabs agents push
```
## Example Configurations
### Customer Support Agent
```python
agent = client.conversational_ai.agents.create(
name="Support Agent",
conversation_config={
"agent": {
"first_message": "Hi! Thanks for calling TechCorp support.",
"language": "en",
"prompt": {
"prompt": "You are a customer support agent. Be helpful, professional, concise.",
"llm": "gemini-2.0-flash",
"temperature": 0.5,
"built_in_tools": {
"end_call": {},
"transfer_to_number": {
"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]
}
}
}
},
"tts": {"voice_id": "XB0fDUnXU5powFXDhCwa", "model_id": "eleven_flash_v2_5"},
"turn": {"turn_eagerness": "normal", "turn_timeout": 7},
"conversation": {"max_duration_seconds": 900}
}
)
```
### Low-Latency Assistant
```python
agent = client.conversational_ai.agents.create(
name="Quick Assistant",
conversation_config={
"agent": {
"first_message": "Hey! What do you need?",
"prompt": {
"prompt": "Fast, efficient assistant. Brief answers.",
"llm": "gemini-2.0-flash",
"temperature": 0.3,
"max_tokens": 100
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_flash_v2_5", "optimize_streaming_latency": 4},
"turn": {"turn_eagerness": "eager", "turn_timeout": 3}
}
)
```
@@ -0,0 +1,545 @@
# Client Tools
Extend your agent with custom capabilities. Tools let the agent take actions beyond just talking.
## Tool Types
| Type | Execution | Use Case |
|------|-----------|----------|
| **Webhook** | Server-side via HTTP | Database queries, API calls, secure operations |
| **Client** | Browser-side JavaScript | UI updates, local storage, navigation |
| **System** | Built-in ElevenLabs | End call, transfer, standard actions |
## Where Tools Live
Tools are defined inside `conversation_config.agent.prompt`. Webhook and client tools go in the `tools` array. System tools go in `built_in_tools`:
```python
conversation_config={
"agent": {
"prompt": {
"prompt": "You are helpful.",
"llm": "gemini-2.0-flash",
"tools": [...], # Webhook and client tools
"built_in_tools": {...} # System tools (end_call, transfer, etc.)
}
}
}
```
## Webhook Tools
Execute server-side logic when the agent needs external data or actions.
### Basic Webhook
```python
agent = client.conversational_ai.agents.create(
name="Weather Assistant",
conversation_config={
"agent": {
"prompt": {
"prompt": "You are a helpful assistant that can check the weather.",
"llm": "gemini-2.0-flash",
"tools": [{
"type": "webhook",
"name": "get_weather",
"description": "Get current weather for a city. Use when user asks about weather.",
"api_schema": {
"url": "https://api.example.com/weather",
"method": "POST",
"request_headers": {
"Authorization": "Bearer {{API_KEY}}"
},
"request_body_schema": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g., 'San Francisco'"
},
"units": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature units"
}
},
"required": ["city"]
}
}
}]
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
}
)
```
### Webhook Request Format
When the agent calls a webhook tool, ElevenLabs sends:
```json
{
"tool_call_id": "call_abc123",
"tool_name": "get_weather",
"parameters": {
"city": "San Francisco",
"units": "fahrenheit"
},
"conversation_id": "conv_xyz789"
}
```
### Webhook Response Format
Your server should respond with:
```json
{
"result": "The weather in San Francisco is 68°F and sunny."
}
```
Or for structured data:
```json
{
"result": {
"temperature": 68,
"condition": "sunny",
"humidity": 45
}
}
```
### Webhook with Authentication
```python
# Inside conversation_config.agent.prompt.tools:
{
"type": "webhook",
"name": "lookup_order",
"description": "Look up order status by order ID",
"response_timeout_secs": 10,
"api_schema": {
"url": "https://api.mystore.com/orders/lookup",
"method": "POST",
"request_headers": {
"Authorization": "Bearer {{ORDER_API_KEY}}",
"X-Store-ID": "store_123"
},
"request_body_schema": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "Order ID (e.g., ORD-12345)"
}
},
"required": ["order_id"]
}
}
}
```
Use workspace environment variables to keep a single server tool configuration working across
staging and production. `{{system_env__label}}` works in server tool URLs, secret environment
variables can populate `request_headers`, and auth-connection environment variables can populate
`api_schema.auth_connection`. The same environment-variable resolution model also applies to MCP
server connections.
```json
{
"api_schema": {
"url": "https://{{system_env__api_host}}.example.com/orders",
"method": "GET",
"request_headers": {
"X-Api-Key": { "env_var_label": "orders_api_key" }
},
"auth_connection": { "env_var_label": "orders_oauth" }
}
}
```
Workspace auth connections support OAuth2 client credentials, OAuth2 JWT, private key JWT,
basic auth, bearer auth, and custom header auth.
System dynamic variables are also available in tool parameters and headers. Use
`{{system__conversation_history}}` when a webhook or sub-agent needs the full conversation
context as a lazily evaluated JSON history object with user, agent, and tool entries.
### Webhook Tool Options
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `response_timeout_secs` | int | `20` | Timeout in seconds (5-120) |
| `disable_interruptions` | bool | `false` | Prevent user interruptions during tool execution |
| `execution_mode` | string | `"immediate"` | `immediate`, `post_tool_speech`, or `async` |
| `tool_call_sound` | string | - | Sound during execution: `typing`, `elevator1`-`elevator4` |
| `force_pre_tool_speech` | bool | `false` | Force agent to speak before executing tool |
| `tool_error_handling_mode` | string | `"auto"` | `auto`, `summarized`, `passthrough`, or `hide` |
**Note:** The default `api_schema.method` is `GET`. Always set `"method": "POST"` explicitly for webhook tools that send request bodies.
### Server Implementation (Node.js)
```javascript
app.post("/webhook/get_weather", async (req, res) => {
const { parameters, conversation_id } = req.body;
const { city, units = "fahrenheit" } = parameters;
// Fetch weather from your data source
const weather = await weatherService.get(city, units);
res.json({
result: `It's ${weather.temp}°${units === "celsius" ? "C" : "F"} and ${weather.condition} in ${city}.`,
});
});
```
### Server Implementation (Python)
```python
@app.post("/webhook/get_weather")
async def get_weather(request: Request):
data = await request.json()
city = data["parameters"]["city"]
units = data["parameters"].get("units", "fahrenheit")
# Fetch weather from your data source
weather = weather_service.get(city, units)
return {
"result": f"It's {weather['temp']}°{'C' if units == 'celsius' else 'F'} and {weather['condition']} in {city}."
}
```
## Client Tools
Execute JavaScript in the user's browser. Useful for UI updates, navigation, or accessing browser APIs.
### Defining Client Tools
Client tools are registered when starting a conversation:
```javascript
import { Conversation } from "@elevenlabs/client";
const conversation = await Conversation.startSession({
agentId: "your-agent-id",
clientTools: {
show_product: async ({ productId }) => {
// Update UI to show product
const modal = document.getElementById("product-modal");
modal.innerHTML = await fetchProductCard(productId);
modal.showModal();
return { success: true, message: "Showing product" };
},
navigate_to: async ({ page }) => {
// Navigate to a page
window.location.href = `/${page}`;
return { success: true };
},
save_preference: async ({ key, value }) => {
// Store in localStorage
localStorage.setItem(key, value);
return { saved: true };
},
},
});
```
### Registering Client Tools with Agent
Tell the agent about available client tools in `conversation_config.agent.prompt.tools`:
```python
agent = client.conversational_ai.agents.create(
name="Shopping Assistant",
conversation_config={
"agent": {
"prompt": {
"prompt": """You are a shopping assistant.
When users want to see a product, use show_product.
When users want to go somewhere, use navigate_to.""",
"llm": "gemini-2.0-flash",
"tools": [
{
"type": "client",
"name": "show_product",
"description": "Display a product card to the user",
"parameters": {
"type": "object",
"properties": {
"productId": {
"type": "string",
"description": "Product ID to display"
}
},
"required": ["productId"]
}
},
{
"type": "client",
"name": "navigate_to",
"description": "Navigate user to a different page",
"parameters": {
"type": "object",
"properties": {
"page": {
"type": "string",
"enum": ["cart", "checkout", "account", "home"],
"description": "Page to navigate to"
}
},
"required": ["page"]
}
}
]
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
}
)
```
### Client Tool Options
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `expects_response` | bool | `false` | Whether the tool returns data to the agent |
### Client Tool Return Values
Return data that the agent can use in conversation:
```javascript
clientTools: {
check_cart: async () => {
const cart = JSON.parse(localStorage.getItem("cart") || "[]");
return {
itemCount: cart.length,
total: cart.reduce((sum, item) => sum + item.price, 0),
items: cart.map((item) => item.name),
};
};
}
```
The agent receives this data and can say: "You have 3 items in your cart totaling $45.99."
## System Tools (built_in_tools)
Built-in tools provided by ElevenLabs. These are configured in `conversation_config.agent.prompt.built_in_tools` (not in the `tools` array):
```python
"built_in_tools": {
"end_call": {},
"transfer_to_number": {...},
"transfer_to_agent": {...},
"language_detection": {},
"skip_turn": {},
"voicemail_detection": {...},
"play_keypad_touch_tone": {}
}
```
Current API schemas also expose `agent_prompt_change`, `memory_entry_create`, `memory_entry_delete`, `memory_entry_search`, and `memory_entry_update` in `built_in_tools`.
### end_call
Ends the current conversation:
```python
"built_in_tools": {
"end_call": {}
}
```
The agent can say "Goodbye!" and then end the call programmatically.
### transfer_to_number
Transfer to a phone number (requires telephony integration):
```python
"built_in_tools": {
"transfer_to_number": {
"transfers": [{
"transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
"condition": "User asks to speak with a human agent"
}]
}
}
```
### transfer_to_agent
Transfer to another ElevenLabs agent:
```python
"built_in_tools": {
"transfer_to_agent": {
"transfers": [{
"agent_id": "other-agent-id",
"condition": "User asks about sales"
}]
}
}
```
## Best Practices
### Tool Descriptions
Write clear descriptions so the LLM knows when to use tools:
```python
# Good - specific and actionable
"description": "Look up order status. Use when customer asks about their order, delivery, or shipping."
# Bad - vague
"description": "Order tool"
```
### Parameter Descriptions
Help the LLM extract correct values:
```python
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "Order ID in format ORD-XXXXX (e.g., ORD-12345)"
},
"email": {
"type": "string",
"description": "Customer email address for verification"
}
}
}
```
### Error Handling
Configure how tool errors are shared with the agent using `tool_error_handling_mode`:
| Mode | Behavior |
|------|----------|
| `auto` | ElevenLabs automatically decides how to handle errors |
| `summarized` | Errors are summarized before being sent to the agent |
| `passthrough` | Full error details are passed to the agent |
| `hide` | Errors are hidden from the agent |
Return helpful error messages:
```javascript
// Server webhook
app.post("/webhook/lookup_order", async (req, res) => {
const { order_id } = req.body.parameters;
const order = await db.orders.find(order_id);
if (!order) {
return res.json({
result: {
error: true,
message: `Order ${order_id} not found. Please verify the order ID.`,
},
});
}
res.json({ result: order });
});
```
### Timeouts
Set reasonable timeouts for webhooks using `response_timeout_secs` (5-120 seconds, default 20):
```python
{
"type": "webhook",
"name": "slow_operation",
"description": "Run a slow operation",
"response_timeout_secs": 30,
"api_schema": {
"url": "https://api.example.com/slow-operation",
"method": "POST"
}
}
```
## Complete Example
```python
agent = client.conversational_ai.agents.create(
name="E-commerce Assistant",
conversation_config={
"agent": {
"first_message": "Hi! How can I help you today?",
"language": "en",
"prompt": {
"prompt": """You are an e-commerce support assistant.
Available actions:
- lookup_order: Check order status
- show_product: Display products to customer
- end_call: End conversation politely
- transfer_to_number: Transfer to human support
Always verify order ID before lookup. Offer transfer for complex issues.""",
"llm": "gemini-2.0-flash",
"tools": [
# Webhook: Server-side order lookup
{
"type": "webhook",
"name": "lookup_order",
"description": "Look up order status by order ID or email",
"api_schema": {
"url": "https://api.mystore.com/orders/lookup",
"method": "POST",
"request_headers": {"Authorization": "Bearer {{API_KEY}}"},
"request_body_schema": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"email": {"type": "string"}
}
}
}
},
# Client: Browser-side product display
{
"type": "client",
"name": "show_product",
"description": "Display product details to the customer",
"parameters": {
"type": "object",
"properties": {
"product_id": {"type": "string"}
},
"required": ["product_id"]
}
}
],
"built_in_tools": {
"end_call": {},
"transfer_to_number": {
"transfers": [{
"transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
"condition": "User asks for human support"
}]
}
}
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_flash_v2_5"}
}
)
```
@@ -0,0 +1,131 @@
# Installation
## CLI (Recommended)
The ElevenLabs CLI is the recommended way to create and manage agents:
```bash
npm install -g @elevenlabs/cli
# or
pnpm add -g @elevenlabs/cli
# or
yarn global add @elevenlabs/cli
```
Requires Node.js 16.0.0 or higher.
### Authentication
```bash
elevenlabs auth login # Authenticate with API key
elevenlabs auth whoami # Verify current login status
elevenlabs auth logout # Remove stored credentials
```
API keys are securely stored in `~/.agents/api_keys.json`.
### Quick Start
```bash
# Initialize a new project
elevenlabs agents init
# Create an agent from template
elevenlabs agents add "My Assistant" --template complete
# Push to ElevenLabs platform
elevenlabs agents push
```
## JavaScript / TypeScript SDK
For programmatic access and client-side integration:
```bash
npm install @elevenlabs/elevenlabs-js
```
> **Important:** Always use `@elevenlabs/elevenlabs-js`. The old `elevenlabs` npm package (v1.x) is deprecated and should not be used.
```javascript
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
// Option 1: Environment variable (recommended)
// Set ELEVENLABS_API_KEY in your environment
const client = new ElevenLabsClient();
// Option 2: Pass directly
const client = new ElevenLabsClient({ apiKey: "your-api-key" });
```
### Migrating from deprecated packages
If you have old packages installed, remove them:
```bash
# Remove deprecated packages
npm uninstall elevenlabs
# Install the current packages
npm install @elevenlabs/elevenlabs-js
# For client-side/browser usage, also install:
npm install @elevenlabs/client # Browser client
npm install @elevenlabs/react # React hooks
```
**Import changes:**
```javascript
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
import { Conversation } from "@elevenlabs/client";
import { useConversation } from "@elevenlabs/react";
```
## Python
```bash
pip install elevenlabs
```
```python
from elevenlabs import ElevenLabs
# Option 1: Environment variable (recommended)
# Set ELEVENLABS_API_KEY in your environment
client = ElevenLabs()
# Option 2: Pass directly
client = ElevenLabs(api_key="your-api-key")
```
## cURL / REST API
Set your API key as an environment variable:
```bash
export ELEVENLABS_API_KEY="your-api-key"
```
Include in requests via the `xi-api-key` header:
```bash
curl -X POST "https://api.elevenlabs.io/v1/convai/agents/create" \
-H "xi-api-key: $ELEVENLABS_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "My Agent", "conversation_config": {"agent": {"prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}'
```
## Getting an API Key
1. Sign up at [elevenlabs.io](https://elevenlabs.io)
2. Go to [API Keys](https://elevenlabs.io/app/settings/api-keys)
3. Click **Create API Key**
4. Copy and store securely
Or use the `setup-api-key` skill for guided setup.
## Environment Variables
| Variable | Description |
|----------|-------------|
| `ELEVENLABS_API_KEY` | Your ElevenLabs API key (required) |
@@ -0,0 +1,170 @@
# Outbound Calls
Make outbound phone calls using your ElevenLabs agent via Twilio integration.
## Prerequisites
1. A configured ElevenLabs agent
2. A Twilio phone number linked to your agent (obtain `agent_phone_number_id` from ElevenLabs dashboard)
3. Your ElevenLabs API key
## Basic Usage
See the [main agents skill](../SKILL.md#outbound-calls) for basic Python, JavaScript, and cURL examples.
## Request Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `agent_id` | string | Yes | The ID of your ElevenLabs agent |
| `agent_phone_number_id` | string | Yes | The ID of the Twilio phone number linked to your agent |
| `to_number` | string | Yes | The destination phone number (E.164 format) |
| `conversation_initiation_client_data` | object | No | Override conversation settings for this call |
| `call_recording_enabled` | boolean | No | Whether to let Twilio record the call |
| `telephony_call_config` | object | No | Telephony call settings like ringing timeout |
## Response
```json
{
"success": true,
"message": "Call initiated successfully",
"conversation_id": "conv_abc123",
"callSid": "CA1234567890abcdef"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `success` | boolean | Whether the call was initiated successfully |
| `message` | string | Status message |
| `conversation_id` | string | ElevenLabs conversation ID for tracking |
| `callSid` | string | Twilio Call SID for reference |
## Customizing the Call
Override agent settings for a specific call using `conversation_initiation_client_data`:
### Python
```python
response = client.conversational_ai.twilio.outbound_call(
agent_id="your-agent-id",
agent_phone_number_id="your-phone-number-id",
to_number="+1234567890",
call_recording_enabled=True,
conversation_initiation_client_data={
"conversation_config_override": {
"agent": {
"first_message": "Hello! This is a reminder about your appointment tomorrow.",
"language": "en"
},
"tts": {
"voice_id": "JBFqnCBsd6RMkjVDRZzb"
}
},
"dynamic_variables": {
"customer_name": "John",
"appointment_time": "2:00 PM"
}
}
)
```
### JavaScript
```javascript
const response = await client.conversationalAi.twilio.outboundCall({
agentId: "your-agent-id",
agentPhoneNumberId: "your-phone-number-id",
toNumber: "+1234567890",
callRecordingEnabled: true,
conversationInitiationClientData: {
conversationConfigOverride: {
agent: {
firstMessage: "Hello! This is a reminder about your appointment tomorrow.",
language: "en",
},
tts: {
voiceId: "JBFqnCBsd6RMkjVDRZzb",
},
},
dynamicVariables: {
customer_name: "John",
appointment_time: "2:00 PM",
},
},
});
```
## Configuration Overrides
### Agent Settings
| Option | Type | Description |
|--------|------|-------------|
| `first_message` | string | Custom greeting for this call |
| `language` | string | Language code (e.g., "en", "es", "fr") |
| `prompt` | object | Override agent prompt and LLM settings |
### TTS Settings
| Option | Type | Description |
|--------|------|-------------|
| `voice_id` | string | Voice ID to use for this call |
| `stability` | number | Voice stability (0.0-1.0) |
| `similarity_boost` | number | Voice similarity boost (0.0-1.0) |
| `speed` | number | Speech speed multiplier |
### Telephony Call Configuration
| Option | Type | Description |
|--------|------|-------------|
| `ringing_timeout_secs` | integer | How long to ring the recipient before giving up (default: `60`) |
### Dynamic Variables
Pass custom data to your agent's prompt using `dynamic_variables`. Reference them in your agent's prompt with `{{variable_name}}` syntax.
When assigning dynamic variables, you can use the `sanitize` option to remove sensitive values from tool responses before they are sent to the LLM and transcript, while still allowing variable assignment:
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `sanitize` | boolean | `false` | If true, the assignment's value is removed from tool responses before sending to LLM/transcript but still processed for variable assignment |
## Complete Example
```python
from elevenlabs import ElevenLabs
client = ElevenLabs()
# Make personalized outbound calls
customers = [
{"name": "Alice", "phone": "+1234567890", "balance": "$150.00"},
{"name": "Bob", "phone": "+0987654321", "balance": "$75.50"},
]
for customer in customers:
try:
response = client.conversational_ai.twilio.outbound_call(
agent_id="payment-reminder-agent",
agent_phone_number_id="your-phone-number-id",
to_number=customer["phone"],
call_recording_enabled=True,
conversation_initiation_client_data={
"conversation_config_override": {
"agent": {
"first_message": f"Hello {customer['name']}, this is a friendly reminder about your account."
}
},
"dynamic_variables": {
"customer_name": customer["name"],
"balance": customer["balance"]
}
}
)
print(f"Called {customer['name']}: {response.conversation_id}")
except Exception as e:
print(f"Failed to call {customer['name']}: {e}")
```
@@ -0,0 +1,365 @@
# Widget Embedding
Add a voice AI agent to any website with the ElevenLabs conversation widget.
## Basic Embed
```html
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
<script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>
```
This creates a floating button that users can click to start a voice conversation.
> **Note:** Widgets currently require public agents with authentication disabled. For authenticated flows, use the SDKs.
## Widget Attributes
### Required
| Attribute | Description |
|-----------|-------------|
| `agent-id` | Your ElevenLabs agent ID |
| `signed-url` | Alternative to `agent-id` when using signed URLs |
### Appearance
| Attribute | Description | Default |
|-----------|-------------|---------|
| `avatar-image-url` | URL for agent avatar image | ElevenLabs logo |
| `avatar-orb-color-1` | Primary orb gradient color | `#2792dc` |
| `avatar-orb-color-2` | Secondary orb gradient color | `#9ce6e6` |
### Text Labels
| Attribute | Description | Default |
|-----------|-------------|---------|
| `action-text` | Tooltip when hovering | "Talk to AI" |
| `start-call-text` | Button to start call | "Start call" |
| `end-call-text` | Button to end call | "End call" |
| `expand-text` | Expand chat button | "Open" |
| `collapse-text` | Collapse chat button | "Close" |
| `listening-text` | Listening state label | "Listening..." |
| `speaking-text` | Speaking state label | "Assistant speaking" |
### Behavior
| Attribute | Description | Default |
|-----------|-------------|---------|
| `variant` | Widget style: `compact` or `expanded` | `compact` |
| `server-location` | Server region (`us`, `eu-residency`, `in-residency`, `global`) | `us` |
| `dismissible` | Allow the user to minimize the widget | `false` |
| `disable-banner` | Hide "Powered by ElevenLabs" | `false` |
## Examples
### Custom Avatar
```html
<elevenlabs-convai
agent-id="your-agent-id"
avatar-image-url="https://example.com/your-avatar.png"
></elevenlabs-convai>
```
### Custom Colors
```html
<elevenlabs-convai
agent-id="your-agent-id"
avatar-orb-color-1="#ff6b6b"
avatar-orb-color-2="#ffd93d"
></elevenlabs-convai>
```
### Custom Text
```html
<elevenlabs-convai
agent-id="your-agent-id"
action-text="Chat with our AI assistant"
start-call-text="Begin conversation"
end-call-text="Hang up"
></elevenlabs-convai>
```
### Expanded Variant
```html
<elevenlabs-convai
agent-id="your-agent-id"
variant="expanded"
></elevenlabs-convai>
```
### Full Customization
```html
<elevenlabs-convai
agent-id="your-agent-id"
avatar-image-url="https://example.com/support-agent.png"
avatar-orb-color-1="#4f46e5"
avatar-orb-color-2="#818cf8"
action-text="Talk to Support"
start-call-text="Start voice chat"
end-call-text="End conversation"
expand-text="Open assistant"
collapse-text="Minimize"
></elevenlabs-convai>
```
## CSS Customization
The widget uses Shadow DOM but exposes CSS custom properties:
```css
elevenlabs-convai {
--elevenlabs-convai-widget-width: 400px;
--elevenlabs-convai-widget-height: 600px;
}
```
### Positioning
By default, the widget appears in the bottom-right corner. Override with CSS:
```css
elevenlabs-convai {
position: fixed;
bottom: 20px;
right: 20px;
/* Or position differently */
left: 20px;
right: auto;
}
```
### Z-Index
```css
elevenlabs-convai {
z-index: 9999;
}
```
## JavaScript Control
Access the widget element to control it programmatically:
```html
<elevenlabs-convai id="my-widget" agent-id="your-agent-id"></elevenlabs-convai>
<script>
const widget = document.getElementById("my-widget");
// Start a conversation
widget.startConversation();
// End the conversation
widget.endConversation();
// Listen for events
widget.addEventListener("conversationStarted", () => {
console.log("Conversation started");
});
widget.addEventListener("conversationEnded", () => {
console.log("Conversation ended");
});
</script>
```
### Custom Trigger Button
Hide the default widget and use your own button:
```html
<style>
elevenlabs-convai {
display: none;
}
</style>
<button onclick="document.getElementById('widget').startConversation()">
Talk to AI
</button>
<elevenlabs-convai id="widget" agent-id="your-agent-id"></elevenlabs-convai>
```
## Authentication
For agents with authentication enabled, pass a signed URL:
```html
<elevenlabs-convai id="widget" agent-id="your-agent-id"></elevenlabs-convai>
<script>
async function startAuthenticatedConversation() {
// Get signed URL from your backend
const response = await fetch("/api/get-signed-url");
const { signedUrl } = await response.json();
const widget = document.getElementById("widget");
widget.setAttribute("signed-url", signedUrl);
widget.startConversation();
}
</script>
```
Your backend:
```python
@app.get("/api/get-signed-url")
def get_signed_url():
signed_url = client.conversational_ai.conversations.get_signed_url(
agent_id="your-agent-id"
)
return {"signedUrl": signed_url.signed_url}
```
## Mobile Considerations
### Responsive Positioning
```css
/* Desktop: bottom-right */
elevenlabs-convai {
position: fixed;
bottom: 20px;
right: 20px;
}
/* Mobile: full-width bottom */
@media (max-width: 768px) {
elevenlabs-convai {
bottom: 0;
right: 0;
left: 0;
--elevenlabs-convai-widget-width: 100%;
}
}
```
### Touch-Friendly
The widget is touch-optimized by default. For better mobile UX:
```css
@media (max-width: 768px) {
elevenlabs-convai {
/* Larger touch target */
transform: scale(1.1);
transform-origin: bottom right;
}
}
```
## Multiple Widgets
You can have multiple widgets for different agents:
```html
<elevenlabs-convai
agent-id="support-agent-id"
action-text="Support"
style="right: 20px"
></elevenlabs-convai>
<elevenlabs-convai
agent-id="sales-agent-id"
action-text="Sales"
style="right: 100px"
></elevenlabs-convai>
```
## Framework Integration
### React
```jsx
function App() {
useEffect(() => {
// Load widget script
const script = document.createElement("script");
script.src = "https://unpkg.com/@elevenlabs/convai-widget-embed";
script.async = true;
document.body.appendChild(script);
return () => document.body.removeChild(script);
}, []);
return (
<div>
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
</div>
);
}
```
### Vue
```vue
<template>
<div>
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
</div>
</template>
<script setup>
import { onMounted } from "vue";
onMounted(() => {
const script = document.createElement("script");
script.src = "https://unpkg.com/@elevenlabs/convai-widget-embed";
script.async = true;
document.body.appendChild(script);
});
</script>
```
### Next.js
```jsx
import Script from "next/script";
export default function Page() {
return (
<>
<Script
src="https://unpkg.com/@elevenlabs/convai-widget-embed"
strategy="lazyOnload"
/>
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
</>
);
}
```
## Troubleshooting
### Widget Not Appearing
1. Check that the agent ID is correct
2. Verify the script is loaded (check Network tab)
3. Check for JavaScript errors in console
4. Ensure no CSS is hiding the widget
### Audio Issues
1. Ensure HTTPS (microphone requires secure context)
2. Check browser permissions for microphone
3. Test in a supported browser (Chrome, Firefox, Safari, Edge)
### CORS Errors
If using authentication, ensure your domain is in the agent's allowlist:
```python
platform_settings={
"auth": {
"enable_auth": True,
"allowlist": ["https://yourdomain.com"]
}
}
```