Initial release — OpenMontage: the first open-source agentic video production system
11 production pipelines, 47 tools, 124 agent skills. Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,302 @@
|
||||
---
|
||||
name: agents
|
||||
description: Build voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversation experience.
|
||||
license: MIT
|
||||
compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
|
||||
metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
|
||||
---
|
||||
|
||||
# ElevenLabs Agents Platform
|
||||
|
||||
Build voice AI agents with natural conversations, multiple LLM providers, custom tools, and easy web embedding.
|
||||
|
||||
> **Setup:** See [Installation Guide](references/installation.md) for CLI and SDK setup.
|
||||
|
||||
## Quick Start with CLI
|
||||
|
||||
The ElevenLabs CLI is the recommended way to create and manage agents:
|
||||
|
||||
```bash
|
||||
# Install CLI and authenticate
|
||||
npm install -g @elevenlabs/cli
|
||||
elevenlabs auth login
|
||||
|
||||
# Initialize project and create an agent
|
||||
elevenlabs agents init
|
||||
elevenlabs agents add "My Assistant" --template complete
|
||||
|
||||
# Push to ElevenLabs platform
|
||||
elevenlabs agents push
|
||||
```
|
||||
|
||||
**Available templates:** `complete`, `minimal`, `voice-only`, `text-only`, `customer-service`, `assistant`
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
from elevenlabs import ElevenLabs
|
||||
|
||||
client = ElevenLabs()
|
||||
|
||||
agent = client.conversational_ai.agents.create(
|
||||
name="My Assistant",
|
||||
enable_versioning=True,
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"first_message": "Hello! How can I help?",
|
||||
"language": "en",
|
||||
"prompt": {
|
||||
"prompt": "You are a helpful assistant. Be concise and friendly.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"temperature": 0.7
|
||||
}
|
||||
},
|
||||
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
### JavaScript
|
||||
|
||||
```javascript
|
||||
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
|
||||
const client = new ElevenLabsClient();
|
||||
|
||||
const agent = await client.conversationalAi.agents.create({
|
||||
name: "My Assistant",
|
||||
enableVersioning: true,
|
||||
conversationConfig: {
|
||||
agent: {
|
||||
firstMessage: "Hello! How can I help?",
|
||||
language: "en",
|
||||
prompt: {
|
||||
prompt: "You are a helpful assistant.",
|
||||
llm: "gemini-2.0-flash",
|
||||
temperature: 0.7
|
||||
}
|
||||
},
|
||||
tts: { voiceId: "JBFqnCBsd6RMkjVDRZzb" }
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### cURL
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.elevenlabs.io/v1/convai/agents/create?enable_versioning=true" \
|
||||
-H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
|
||||
-d '{"name": "My Assistant", "conversation_config": {"agent": {"first_message": "Hello!", "language": "en", "prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}'
|
||||
```
|
||||
|
||||
## Starting Conversations
|
||||
|
||||
**Server-side (Python):** Get signed URL for client connection:
|
||||
```python
|
||||
signed_url = client.conversational_ai.conversations.get_signed_url(
|
||||
agent_id="your-agent-id",
|
||||
environment="staging",
|
||||
)
|
||||
```
|
||||
|
||||
**Client-side (JavaScript):**
|
||||
```javascript
|
||||
import { Conversation } from "@elevenlabs/client";
|
||||
|
||||
const conversation = await Conversation.startSession({
|
||||
agentId: "your-agent-id",
|
||||
environment: "staging",
|
||||
onMessage: (msg) => console.log("Agent:", msg.message),
|
||||
onUserTranscript: (t) => console.log("User:", t.message),
|
||||
onError: (e) => console.error(e)
|
||||
});
|
||||
```
|
||||
|
||||
**React Hook:**
|
||||
```typescript
|
||||
import { useConversation } from "@elevenlabs/react";
|
||||
|
||||
const conversation = useConversation({ onMessage: (msg) => console.log(msg) });
|
||||
// Get a signed URL for the target environment from your backend, then:
|
||||
await conversation.startSession({ signedUrl: token });
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
| Provider | Models |
|
||||
|----------|--------|
|
||||
| OpenAI | `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` |
|
||||
| Anthropic | `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` |
|
||||
| Google | `gemini-3.1-flash-lite-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` |
|
||||
| ElevenLabs | `glm-45-air-fp8`, `qwen3-30b-a3b`, `gpt-oss-120b` |
|
||||
| Custom | `custom-llm` (bring your own endpoint) |
|
||||
|
||||
Use `GET /v1/convai/llm/list` to inspect the current model catalog, including deprecation state, token/context limits, and capability flags such as image-input support.
|
||||
|
||||
**Popular voices:** `JBFqnCBsd6RMkjVDRZzb` (George), `EXAVITQu4vr4xnSDxMaL` (Sarah), `onwK4e9ZLuTAKqWW03F9` (Daniel), `XB0fDUnXU5powFXDhCwa` (Charlotte)
|
||||
|
||||
**Turn eagerness:** `patient` (waits longer for user to finish), `normal`, or `eager` (responds quickly)
|
||||
|
||||
See [Agent Configuration](references/agent-configuration.md) for all options.
|
||||
|
||||
## Tools
|
||||
|
||||
Extend agents with webhook, client, or built-in system tools. Tools are defined inside `conversation_config.agent.prompt`:
|
||||
|
||||
Workspace environment variables can resolve per-environment server tool URLs, headers, and auth connections, and runtime system variables such as `{{system__conversation_history}}` can pass full conversation context into tool calls when needed.
|
||||
|
||||
```python
|
||||
"prompt": {
|
||||
"prompt": "You are a helpful assistant that can check the weather.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"tools": [
|
||||
# Webhook: server-side API call
|
||||
{"type": "webhook", "name": "get_weather", "description": "Get weather",
|
||||
"api_schema": {"url": "https://api.example.com/weather", "method": "POST",
|
||||
"request_body_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}},
|
||||
# Client: runs in the browser
|
||||
{"type": "client", "name": "show_product", "description": "Display a product",
|
||||
"parameters": {"type": "object", "properties": {"productId": {"type": "string"}}, "required": ["productId"]}}
|
||||
],
|
||||
"built_in_tools": {
|
||||
"end_call": {},
|
||||
"transfer_to_number": {"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Client tools** run in browser:
|
||||
```javascript
|
||||
clientTools: {
|
||||
show_product: async ({ productId }) => {
|
||||
document.getElementById("product").src = `/products/${productId}`;
|
||||
return { success: true };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
See [Client Tools Reference](references/client-tools.md) for complete documentation.
|
||||
|
||||
## Widget Embedding
|
||||
|
||||
```html
|
||||
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
|
||||
<script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>
|
||||
```
|
||||
|
||||
Customize with attributes: `avatar-image-url`, `action-text`, `start-call-text`, `end-call-text`.
|
||||
|
||||
See [Widget Embedding Reference](references/widget-embedding.md) for all options.
|
||||
|
||||
## Outbound Calls
|
||||
|
||||
Make outbound phone calls using your agent via Twilio integration:
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
response = client.conversational_ai.twilio.outbound_call(
|
||||
agent_id="your-agent-id",
|
||||
agent_phone_number_id="your-phone-number-id",
|
||||
to_number="+1234567890",
|
||||
call_recording_enabled=True
|
||||
)
|
||||
print(f"Call initiated: {response.conversation_id}")
|
||||
```
|
||||
|
||||
### JavaScript
|
||||
|
||||
```javascript
|
||||
const response = await client.conversationalAi.twilio.outboundCall({
|
||||
agentId: "your-agent-id",
|
||||
agentPhoneNumberId: "your-phone-number-id",
|
||||
toNumber: "+1234567890",
|
||||
callRecordingEnabled: true,
|
||||
});
|
||||
```
|
||||
|
||||
### cURL
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.elevenlabs.io/v1/convai/twilio/outbound-call" \
|
||||
-H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
|
||||
-d '{"agent_id": "your-agent-id", "agent_phone_number_id": "your-phone-number-id", "to_number": "+1234567890", "call_recording_enabled": true}'
|
||||
```
|
||||
|
||||
See [Outbound Calls Reference](references/outbound-calls.md) for configuration overrides and dynamic variables.
|
||||
|
||||
## Managing Agents
|
||||
|
||||
### Using CLI (Recommended)
|
||||
|
||||
```bash
|
||||
# List agents and check status
|
||||
elevenlabs agents list
|
||||
elevenlabs agents status
|
||||
|
||||
# Import agents from platform to local config
|
||||
elevenlabs agents pull # Import all agents
|
||||
elevenlabs agents pull --agent <agent-id> # Import specific agent
|
||||
|
||||
# Push local changes to platform
|
||||
elevenlabs agents push # Upload configurations
|
||||
elevenlabs agents push --dry-run # Preview changes first
|
||||
|
||||
# Add tools
|
||||
elevenlabs tools add-webhook "Weather API"
|
||||
elevenlabs tools add-client "UI Tool"
|
||||
```
|
||||
|
||||
### Project Structure
|
||||
|
||||
The CLI creates a project structure for managing agents:
|
||||
|
||||
```
|
||||
your_project/
|
||||
├── agents.json # Agent definitions
|
||||
├── tools.json # Tool configurations
|
||||
├── tests.json # Test configurations
|
||||
├── agent_configs/ # Individual agent configs
|
||||
├── tool_configs/ # Individual tool configs
|
||||
└── test_configs/ # Individual test configs
|
||||
```
|
||||
|
||||
### SDK Examples
|
||||
|
||||
```python
|
||||
# List
|
||||
agents = client.conversational_ai.agents.list()
|
||||
|
||||
# Get
|
||||
agent = client.conversational_ai.agents.get(agent_id="your-agent-id")
|
||||
|
||||
# Update (partial - only include fields to change)
|
||||
client.conversational_ai.agents.update(agent_id="your-agent-id", name="New Name")
|
||||
client.conversational_ai.agents.update(agent_id="your-agent-id",
|
||||
conversation_config={
|
||||
"agent": {"prompt": {"prompt": "New instructions", "llm": "claude-sonnet-4"}}
|
||||
})
|
||||
|
||||
# Delete
|
||||
client.conversational_ai.agents.delete(agent_id="your-agent-id")
|
||||
```
|
||||
|
||||
See [Agent Configuration](references/agent-configuration.md) for all configuration options and SDK examples.
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
try:
|
||||
agent = client.conversational_ai.agents.create(...)
|
||||
except Exception as e:
|
||||
print(f"API error: {e}")
|
||||
```
|
||||
|
||||
Common errors: **401** (invalid key), **404** (not found), **422** (invalid config), **429** (rate limit)
|
||||
|
||||
## References
|
||||
|
||||
- [Installation Guide](references/installation.md) - SDK setup and migration
|
||||
- [Agent Configuration](references/agent-configuration.md) - All config options and CRUD examples
|
||||
- [Client Tools](references/client-tools.md) - Webhook, client, and system tools
|
||||
- [Widget Embedding](references/widget-embedding.md) - Website integration
|
||||
- [Outbound Calls](references/outbound-calls.md) - Twilio phone call integration
|
||||
@@ -0,0 +1,606 @@
|
||||
# Agent Configuration
|
||||
|
||||
Complete reference for configuring conversational AI agents.
|
||||
|
||||
## Configuration Structure
|
||||
|
||||
```python
|
||||
agent = client.conversational_ai.agents.create(
|
||||
name="My Agent",
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"first_message": "Hello!",
|
||||
"language": "en",
|
||||
"prompt": { # LLM, system prompt, tools, and knowledge base
|
||||
"prompt": "You are helpful.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"tools": [...],
|
||||
"built_in_tools": {...}
|
||||
}
|
||||
},
|
||||
"tts": {...}, # Voice and TTS model settings
|
||||
"asr": {...}, # Speech recognition settings
|
||||
"turn": {...}, # Turn-taking behavior
|
||||
"conversation": {...}, # Duration, events, monitoring
|
||||
"vad": {...}, # Voice activity detection config
|
||||
"language_presets": {...} # Language-specific overrides
|
||||
},
|
||||
platform_settings={...} # Auth, call limits
|
||||
)
|
||||
```
|
||||
|
||||
## conversation_config
|
||||
|
||||
Controls the real-time conversation behavior.
|
||||
|
||||
### agent
|
||||
|
||||
```python
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"first_message": "Hello! How can I help you today?",
|
||||
"language": "en",
|
||||
"disable_first_message_interruptions": False,
|
||||
"prompt": {
|
||||
"prompt": "You are a helpful assistant.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"temperature": 0.7
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `first_message` | string | `""` | What the agent says when conversation starts |
|
||||
| `language` | string | `"en"` | ISO 639-1 language code (en, es, fr, etc.) |
|
||||
| `disable_first_message_interruptions` | bool | `false` | Prevent user from interrupting the first message |
|
||||
| `hinglish_mode` | bool | `false` | When enabled and language is Hindi, agent responds in Hinglish |
|
||||
| `dynamic_variables` | object | - | Config with `dynamic_variable_placeholders` containing key-value pairs |
|
||||
| `prompt` | object | - | LLM configuration (see prompt section below) |
|
||||
|
||||
### tts (Text-to-Speech)
|
||||
|
||||
```python
|
||||
conversation_config={
|
||||
"tts": {
|
||||
"voice_id": "JBFqnCBsd6RMkjVDRZzb",
|
||||
"model_id": "eleven_flash_v2_5",
|
||||
"stability": 0.5,
|
||||
"similarity_boost": 0.8,
|
||||
"speed": 1.0,
|
||||
"optimize_streaming_latency": 3,
|
||||
"expressive_mode": True
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `voice_id` | string | `"cjVigY5qzO86Huf0OWal"` | Voice to use |
|
||||
| `model_id` | string | - | TTS model (see below) |
|
||||
| `stability` | float | `0.5` | 0-1, lower = more expressive |
|
||||
| `similarity_boost` | float | `0.8` | 0-1, higher = closer to original voice |
|
||||
| `speed` | float | `1.0` | 0.7-1.2, speech speed multiplier |
|
||||
| `optimize_streaming_latency` | int | - | 0-4, higher = faster but lower quality |
|
||||
| `expressive_mode` | bool | `true` | Enable expressive voice generation |
|
||||
| `agent_output_audio_format` | string | - | Output audio codec format |
|
||||
| `pronunciation_dictionary_locators` | array | - | Pronunciation overrides |
|
||||
|
||||
**Available TTS models for agents:**
|
||||
|
||||
| Model ID | Languages | Latency |
|
||||
|----------|-----------|---------|
|
||||
| `eleven_flash_v2_5` | 32 | ~75ms (recommended) |
|
||||
| `eleven_flash_v2` | English | ~75ms |
|
||||
| `eleven_turbo_v2_5` | 32 | ~250-300ms |
|
||||
| `eleven_turbo_v2` | English | ~250-300ms |
|
||||
| `eleven_multilingual_v2` | 29 | Standard |
|
||||
| `eleven_v3_conversational` | 70+ | Standard |
|
||||
|
||||
### asr (Automatic Speech Recognition)
|
||||
|
||||
```python
|
||||
conversation_config={
|
||||
"asr": {
|
||||
"quality": "high",
|
||||
"keywords": ["ElevenLabs", "TechCorp"],
|
||||
"user_input_audio_format": "pcm_16000"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `quality` | string | `"high"` | Transcription quality level |
|
||||
| `provider` | string | `"elevenlabs"` | ASR provider (`elevenlabs` or `scribe_realtime`) |
|
||||
| `keywords` | array | - | Words to boost recognition accuracy |
|
||||
| `user_input_audio_format` | string | - | Input audio format (e.g., `pcm_16000`, `ulaw_8000`) |
|
||||
|
||||
### turn (Turn-Taking)
|
||||
|
||||
```python
|
||||
conversation_config={
|
||||
"turn": {
|
||||
"turn_timeout": 7,
|
||||
"turn_eagerness": "normal",
|
||||
"silence_end_call_timeout": -1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `turn_timeout` | number | `7` | Seconds to wait before re-engaging the user |
|
||||
| `turn_eagerness` | string | `"normal"` | How quickly agent responds: `patient`, `normal`, or `eager` |
|
||||
| `silence_end_call_timeout` | number | `-1` | Seconds of silence before ending call (-1 = disabled) |
|
||||
| `initial_wait_time` | number | - | Seconds to wait for user to start speaking |
|
||||
| `spelling_patience` | string | `"auto"` | Entity detection patience: `auto` or `off` |
|
||||
| `speculative_turn` | bool | `false` | Enable speculative turn detection |
|
||||
| `soft_timeout_config` | object | - | Configures a message if user is silent (see below) |
|
||||
|
||||
**soft_timeout_config:**
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `timeout_seconds` | number | `-1` | Seconds before soft timeout (-1 = disabled) |
|
||||
| `message` | string | `"Hhmmmm...yeah."` | What agent says on timeout |
|
||||
| `use_llm_generated_message` | bool | `false` | Let LLM generate the timeout message |
|
||||
|
||||
## prompt (nested in conversation_config.agent)
|
||||
|
||||
Configures the LLM behavior. This object lives at `conversation_config.agent.prompt`:
|
||||
|
||||
```python
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"prompt": {
|
||||
"prompt": "You are a helpful customer service agent...",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"temperature": 0.7,
|
||||
"max_tokens": 500,
|
||||
"tools": [...],
|
||||
"built_in_tools": {...},
|
||||
"knowledge_base": [...]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `prompt` | string | `""` | System prompt defining agent behavior |
|
||||
| `llm` | string | - | Model ID (see LLM providers below) |
|
||||
| `temperature` | float | `0` | 0-1, higher = more creative |
|
||||
| `max_tokens` | int | `-1` | Max tokens for LLM response (-1 = unlimited) |
|
||||
| `reasoning_effort` | string | - | Reasoning depth: `none`, `minimal`, `low`, `medium`, `high` (model-dependent) |
|
||||
| `thinking_budget` | int | - | Max thinking tokens for reasoning models |
|
||||
| `tools` | array | - | Webhook and client tool definitions |
|
||||
| `built_in_tools` | object | - | System tools (end_call, transfer, etc.) |
|
||||
| `tool_ids` | array | - | References to pre-configured tools |
|
||||
| `knowledge_base` | array | - | Documents for RAG |
|
||||
| `custom_llm` | object | - | Custom LLM endpoint config |
|
||||
| `timezone` | string | - | IANA timezone (e.g., `America/New_York`) |
|
||||
| `backup_llm_config` | object | - | Fallback LLM configuration |
|
||||
| `cascade_timeout_seconds` | number | `8` | Seconds before cascading to backup LLM (2-15) |
|
||||
| `mcp_server_ids` | array | - | MCP server IDs to connect |
|
||||
| `native_mcp_server_ids` | array | - | Native MCP server IDs |
|
||||
| `ignore_default_personality` | bool | - | Skip default personality instructions |
|
||||
|
||||
Workspace environment variables let one agent configuration span multiple deployments. Use
|
||||
`{{system_env__label}}` in server tool and MCP server URLs, `{ "env_var_label": "orders_api_key" }`
|
||||
for secret-backed tool headers, and `{ "env_var_label": "orders_oauth" }` in `auth_connection`
|
||||
to resolve per-environment auth connections at runtime.
|
||||
|
||||
### LLM Providers
|
||||
|
||||
| Provider | Model IDs |
|
||||
|----------|-----------|
|
||||
| OpenAI | `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` |
|
||||
| Anthropic | `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` |
|
||||
| Google | `gemini-3.1-flash-lite-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` |
|
||||
| ElevenLabs | `glm-45-air-fp8`, `qwen3-30b-a3b`, `gpt-oss-120b` (hosted, ultra-low latency) |
|
||||
| Custom | `custom-llm` (requires custom_llm config) |
|
||||
|
||||
Use `GET /v1/convai/llm/list` to inspect the current model catalog, including deprecation state, token/context limits, and capability flags such as image-input support.
|
||||
|
||||
### Custom LLM
|
||||
|
||||
The `custom_llm` field is nested inside `conversation_config.agent.prompt`:
|
||||
|
||||
```python
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"prompt": {
|
||||
"prompt": "You are helpful.",
|
||||
"llm": "custom-llm",
|
||||
"custom_llm": {
|
||||
"url": "https://your-llm-endpoint.com/v1/chat/completions",
|
||||
"model_id": "your-model-id",
|
||||
"api_key": {"secret_id": "your-secret-id"},
|
||||
"api_type": "chat_completions" # or "responses"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## platform_settings
|
||||
|
||||
Platform-level configuration for security, limits, summaries, and widget behavior.
|
||||
|
||||
```python
|
||||
platform_settings={
|
||||
"summary_language": "en",
|
||||
"widget": {
|
||||
"show_agent_status": True,
|
||||
"show_conversation_id": True
|
||||
},
|
||||
"auth": {
|
||||
"enable_auth": True,
|
||||
"allowlist": [{"hostname": "example.com"}]
|
||||
},
|
||||
"call_limits": {
|
||||
"agent_concurrency_limit": 10,
|
||||
"daily_limit": 100
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Top-Level Fields
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `summary_language` | string | Language for conversation analysis outputs such as summaries, titles, evaluation rationales, and data collection rationales. If omitted, ElevenLabs infers it from the conversation. |
|
||||
| `widget` | object | Hosted widget and shareable page configuration. See the widget table below for selected options. |
|
||||
| `auth` | object | Authentication and origin restrictions for agent access |
|
||||
| `call_limits` | object | Concurrency and daily usage limits |
|
||||
| `guardrails` | object | Built-in safety and policy controls for agent interactions |
|
||||
| `privacy` | object | Recording, retention, and conversation history redaction settings |
|
||||
|
||||
### auth
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `enable_auth` | bool | Require signed URLs/tokens for connections |
|
||||
| `allowlist` | array | Allowed origins for CORS |
|
||||
| `shareable_token` | string | Public conversation token |
|
||||
|
||||
### call_limits
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `agent_concurrency_limit` | int | Max simultaneous conversations (default: -1, unlimited) |
|
||||
| `daily_limit` | int | Max conversations per day (default: 100000) |
|
||||
| `bursting_enabled` | bool | Allow exceeding limits at 2x cost (default: true) |
|
||||
|
||||
### guardrails
|
||||
|
||||
Use `platform_settings.guardrails` to configure built-in safety controls for user input and agent behavior. The fields below cover the current schema additions that are most relevant in agent configs.
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `version` | string | Guardrail config version. Use `"1"` for the current schema. |
|
||||
| `focus` | object | Keeps the agent on-topic and aligned with the configured task. |
|
||||
| `prompt_injection` | object | Detects prompt injection and instruction override attempts. |
|
||||
| `custom` | object | Configures user-defined response validation guardrails. |
|
||||
| `content` | object | Configures category-specific content moderation guardrails. |
|
||||
|
||||
**focus / prompt_injection:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `is_enabled` | bool | Enables the guardrail. |
|
||||
|
||||
**content:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `execution_mode` | string | Guardrail execution mode: `streaming` or `blocking`. |
|
||||
| `config` | object | Category threshold settings for content moderation. |
|
||||
|
||||
**content.config:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `sexual` | object | Threshold settings for sexual content. |
|
||||
| `violence` | object | Threshold settings for violent content. |
|
||||
| `harassment` | object | Threshold settings for harassment. |
|
||||
| `self_harm` | object | Threshold settings for self-harm content. |
|
||||
| `profanity` | object | Threshold settings for profanity. |
|
||||
| `religion_or_politics` | object | Threshold settings for religion or politics content. |
|
||||
| `medical_and_legal_information` | object | Threshold settings for medical or legal information. |
|
||||
|
||||
**content.config.\<category\>:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `is_enabled` | bool | Enables moderation for the category. |
|
||||
| `threshold` | number or string | Category threshold as a numeric score or one of `low`, `medium`, or `high`. |
|
||||
|
||||
Blocking content guardrails and custom guardrails support a `trigger_action` that either ends
|
||||
the session immediately or retries the response. Retry removes the blocked reply, injects your
|
||||
feedback as a system message, and re-generates up to 3 times before the platform falls back to
|
||||
ending the session. Feedback templates can use `{{trigger_reason}}` and `{{agent_message}}`.
|
||||
|
||||
### privacy
|
||||
|
||||
Use `platform_settings.privacy` to control recording, retention, and redaction behavior. The redaction-specific field is:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `conversation_history_redaction` | object | Redacts configured entity types from stored transcripts, audio, and analysis. |
|
||||
|
||||
**conversation_history_redaction:**
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `enabled` | bool | `false` | Whether conversation history redaction is enabled |
|
||||
| `entities` | array | - | Entity types to redact. Use parent types such as `name` or specific values such as `name.name_given`, `email_address`, `contact_number`, `dob`, and `age`. |
|
||||
|
||||
### widget
|
||||
|
||||
Use `platform_settings.widget` to configure the hosted widget and shareable page defaults. For client-side embed attributes, see the widget embedding reference.
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `dismissible` | bool | `false` | Whether the widget can be dismissed by the user |
|
||||
| `show_agent_status` | bool | `false` | Whether to show working, done, or error status while tools are running |
|
||||
| `show_conversation_id` | bool | `true` | Whether to show the conversation ID after disconnection |
|
||||
| `strip_audio_tags` | bool | `true` | Whether to strip audio markup from messages |
|
||||
| `syntax_highlight_theme` | string | auto | Code block syntax highlighting theme (`light` or `dark`); omit it to let the widget auto-detect |
|
||||
|
||||
### conversation (inside conversation_config)
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `max_duration_seconds` | int | `600` | Max conversation duration |
|
||||
| `text_only` | bool | `false` | Text-only mode (avoids audio pricing) |
|
||||
| `monitoring_enabled` | bool | `false` | Enable real-time WebSocket monitoring |
|
||||
|
||||
## Additional Top-Level Fields
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `tags` | array | Classification labels for filtering (e.g., `["production"]`, `["test"]`) |
|
||||
| `workflow` | object | Conversation flow definition and tool interaction sequences |
|
||||
|
||||
## Knowledge Base / RAG
|
||||
|
||||
Knowledge base is configured inside `conversation_config.agent.prompt`:
|
||||
|
||||
```python
|
||||
agent = client.conversational_ai.agents.create(
|
||||
name="Support Agent",
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"prompt": {
|
||||
"prompt": "You are a support agent. Use the knowledge base to answer questions.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"knowledge_base": [
|
||||
{"type": "file", "id": "doc-id", "name": "Product Guide", "usage_mode": "auto"}
|
||||
],
|
||||
"rag": {
|
||||
"enabled": True,
|
||||
"embedding_model": "qwen3_embedding_4b",
|
||||
"max_documents_length": 50000,
|
||||
"max_retrieved_rag_chunks_count": 20
|
||||
}
|
||||
}
|
||||
},
|
||||
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
`rag.embedding_model` supports `e5_mistral_7b_instruct`, `multilingual_e5_large_instruct`, and `qwen3_embedding_4b`.
|
||||
|
||||
## CRUD Operations
|
||||
|
||||
### Using CLI (Recommended)
|
||||
|
||||
```bash
|
||||
# Initialize project
|
||||
elevenlabs agents init
|
||||
|
||||
# Create agent from template
|
||||
elevenlabs agents add "My Agent" --template complete
|
||||
elevenlabs agents add "Support Bot" --template customer-service
|
||||
|
||||
# List agents
|
||||
elevenlabs agents list
|
||||
|
||||
# Check status
|
||||
elevenlabs agents status
|
||||
|
||||
# Push local changes to platform
|
||||
elevenlabs agents push
|
||||
elevenlabs agents push --dry-run # Preview changes first
|
||||
|
||||
# Import agents from platform
|
||||
elevenlabs agents pull # Import all
|
||||
elevenlabs agents pull --agent <agent-id> # Import specific agent
|
||||
elevenlabs agents pull --update # Override local configs
|
||||
|
||||
# View available templates
|
||||
elevenlabs agents templates list
|
||||
elevenlabs agents templates show <template-name>
|
||||
|
||||
# Add tools
|
||||
elevenlabs tools add-webhook "API Tool"
|
||||
elevenlabs tools add-client "UI Tool"
|
||||
|
||||
# Generate widget code
|
||||
elevenlabs agents widget <agent-id>
|
||||
```
|
||||
|
||||
### SDK: List Agents
|
||||
|
||||
```python
|
||||
agents = client.conversational_ai.agents.list()
|
||||
for agent in agents.agents:
|
||||
print(f"{agent.name}: {agent.agent_id}")
|
||||
```
|
||||
|
||||
```javascript
|
||||
const agents = await client.conversationalAi.agents.list();
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.elevenlabs.io/v1/convai/agents" -H "xi-api-key: $ELEVENLABS_API_KEY"
|
||||
```
|
||||
|
||||
### SDK: Get Agent
|
||||
|
||||
```python
|
||||
agent = client.conversational_ai.agents.get(agent_id="your-agent-id")
|
||||
```
|
||||
|
||||
```javascript
|
||||
const agent = await client.conversationalAi.agents.get("your-agent-id");
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" -H "xi-api-key: $ELEVENLABS_API_KEY"
|
||||
```
|
||||
|
||||
### SDK: Update Agent
|
||||
|
||||
Only include fields you want to change. All other settings remain unchanged.
|
||||
|
||||
**Python:**
|
||||
```python
|
||||
# Update name
|
||||
client.conversational_ai.agents.update(agent_id="id", name="New Name")
|
||||
|
||||
# Update TTS voice
|
||||
client.conversational_ai.agents.update(agent_id="id", conversation_config={
|
||||
"tts": {"voice_id": "EXAVITQu4vr4xnSDxMaL", "model_id": "eleven_flash_v2_5"}
|
||||
})
|
||||
|
||||
# Update prompt/LLM (nested in agent)
|
||||
client.conversational_ai.agents.update(agent_id="id", conversation_config={
|
||||
"agent": {"prompt": {"prompt": "New instructions.", "llm": "claude-sonnet-4", "temperature": 0.8}}
|
||||
})
|
||||
|
||||
# Update first message
|
||||
client.conversational_ai.agents.update(agent_id="id", conversation_config={
|
||||
"agent": {"first_message": "Welcome back!"}
|
||||
})
|
||||
|
||||
# Update platform settings
|
||||
client.conversational_ai.agents.update(agent_id="id", platform_settings={
|
||||
"auth": {"enable_auth": True, "allowlist": [{"hostname": "myapp.com"}]}
|
||||
})
|
||||
```
|
||||
|
||||
**JavaScript:**
|
||||
```javascript
|
||||
await client.conversationalAi.agents.update("id", { name: "New Name" });
|
||||
await client.conversationalAi.agents.update("id", {
|
||||
conversationConfig: { tts: { voiceId: "EXAVITQu4vr4xnSDxMaL" } }
|
||||
});
|
||||
await client.conversationalAi.agents.update("id", {
|
||||
conversationConfig: { agent: { prompt: { prompt: "New instructions.", llm: "claude-sonnet-4" } } }
|
||||
});
|
||||
```
|
||||
|
||||
**cURL:**
|
||||
```bash
|
||||
curl -X PATCH "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" \
|
||||
-H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
|
||||
-d '{"name": "New Name"}'
|
||||
```
|
||||
|
||||
#### Updatable Fields
|
||||
|
||||
| Section | Fields |
|
||||
|---------|--------|
|
||||
| Root | `name`, `tags` |
|
||||
| `conversation_config.agent` | `first_message`, `language`, `disable_first_message_interruptions`, `dynamic_variables` |
|
||||
| `conversation_config.agent.prompt` | `prompt`, `llm`, `temperature`, `max_tokens`, `reasoning_effort`, `tools`, `built_in_tools`, `knowledge_base`, `custom_llm`, `timezone` |
|
||||
| `conversation_config.tts` | `voice_id`, `model_id`, `stability`, `similarity_boost`, `speed`, `optimize_streaming_latency`, `expressive_mode` |
|
||||
| `conversation_config.asr` | `quality`, `provider`, `keywords`, `user_input_audio_format` |
|
||||
| `conversation_config.turn` | `turn_timeout`, `turn_eagerness`, `silence_end_call_timeout`, `soft_timeout_config` |
|
||||
| `conversation_config.conversation` | `max_duration_seconds`, `text_only`, `monitoring_enabled` |
|
||||
| `platform_settings` | `summary_language`, `guardrails`, `privacy` |
|
||||
| `platform_settings.widget` | `dismissible`, `show_agent_status`, `show_conversation_id`, `strip_audio_tags`, `syntax_highlight_theme` |
|
||||
| `platform_settings.auth` | `enable_auth`, `allowlist` |
|
||||
| `platform_settings.call_limits` | `agent_concurrency_limit`, `daily_limit`, `bursting_enabled` |
|
||||
|
||||
### SDK: Delete Agent
|
||||
|
||||
```python
|
||||
client.conversational_ai.agents.delete(agent_id="your-agent-id")
|
||||
```
|
||||
|
||||
```javascript
|
||||
await client.conversationalAi.agents.delete("your-agent-id");
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -X DELETE "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" -H "xi-api-key: $ELEVENLABS_API_KEY"
|
||||
```
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
Use the CLI in your deployment pipeline:
|
||||
|
||||
```bash
|
||||
# Set API key as environment variable
|
||||
export ELEVENLABS_API_KEY="your-api-key"
|
||||
|
||||
# Push changes (non-interactive)
|
||||
elevenlabs agents push
|
||||
```
|
||||
|
||||
## Example Configurations
|
||||
|
||||
### Customer Support Agent
|
||||
|
||||
```python
|
||||
agent = client.conversational_ai.agents.create(
|
||||
name="Support Agent",
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"first_message": "Hi! Thanks for calling TechCorp support.",
|
||||
"language": "en",
|
||||
"prompt": {
|
||||
"prompt": "You are a customer support agent. Be helpful, professional, concise.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"temperature": 0.5,
|
||||
"built_in_tools": {
|
||||
"end_call": {},
|
||||
"transfer_to_number": {
|
||||
"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"tts": {"voice_id": "XB0fDUnXU5powFXDhCwa", "model_id": "eleven_flash_v2_5"},
|
||||
"turn": {"turn_eagerness": "normal", "turn_timeout": 7},
|
||||
"conversation": {"max_duration_seconds": 900}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
### Low-Latency Assistant
|
||||
|
||||
```python
|
||||
agent = client.conversational_ai.agents.create(
|
||||
name="Quick Assistant",
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"first_message": "Hey! What do you need?",
|
||||
"prompt": {
|
||||
"prompt": "Fast, efficient assistant. Brief answers.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"temperature": 0.3,
|
||||
"max_tokens": 100
|
||||
}
|
||||
},
|
||||
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_flash_v2_5", "optimize_streaming_latency": 4},
|
||||
"turn": {"turn_eagerness": "eager", "turn_timeout": 3}
|
||||
}
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,545 @@
|
||||
# Client Tools
|
||||
|
||||
Extend your agent with custom capabilities. Tools let the agent take actions beyond just talking.
|
||||
|
||||
## Tool Types
|
||||
|
||||
| Type | Execution | Use Case |
|
||||
|------|-----------|----------|
|
||||
| **Webhook** | Server-side via HTTP | Database queries, API calls, secure operations |
|
||||
| **Client** | Browser-side JavaScript | UI updates, local storage, navigation |
|
||||
| **System** | Built-in ElevenLabs | End call, transfer, standard actions |
|
||||
|
||||
## Where Tools Live
|
||||
|
||||
Tools are defined inside `conversation_config.agent.prompt`. Webhook and client tools go in the `tools` array. System tools go in `built_in_tools`:
|
||||
|
||||
```python
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"prompt": {
|
||||
"prompt": "You are helpful.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"tools": [...], # Webhook and client tools
|
||||
"built_in_tools": {...} # System tools (end_call, transfer, etc.)
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Webhook Tools
|
||||
|
||||
Execute server-side logic when the agent needs external data or actions.
|
||||
|
||||
### Basic Webhook
|
||||
|
||||
```python
|
||||
agent = client.conversational_ai.agents.create(
|
||||
name="Weather Assistant",
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"prompt": {
|
||||
"prompt": "You are a helpful assistant that can check the weather.",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"tools": [{
|
||||
"type": "webhook",
|
||||
"name": "get_weather",
|
||||
"description": "Get current weather for a city. Use when user asks about weather.",
|
||||
"api_schema": {
|
||||
"url": "https://api.example.com/weather",
|
||||
"method": "POST",
|
||||
"request_headers": {
|
||||
"Authorization": "Bearer {{API_KEY}}"
|
||||
},
|
||||
"request_body_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"city": {
|
||||
"type": "string",
|
||||
"description": "City name, e.g., 'San Francisco'"
|
||||
},
|
||||
"units": {
|
||||
"type": "string",
|
||||
"enum": ["celsius", "fahrenheit"],
|
||||
"description": "Temperature units"
|
||||
}
|
||||
},
|
||||
"required": ["city"]
|
||||
}
|
||||
}
|
||||
}]
|
||||
}
|
||||
},
|
||||
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
### Webhook Request Format
|
||||
|
||||
When the agent calls a webhook tool, ElevenLabs sends:
|
||||
|
||||
```json
|
||||
{
|
||||
"tool_call_id": "call_abc123",
|
||||
"tool_name": "get_weather",
|
||||
"parameters": {
|
||||
"city": "San Francisco",
|
||||
"units": "fahrenheit"
|
||||
},
|
||||
"conversation_id": "conv_xyz789"
|
||||
}
|
||||
```
|
||||
|
||||
### Webhook Response Format
|
||||
|
||||
Your server should respond with:
|
||||
|
||||
```json
|
||||
{
|
||||
"result": "The weather in San Francisco is 68°F and sunny."
|
||||
}
|
||||
```
|
||||
|
||||
Or for structured data:
|
||||
|
||||
```json
|
||||
{
|
||||
"result": {
|
||||
"temperature": 68,
|
||||
"condition": "sunny",
|
||||
"humidity": 45
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Webhook with Authentication
|
||||
|
||||
```python
|
||||
# Inside conversation_config.agent.prompt.tools:
|
||||
{
|
||||
"type": "webhook",
|
||||
"name": "lookup_order",
|
||||
"description": "Look up order status by order ID",
|
||||
"response_timeout_secs": 10,
|
||||
"api_schema": {
|
||||
"url": "https://api.mystore.com/orders/lookup",
|
||||
"method": "POST",
|
||||
"request_headers": {
|
||||
"Authorization": "Bearer {{ORDER_API_KEY}}",
|
||||
"X-Store-ID": "store_123"
|
||||
},
|
||||
"request_body_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"order_id": {
|
||||
"type": "string",
|
||||
"description": "Order ID (e.g., ORD-12345)"
|
||||
}
|
||||
},
|
||||
"required": ["order_id"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use workspace environment variables to keep a single server tool configuration working across
|
||||
staging and production. `{{system_env__label}}` works in server tool URLs, secret environment
|
||||
variables can populate `request_headers`, and auth-connection environment variables can populate
|
||||
`api_schema.auth_connection`. The same environment-variable resolution model also applies to MCP
|
||||
server connections.
|
||||
|
||||
```json
|
||||
{
|
||||
"api_schema": {
|
||||
"url": "https://{{system_env__api_host}}.example.com/orders",
|
||||
"method": "GET",
|
||||
"request_headers": {
|
||||
"X-Api-Key": { "env_var_label": "orders_api_key" }
|
||||
},
|
||||
"auth_connection": { "env_var_label": "orders_oauth" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Workspace auth connections support OAuth2 client credentials, OAuth2 JWT, private key JWT,
|
||||
basic auth, bearer auth, and custom header auth.
|
||||
|
||||
System dynamic variables are also available in tool parameters and headers. Use
|
||||
`{{system__conversation_history}}` when a webhook or sub-agent needs the full conversation
|
||||
context as a lazily evaluated JSON history object with user, agent, and tool entries.
|
||||
|
||||
### Webhook Tool Options
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `response_timeout_secs` | int | `20` | Timeout in seconds (5-120) |
|
||||
| `disable_interruptions` | bool | `false` | Prevent user interruptions during tool execution |
|
||||
| `execution_mode` | string | `"immediate"` | `immediate`, `post_tool_speech`, or `async` |
|
||||
| `tool_call_sound` | string | - | Sound during execution: `typing`, `elevator1`-`elevator4` |
|
||||
| `force_pre_tool_speech` | bool | `false` | Force agent to speak before executing tool |
|
||||
| `tool_error_handling_mode` | string | `"auto"` | `auto`, `summarized`, `passthrough`, or `hide` |
|
||||
|
||||
**Note:** The default `api_schema.method` is `GET`. Always set `"method": "POST"` explicitly for webhook tools that send request bodies.
|
||||
|
||||
### Server Implementation (Node.js)
|
||||
|
||||
```javascript
|
||||
app.post("/webhook/get_weather", async (req, res) => {
|
||||
const { parameters, conversation_id } = req.body;
|
||||
const { city, units = "fahrenheit" } = parameters;
|
||||
|
||||
// Fetch weather from your data source
|
||||
const weather = await weatherService.get(city, units);
|
||||
|
||||
res.json({
|
||||
result: `It's ${weather.temp}°${units === "celsius" ? "C" : "F"} and ${weather.condition} in ${city}.`,
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Server Implementation (Python)
|
||||
|
||||
```python
|
||||
@app.post("/webhook/get_weather")
|
||||
async def get_weather(request: Request):
|
||||
data = await request.json()
|
||||
city = data["parameters"]["city"]
|
||||
units = data["parameters"].get("units", "fahrenheit")
|
||||
|
||||
# Fetch weather from your data source
|
||||
weather = weather_service.get(city, units)
|
||||
|
||||
return {
|
||||
"result": f"It's {weather['temp']}°{'C' if units == 'celsius' else 'F'} and {weather['condition']} in {city}."
|
||||
}
|
||||
```
|
||||
|
||||
## Client Tools
|
||||
|
||||
Execute JavaScript in the user's browser. Useful for UI updates, navigation, or accessing browser APIs.
|
||||
|
||||
### Defining Client Tools
|
||||
|
||||
Client tools are registered when starting a conversation:
|
||||
|
||||
```javascript
|
||||
import { Conversation } from "@elevenlabs/client";
|
||||
|
||||
const conversation = await Conversation.startSession({
|
||||
agentId: "your-agent-id",
|
||||
clientTools: {
|
||||
show_product: async ({ productId }) => {
|
||||
// Update UI to show product
|
||||
const modal = document.getElementById("product-modal");
|
||||
modal.innerHTML = await fetchProductCard(productId);
|
||||
modal.showModal();
|
||||
return { success: true, message: "Showing product" };
|
||||
},
|
||||
|
||||
navigate_to: async ({ page }) => {
|
||||
// Navigate to a page
|
||||
window.location.href = `/${page}`;
|
||||
return { success: true };
|
||||
},
|
||||
|
||||
save_preference: async ({ key, value }) => {
|
||||
// Store in localStorage
|
||||
localStorage.setItem(key, value);
|
||||
return { saved: true };
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Registering Client Tools with Agent
|
||||
|
||||
Tell the agent about available client tools in `conversation_config.agent.prompt.tools`:
|
||||
|
||||
```python
|
||||
agent = client.conversational_ai.agents.create(
|
||||
name="Shopping Assistant",
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"prompt": {
|
||||
"prompt": """You are a shopping assistant.
|
||||
When users want to see a product, use show_product.
|
||||
When users want to go somewhere, use navigate_to.""",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"tools": [
|
||||
{
|
||||
"type": "client",
|
||||
"name": "show_product",
|
||||
"description": "Display a product card to the user",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"productId": {
|
||||
"type": "string",
|
||||
"description": "Product ID to display"
|
||||
}
|
||||
},
|
||||
"required": ["productId"]
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "client",
|
||||
"name": "navigate_to",
|
||||
"description": "Navigate user to a different page",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"page": {
|
||||
"type": "string",
|
||||
"enum": ["cart", "checkout", "account", "home"],
|
||||
"description": "Page to navigate to"
|
||||
}
|
||||
},
|
||||
"required": ["page"]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
### Client Tool Options
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `expects_response` | bool | `false` | Whether the tool returns data to the agent |
|
||||
|
||||
### Client Tool Return Values
|
||||
|
||||
Return data that the agent can use in conversation:
|
||||
|
||||
```javascript
|
||||
clientTools: {
|
||||
check_cart: async () => {
|
||||
const cart = JSON.parse(localStorage.getItem("cart") || "[]");
|
||||
return {
|
||||
itemCount: cart.length,
|
||||
total: cart.reduce((sum, item) => sum + item.price, 0),
|
||||
items: cart.map((item) => item.name),
|
||||
};
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
The agent receives this data and can say: "You have 3 items in your cart totaling $45.99."
|
||||
|
||||
## System Tools (built_in_tools)
|
||||
|
||||
Built-in tools provided by ElevenLabs. These are configured in `conversation_config.agent.prompt.built_in_tools` (not in the `tools` array):
|
||||
|
||||
```python
|
||||
"built_in_tools": {
|
||||
"end_call": {},
|
||||
"transfer_to_number": {...},
|
||||
"transfer_to_agent": {...},
|
||||
"language_detection": {},
|
||||
"skip_turn": {},
|
||||
"voicemail_detection": {...},
|
||||
"play_keypad_touch_tone": {}
|
||||
}
|
||||
```
|
||||
|
||||
Current API schemas also expose `agent_prompt_change`, `memory_entry_create`, `memory_entry_delete`, `memory_entry_search`, and `memory_entry_update` in `built_in_tools`.
|
||||
|
||||
### end_call
|
||||
|
||||
Ends the current conversation:
|
||||
|
||||
```python
|
||||
"built_in_tools": {
|
||||
"end_call": {}
|
||||
}
|
||||
```
|
||||
|
||||
The agent can say "Goodbye!" and then end the call programmatically.
|
||||
|
||||
### transfer_to_number
|
||||
|
||||
Transfer to a phone number (requires telephony integration):
|
||||
|
||||
```python
|
||||
"built_in_tools": {
|
||||
"transfer_to_number": {
|
||||
"transfers": [{
|
||||
"transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
|
||||
"condition": "User asks to speak with a human agent"
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### transfer_to_agent
|
||||
|
||||
Transfer to another ElevenLabs agent:
|
||||
|
||||
```python
|
||||
"built_in_tools": {
|
||||
"transfer_to_agent": {
|
||||
"transfers": [{
|
||||
"agent_id": "other-agent-id",
|
||||
"condition": "User asks about sales"
|
||||
}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Tool Descriptions
|
||||
|
||||
Write clear descriptions so the LLM knows when to use tools:
|
||||
|
||||
```python
|
||||
# Good - specific and actionable
|
||||
"description": "Look up order status. Use when customer asks about their order, delivery, or shipping."
|
||||
|
||||
# Bad - vague
|
||||
"description": "Order tool"
|
||||
```
|
||||
|
||||
### Parameter Descriptions
|
||||
|
||||
Help the LLM extract correct values:
|
||||
|
||||
```python
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"order_id": {
|
||||
"type": "string",
|
||||
"description": "Order ID in format ORD-XXXXX (e.g., ORD-12345)"
|
||||
},
|
||||
"email": {
|
||||
"type": "string",
|
||||
"description": "Customer email address for verification"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
Configure how tool errors are shared with the agent using `tool_error_handling_mode`:
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| `auto` | ElevenLabs automatically decides how to handle errors |
|
||||
| `summarized` | Errors are summarized before being sent to the agent |
|
||||
| `passthrough` | Full error details are passed to the agent |
|
||||
| `hide` | Errors are hidden from the agent |
|
||||
|
||||
Return helpful error messages:
|
||||
|
||||
```javascript
|
||||
// Server webhook
|
||||
app.post("/webhook/lookup_order", async (req, res) => {
|
||||
const { order_id } = req.body.parameters;
|
||||
|
||||
const order = await db.orders.find(order_id);
|
||||
|
||||
if (!order) {
|
||||
return res.json({
|
||||
result: {
|
||||
error: true,
|
||||
message: `Order ${order_id} not found. Please verify the order ID.`,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
res.json({ result: order });
|
||||
});
|
||||
```
|
||||
|
||||
### Timeouts
|
||||
|
||||
Set reasonable timeouts for webhooks using `response_timeout_secs` (5-120 seconds, default 20):
|
||||
|
||||
```python
|
||||
{
|
||||
"type": "webhook",
|
||||
"name": "slow_operation",
|
||||
"description": "Run a slow operation",
|
||||
"response_timeout_secs": 30,
|
||||
"api_schema": {
|
||||
"url": "https://api.example.com/slow-operation",
|
||||
"method": "POST"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Complete Example
|
||||
|
||||
```python
|
||||
agent = client.conversational_ai.agents.create(
|
||||
name="E-commerce Assistant",
|
||||
conversation_config={
|
||||
"agent": {
|
||||
"first_message": "Hi! How can I help you today?",
|
||||
"language": "en",
|
||||
"prompt": {
|
||||
"prompt": """You are an e-commerce support assistant.
|
||||
|
||||
Available actions:
|
||||
- lookup_order: Check order status
|
||||
- show_product: Display products to customer
|
||||
- end_call: End conversation politely
|
||||
- transfer_to_number: Transfer to human support
|
||||
|
||||
Always verify order ID before lookup. Offer transfer for complex issues.""",
|
||||
"llm": "gemini-2.0-flash",
|
||||
"tools": [
|
||||
# Webhook: Server-side order lookup
|
||||
{
|
||||
"type": "webhook",
|
||||
"name": "lookup_order",
|
||||
"description": "Look up order status by order ID or email",
|
||||
"api_schema": {
|
||||
"url": "https://api.mystore.com/orders/lookup",
|
||||
"method": "POST",
|
||||
"request_headers": {"Authorization": "Bearer {{API_KEY}}"},
|
||||
"request_body_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"order_id": {"type": "string"},
|
||||
"email": {"type": "string"}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
# Client: Browser-side product display
|
||||
{
|
||||
"type": "client",
|
||||
"name": "show_product",
|
||||
"description": "Display product details to the customer",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"product_id": {"type": "string"}
|
||||
},
|
||||
"required": ["product_id"]
|
||||
}
|
||||
}
|
||||
],
|
||||
"built_in_tools": {
|
||||
"end_call": {},
|
||||
"transfer_to_number": {
|
||||
"transfers": [{
|
||||
"transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
|
||||
"condition": "User asks for human support"
|
||||
}]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_flash_v2_5"}
|
||||
}
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,131 @@
|
||||
# Installation
|
||||
|
||||
## CLI (Recommended)
|
||||
|
||||
The ElevenLabs CLI is the recommended way to create and manage agents:
|
||||
|
||||
```bash
|
||||
npm install -g @elevenlabs/cli
|
||||
# or
|
||||
pnpm add -g @elevenlabs/cli
|
||||
# or
|
||||
yarn global add @elevenlabs/cli
|
||||
```
|
||||
|
||||
Requires Node.js 16.0.0 or higher.
|
||||
|
||||
### Authentication
|
||||
|
||||
```bash
|
||||
elevenlabs auth login # Authenticate with API key
|
||||
elevenlabs auth whoami # Verify current login status
|
||||
elevenlabs auth logout # Remove stored credentials
|
||||
```
|
||||
|
||||
API keys are securely stored in `~/.agents/api_keys.json`.
|
||||
|
||||
### Quick Start
|
||||
|
||||
```bash
|
||||
# Initialize a new project
|
||||
elevenlabs agents init
|
||||
|
||||
# Create an agent from template
|
||||
elevenlabs agents add "My Assistant" --template complete
|
||||
|
||||
# Push to ElevenLabs platform
|
||||
elevenlabs agents push
|
||||
```
|
||||
|
||||
## JavaScript / TypeScript SDK
|
||||
|
||||
For programmatic access and client-side integration:
|
||||
|
||||
```bash
|
||||
npm install @elevenlabs/elevenlabs-js
|
||||
```
|
||||
|
||||
> **Important:** Always use `@elevenlabs/elevenlabs-js`. The old `elevenlabs` npm package (v1.x) is deprecated and should not be used.
|
||||
|
||||
```javascript
|
||||
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
|
||||
|
||||
// Option 1: Environment variable (recommended)
|
||||
// Set ELEVENLABS_API_KEY in your environment
|
||||
const client = new ElevenLabsClient();
|
||||
|
||||
// Option 2: Pass directly
|
||||
const client = new ElevenLabsClient({ apiKey: "your-api-key" });
|
||||
```
|
||||
|
||||
### Migrating from deprecated packages
|
||||
|
||||
If you have old packages installed, remove them:
|
||||
|
||||
```bash
|
||||
# Remove deprecated packages
|
||||
npm uninstall elevenlabs
|
||||
|
||||
# Install the current packages
|
||||
npm install @elevenlabs/elevenlabs-js
|
||||
|
||||
# For client-side/browser usage, also install:
|
||||
npm install @elevenlabs/client # Browser client
|
||||
npm install @elevenlabs/react # React hooks
|
||||
```
|
||||
|
||||
**Import changes:**
|
||||
```javascript
|
||||
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
|
||||
import { Conversation } from "@elevenlabs/client";
|
||||
import { useConversation } from "@elevenlabs/react";
|
||||
```
|
||||
|
||||
## Python
|
||||
|
||||
```bash
|
||||
pip install elevenlabs
|
||||
```
|
||||
|
||||
```python
|
||||
from elevenlabs import ElevenLabs
|
||||
|
||||
# Option 1: Environment variable (recommended)
|
||||
# Set ELEVENLABS_API_KEY in your environment
|
||||
client = ElevenLabs()
|
||||
|
||||
# Option 2: Pass directly
|
||||
client = ElevenLabs(api_key="your-api-key")
|
||||
```
|
||||
|
||||
## cURL / REST API
|
||||
|
||||
Set your API key as an environment variable:
|
||||
|
||||
```bash
|
||||
export ELEVENLABS_API_KEY="your-api-key"
|
||||
```
|
||||
|
||||
Include in requests via the `xi-api-key` header:
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.elevenlabs.io/v1/convai/agents/create" \
|
||||
-H "xi-api-key: $ELEVENLABS_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name": "My Agent", "conversation_config": {"agent": {"prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}'
|
||||
```
|
||||
|
||||
## Getting an API Key
|
||||
|
||||
1. Sign up at [elevenlabs.io](https://elevenlabs.io)
|
||||
2. Go to [API Keys](https://elevenlabs.io/app/settings/api-keys)
|
||||
3. Click **Create API Key**
|
||||
4. Copy and store securely
|
||||
|
||||
Or use the `setup-api-key` skill for guided setup.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `ELEVENLABS_API_KEY` | Your ElevenLabs API key (required) |
|
||||
@@ -0,0 +1,170 @@
|
||||
# Outbound Calls
|
||||
|
||||
Make outbound phone calls using your ElevenLabs agent via Twilio integration.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. A configured ElevenLabs agent
|
||||
2. A Twilio phone number linked to your agent (obtain `agent_phone_number_id` from ElevenLabs dashboard)
|
||||
3. Your ElevenLabs API key
|
||||
|
||||
## Basic Usage
|
||||
|
||||
See the [main agents skill](../SKILL.md#outbound-calls) for basic Python, JavaScript, and cURL examples.
|
||||
|
||||
## Request Parameters
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `agent_id` | string | Yes | The ID of your ElevenLabs agent |
|
||||
| `agent_phone_number_id` | string | Yes | The ID of the Twilio phone number linked to your agent |
|
||||
| `to_number` | string | Yes | The destination phone number (E.164 format) |
|
||||
| `conversation_initiation_client_data` | object | No | Override conversation settings for this call |
|
||||
| `call_recording_enabled` | boolean | No | Whether to let Twilio record the call |
|
||||
| `telephony_call_config` | object | No | Telephony call settings like ringing timeout |
|
||||
|
||||
## Response
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Call initiated successfully",
|
||||
"conversation_id": "conv_abc123",
|
||||
"callSid": "CA1234567890abcdef"
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `success` | boolean | Whether the call was initiated successfully |
|
||||
| `message` | string | Status message |
|
||||
| `conversation_id` | string | ElevenLabs conversation ID for tracking |
|
||||
| `callSid` | string | Twilio Call SID for reference |
|
||||
|
||||
## Customizing the Call
|
||||
|
||||
Override agent settings for a specific call using `conversation_initiation_client_data`:
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
response = client.conversational_ai.twilio.outbound_call(
|
||||
agent_id="your-agent-id",
|
||||
agent_phone_number_id="your-phone-number-id",
|
||||
to_number="+1234567890",
|
||||
call_recording_enabled=True,
|
||||
conversation_initiation_client_data={
|
||||
"conversation_config_override": {
|
||||
"agent": {
|
||||
"first_message": "Hello! This is a reminder about your appointment tomorrow.",
|
||||
"language": "en"
|
||||
},
|
||||
"tts": {
|
||||
"voice_id": "JBFqnCBsd6RMkjVDRZzb"
|
||||
}
|
||||
},
|
||||
"dynamic_variables": {
|
||||
"customer_name": "John",
|
||||
"appointment_time": "2:00 PM"
|
||||
}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
### JavaScript
|
||||
|
||||
```javascript
|
||||
const response = await client.conversationalAi.twilio.outboundCall({
|
||||
agentId: "your-agent-id",
|
||||
agentPhoneNumberId: "your-phone-number-id",
|
||||
toNumber: "+1234567890",
|
||||
callRecordingEnabled: true,
|
||||
conversationInitiationClientData: {
|
||||
conversationConfigOverride: {
|
||||
agent: {
|
||||
firstMessage: "Hello! This is a reminder about your appointment tomorrow.",
|
||||
language: "en",
|
||||
},
|
||||
tts: {
|
||||
voiceId: "JBFqnCBsd6RMkjVDRZzb",
|
||||
},
|
||||
},
|
||||
dynamicVariables: {
|
||||
customer_name: "John",
|
||||
appointment_time: "2:00 PM",
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Configuration Overrides
|
||||
|
||||
### Agent Settings
|
||||
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `first_message` | string | Custom greeting for this call |
|
||||
| `language` | string | Language code (e.g., "en", "es", "fr") |
|
||||
| `prompt` | object | Override agent prompt and LLM settings |
|
||||
|
||||
### TTS Settings
|
||||
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `voice_id` | string | Voice ID to use for this call |
|
||||
| `stability` | number | Voice stability (0.0-1.0) |
|
||||
| `similarity_boost` | number | Voice similarity boost (0.0-1.0) |
|
||||
| `speed` | number | Speech speed multiplier |
|
||||
|
||||
### Telephony Call Configuration
|
||||
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `ringing_timeout_secs` | integer | How long to ring the recipient before giving up (default: `60`) |
|
||||
|
||||
### Dynamic Variables
|
||||
|
||||
Pass custom data to your agent's prompt using `dynamic_variables`. Reference them in your agent's prompt with `{{variable_name}}` syntax.
|
||||
|
||||
When assigning dynamic variables, you can use the `sanitize` option to remove sensitive values from tool responses before they are sent to the LLM and transcript, while still allowing variable assignment:
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `sanitize` | boolean | `false` | If true, the assignment's value is removed from tool responses before sending to LLM/transcript but still processed for variable assignment |
|
||||
|
||||
## Complete Example
|
||||
|
||||
```python
|
||||
from elevenlabs import ElevenLabs
|
||||
|
||||
client = ElevenLabs()
|
||||
|
||||
# Make personalized outbound calls
|
||||
customers = [
|
||||
{"name": "Alice", "phone": "+1234567890", "balance": "$150.00"},
|
||||
{"name": "Bob", "phone": "+0987654321", "balance": "$75.50"},
|
||||
]
|
||||
|
||||
for customer in customers:
|
||||
try:
|
||||
response = client.conversational_ai.twilio.outbound_call(
|
||||
agent_id="payment-reminder-agent",
|
||||
agent_phone_number_id="your-phone-number-id",
|
||||
to_number=customer["phone"],
|
||||
call_recording_enabled=True,
|
||||
conversation_initiation_client_data={
|
||||
"conversation_config_override": {
|
||||
"agent": {
|
||||
"first_message": f"Hello {customer['name']}, this is a friendly reminder about your account."
|
||||
}
|
||||
},
|
||||
"dynamic_variables": {
|
||||
"customer_name": customer["name"],
|
||||
"balance": customer["balance"]
|
||||
}
|
||||
}
|
||||
)
|
||||
print(f"Called {customer['name']}: {response.conversation_id}")
|
||||
except Exception as e:
|
||||
print(f"Failed to call {customer['name']}: {e}")
|
||||
```
|
||||
@@ -0,0 +1,365 @@
|
||||
# Widget Embedding
|
||||
|
||||
Add a voice AI agent to any website with the ElevenLabs conversation widget.
|
||||
|
||||
## Basic Embed
|
||||
|
||||
```html
|
||||
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
|
||||
<script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>
|
||||
```
|
||||
|
||||
This creates a floating button that users can click to start a voice conversation.
|
||||
|
||||
> **Note:** Widgets currently require public agents with authentication disabled. For authenticated flows, use the SDKs.
|
||||
|
||||
## Widget Attributes
|
||||
|
||||
### Required
|
||||
|
||||
| Attribute | Description |
|
||||
|-----------|-------------|
|
||||
| `agent-id` | Your ElevenLabs agent ID |
|
||||
| `signed-url` | Alternative to `agent-id` when using signed URLs |
|
||||
|
||||
### Appearance
|
||||
|
||||
| Attribute | Description | Default |
|
||||
|-----------|-------------|---------|
|
||||
| `avatar-image-url` | URL for agent avatar image | ElevenLabs logo |
|
||||
| `avatar-orb-color-1` | Primary orb gradient color | `#2792dc` |
|
||||
| `avatar-orb-color-2` | Secondary orb gradient color | `#9ce6e6` |
|
||||
|
||||
### Text Labels
|
||||
|
||||
| Attribute | Description | Default |
|
||||
|-----------|-------------|---------|
|
||||
| `action-text` | Tooltip when hovering | "Talk to AI" |
|
||||
| `start-call-text` | Button to start call | "Start call" |
|
||||
| `end-call-text` | Button to end call | "End call" |
|
||||
| `expand-text` | Expand chat button | "Open" |
|
||||
| `collapse-text` | Collapse chat button | "Close" |
|
||||
| `listening-text` | Listening state label | "Listening..." |
|
||||
| `speaking-text` | Speaking state label | "Assistant speaking" |
|
||||
|
||||
### Behavior
|
||||
|
||||
| Attribute | Description | Default |
|
||||
|-----------|-------------|---------|
|
||||
| `variant` | Widget style: `compact` or `expanded` | `compact` |
|
||||
| `server-location` | Server region (`us`, `eu-residency`, `in-residency`, `global`) | `us` |
|
||||
| `dismissible` | Allow the user to minimize the widget | `false` |
|
||||
| `disable-banner` | Hide "Powered by ElevenLabs" | `false` |
|
||||
|
||||
## Examples
|
||||
|
||||
### Custom Avatar
|
||||
|
||||
```html
|
||||
<elevenlabs-convai
|
||||
agent-id="your-agent-id"
|
||||
avatar-image-url="https://example.com/your-avatar.png"
|
||||
></elevenlabs-convai>
|
||||
```
|
||||
|
||||
### Custom Colors
|
||||
|
||||
```html
|
||||
<elevenlabs-convai
|
||||
agent-id="your-agent-id"
|
||||
avatar-orb-color-1="#ff6b6b"
|
||||
avatar-orb-color-2="#ffd93d"
|
||||
></elevenlabs-convai>
|
||||
```
|
||||
|
||||
### Custom Text
|
||||
|
||||
```html
|
||||
<elevenlabs-convai
|
||||
agent-id="your-agent-id"
|
||||
action-text="Chat with our AI assistant"
|
||||
start-call-text="Begin conversation"
|
||||
end-call-text="Hang up"
|
||||
></elevenlabs-convai>
|
||||
```
|
||||
|
||||
### Expanded Variant
|
||||
|
||||
```html
|
||||
<elevenlabs-convai
|
||||
agent-id="your-agent-id"
|
||||
variant="expanded"
|
||||
></elevenlabs-convai>
|
||||
```
|
||||
|
||||
### Full Customization
|
||||
|
||||
```html
|
||||
<elevenlabs-convai
|
||||
agent-id="your-agent-id"
|
||||
avatar-image-url="https://example.com/support-agent.png"
|
||||
avatar-orb-color-1="#4f46e5"
|
||||
avatar-orb-color-2="#818cf8"
|
||||
action-text="Talk to Support"
|
||||
start-call-text="Start voice chat"
|
||||
end-call-text="End conversation"
|
||||
expand-text="Open assistant"
|
||||
collapse-text="Minimize"
|
||||
></elevenlabs-convai>
|
||||
```
|
||||
|
||||
## CSS Customization
|
||||
|
||||
The widget uses Shadow DOM but exposes CSS custom properties:
|
||||
|
||||
```css
|
||||
elevenlabs-convai {
|
||||
--elevenlabs-convai-widget-width: 400px;
|
||||
--elevenlabs-convai-widget-height: 600px;
|
||||
}
|
||||
```
|
||||
|
||||
### Positioning
|
||||
|
||||
By default, the widget appears in the bottom-right corner. Override with CSS:
|
||||
|
||||
```css
|
||||
elevenlabs-convai {
|
||||
position: fixed;
|
||||
bottom: 20px;
|
||||
right: 20px;
|
||||
/* Or position differently */
|
||||
left: 20px;
|
||||
right: auto;
|
||||
}
|
||||
```
|
||||
|
||||
### Z-Index
|
||||
|
||||
```css
|
||||
elevenlabs-convai {
|
||||
z-index: 9999;
|
||||
}
|
||||
```
|
||||
|
||||
## JavaScript Control
|
||||
|
||||
Access the widget element to control it programmatically:
|
||||
|
||||
```html
|
||||
<elevenlabs-convai id="my-widget" agent-id="your-agent-id"></elevenlabs-convai>
|
||||
|
||||
<script>
|
||||
const widget = document.getElementById("my-widget");
|
||||
|
||||
// Start a conversation
|
||||
widget.startConversation();
|
||||
|
||||
// End the conversation
|
||||
widget.endConversation();
|
||||
|
||||
// Listen for events
|
||||
widget.addEventListener("conversationStarted", () => {
|
||||
console.log("Conversation started");
|
||||
});
|
||||
|
||||
widget.addEventListener("conversationEnded", () => {
|
||||
console.log("Conversation ended");
|
||||
});
|
||||
</script>
|
||||
```
|
||||
|
||||
### Custom Trigger Button
|
||||
|
||||
Hide the default widget and use your own button:
|
||||
|
||||
```html
|
||||
<style>
|
||||
elevenlabs-convai {
|
||||
display: none;
|
||||
}
|
||||
</style>
|
||||
|
||||
<button onclick="document.getElementById('widget').startConversation()">
|
||||
Talk to AI
|
||||
</button>
|
||||
|
||||
<elevenlabs-convai id="widget" agent-id="your-agent-id"></elevenlabs-convai>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
For agents with authentication enabled, pass a signed URL:
|
||||
|
||||
```html
|
||||
<elevenlabs-convai id="widget" agent-id="your-agent-id"></elevenlabs-convai>
|
||||
|
||||
<script>
|
||||
async function startAuthenticatedConversation() {
|
||||
// Get signed URL from your backend
|
||||
const response = await fetch("/api/get-signed-url");
|
||||
const { signedUrl } = await response.json();
|
||||
|
||||
const widget = document.getElementById("widget");
|
||||
widget.setAttribute("signed-url", signedUrl);
|
||||
widget.startConversation();
|
||||
}
|
||||
</script>
|
||||
```
|
||||
|
||||
Your backend:
|
||||
|
||||
```python
|
||||
@app.get("/api/get-signed-url")
|
||||
def get_signed_url():
|
||||
signed_url = client.conversational_ai.conversations.get_signed_url(
|
||||
agent_id="your-agent-id"
|
||||
)
|
||||
return {"signedUrl": signed_url.signed_url}
|
||||
```
|
||||
|
||||
## Mobile Considerations
|
||||
|
||||
### Responsive Positioning
|
||||
|
||||
```css
|
||||
/* Desktop: bottom-right */
|
||||
elevenlabs-convai {
|
||||
position: fixed;
|
||||
bottom: 20px;
|
||||
right: 20px;
|
||||
}
|
||||
|
||||
/* Mobile: full-width bottom */
|
||||
@media (max-width: 768px) {
|
||||
elevenlabs-convai {
|
||||
bottom: 0;
|
||||
right: 0;
|
||||
left: 0;
|
||||
--elevenlabs-convai-widget-width: 100%;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Touch-Friendly
|
||||
|
||||
The widget is touch-optimized by default. For better mobile UX:
|
||||
|
||||
```css
|
||||
@media (max-width: 768px) {
|
||||
elevenlabs-convai {
|
||||
/* Larger touch target */
|
||||
transform: scale(1.1);
|
||||
transform-origin: bottom right;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Multiple Widgets
|
||||
|
||||
You can have multiple widgets for different agents:
|
||||
|
||||
```html
|
||||
<elevenlabs-convai
|
||||
agent-id="support-agent-id"
|
||||
action-text="Support"
|
||||
style="right: 20px"
|
||||
></elevenlabs-convai>
|
||||
|
||||
<elevenlabs-convai
|
||||
agent-id="sales-agent-id"
|
||||
action-text="Sales"
|
||||
style="right: 100px"
|
||||
></elevenlabs-convai>
|
||||
```
|
||||
|
||||
## Framework Integration
|
||||
|
||||
### React
|
||||
|
||||
```jsx
|
||||
function App() {
|
||||
useEffect(() => {
|
||||
// Load widget script
|
||||
const script = document.createElement("script");
|
||||
script.src = "https://unpkg.com/@elevenlabs/convai-widget-embed";
|
||||
script.async = true;
|
||||
document.body.appendChild(script);
|
||||
|
||||
return () => document.body.removeChild(script);
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<div>
|
||||
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Vue
|
||||
|
||||
```vue
|
||||
<template>
|
||||
<div>
|
||||
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<script setup>
|
||||
import { onMounted } from "vue";
|
||||
|
||||
onMounted(() => {
|
||||
const script = document.createElement("script");
|
||||
script.src = "https://unpkg.com/@elevenlabs/convai-widget-embed";
|
||||
script.async = true;
|
||||
document.body.appendChild(script);
|
||||
});
|
||||
</script>
|
||||
```
|
||||
|
||||
### Next.js
|
||||
|
||||
```jsx
|
||||
import Script from "next/script";
|
||||
|
||||
export default function Page() {
|
||||
return (
|
||||
<>
|
||||
<Script
|
||||
src="https://unpkg.com/@elevenlabs/convai-widget-embed"
|
||||
strategy="lazyOnload"
|
||||
/>
|
||||
<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
|
||||
</>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Widget Not Appearing
|
||||
|
||||
1. Check that the agent ID is correct
|
||||
2. Verify the script is loaded (check Network tab)
|
||||
3. Check for JavaScript errors in console
|
||||
4. Ensure no CSS is hiding the widget
|
||||
|
||||
### Audio Issues
|
||||
|
||||
1. Ensure HTTPS (microphone requires secure context)
|
||||
2. Check browser permissions for microphone
|
||||
3. Test in a supported browser (Chrome, Firefox, Safari, Edge)
|
||||
|
||||
### CORS Errors
|
||||
|
||||
If using authentication, ensure your domain is in the agent's allowlist:
|
||||
|
||||
```python
|
||||
platform_settings={
|
||||
"auth": {
|
||||
"enable_auth": True,
|
||||
"allowlist": ["https://yourdomain.com"]
|
||||
}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user