Initial release — OpenMontage: the first open-source agentic video production system
11 production pipelines, 47 tools, 124 agent skills. Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
calesthio
@@ -0,0 +1,108 @@
|
||||
---
|
||||
name: music
|
||||
description: Generate music using ElevenLabs Music API. Use when creating instrumental tracks, songs with lyrics, background music, jingles, or any AI-generated music composition. Supports prompt-based generation, composition plans for granular control, and detailed output with metadata.
|
||||
license: MIT
|
||||
compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
|
||||
metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
|
||||
---
|
||||
|
||||
# ElevenLabs Music Generation
|
||||
|
||||
Generate music from text prompts - supports instrumental tracks, songs with lyrics, and fine-grained control via composition plans.
|
||||
|
||||
> **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
from elevenlabs import ElevenLabs
|
||||
|
||||
client = ElevenLabs()
|
||||
|
||||
audio = client.music.compose(
|
||||
prompt="A chill lo-fi hip hop beat with jazzy piano chords",
|
||||
music_length_ms=30000
|
||||
)
|
||||
|
||||
with open("output.mp3", "wb") as f:
|
||||
for chunk in audio:
|
||||
f.write(chunk)
|
||||
```
|
||||
|
||||
### JavaScript
|
||||
|
||||
```javascript
|
||||
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
|
||||
import { createWriteStream } from "fs";
|
||||
|
||||
const client = new ElevenLabsClient();
|
||||
const audio = await client.music.compose({
|
||||
prompt: "A chill lo-fi hip hop beat with jazzy piano chords",
|
||||
musicLengthMs: 30000,
|
||||
});
|
||||
audio.pipe(createWriteStream("output.mp3"));
|
||||
```
|
||||
|
||||
### cURL
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.elevenlabs.io/v1/music" \
|
||||
-H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
|
||||
-d '{"prompt": "A chill lo-fi beat", "music_length_ms": 30000}' --output output.mp3
|
||||
```
|
||||
|
||||
## Methods
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `music.compose` | Generate audio from a prompt or composition plan |
|
||||
| `music.composition_plan.create` | Generate a structured plan for fine-grained control |
|
||||
| `music.compose_detailed` | Generate audio + composition plan + metadata |
|
||||
| `music.upload` | Upload an audio file for later inpainting workflows and optionally extract its composition plan |
|
||||
|
||||
See [API Reference](references/api_reference.md) for full parameter details.
|
||||
|
||||
`music.upload` is available to enterprise clients with access to the inpainting feature.
|
||||
|
||||
## Composition Plans
|
||||
|
||||
For granular control, generate a composition plan first, modify it, then compose:
|
||||
|
||||
```python
|
||||
plan = client.music.composition_plan.create(
|
||||
prompt="An epic orchestral piece building to a climax",
|
||||
music_length_ms=60000
|
||||
)
|
||||
|
||||
# Inspect/modify styles and sections
|
||||
print(plan.positiveGlobalStyles) # e.g. ["orchestral", "epic", "cinematic"]
|
||||
|
||||
audio = client.music.compose(
|
||||
composition_plan=plan,
|
||||
music_length_ms=60000
|
||||
)
|
||||
```
|
||||
|
||||
## Content Restrictions
|
||||
|
||||
- Cannot reference specific artists, bands, or copyrighted lyrics
|
||||
- `bad_prompt` errors include a `prompt_suggestion` with alternative phrasing
|
||||
- `bad_composition_plan` errors include a `composition_plan_suggestion`
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
try:
|
||||
audio = client.music.compose(prompt="...", music_length_ms=30000)
|
||||
except Exception as e:
|
||||
print(f"API error: {e}")
|
||||
```
|
||||
|
||||
Common errors: 401 (invalid key), 422 (invalid params), 429 (rate limit).
|
||||
|
||||
## References
|
||||
|
||||
- [Installation Guide](references/installation.md)
|
||||
- [API Reference](references/api_reference.md)
|
||||
@@ -0,0 +1,200 @@
|
||||
# Music API Reference
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [compose](#compose)
|
||||
- [composition_plan.create](#composition_plancreate)
|
||||
- [compose_detailed](#compose_detailed)
|
||||
- [upload](#upload)
|
||||
- [Error Handling](#error-handling)
|
||||
|
||||
## compose
|
||||
|
||||
Generate music from a text prompt. Returns an audio stream.
|
||||
|
||||
### Parameters
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `prompt` | string | Yes* | Description of desired music |
|
||||
| `composition_plan` | object | Yes* | Pre-defined composition plan (alternative to prompt) |
|
||||
| `music_length_ms` | integer | No | Duration in milliseconds (3,000–600,000) when using `prompt`; if omitted, the model chooses |
|
||||
| `model_id` | string | No | Defaults to `music_v1` |
|
||||
| `force_instrumental` | boolean | No | Guarantee an instrumental output (prompt mode only) |
|
||||
| `respect_sections_durations` | boolean | No | Enforce exact `duration_ms` in each composition plan section |
|
||||
|
||||
*Provide either `prompt` or `composition_plan`, not both.
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
audio = client.music.compose(
|
||||
prompt="An upbeat electronic track with synth leads",
|
||||
music_length_ms=30000
|
||||
)
|
||||
|
||||
with open("output.mp3", "wb") as f:
|
||||
for chunk in audio:
|
||||
f.write(chunk)
|
||||
```
|
||||
|
||||
### JavaScript
|
||||
|
||||
```javascript
|
||||
const audio = await client.music.compose({
|
||||
prompt: "An upbeat electronic track with synth leads",
|
||||
musicLengthMs: 30000,
|
||||
});
|
||||
|
||||
const writeStream = createWriteStream("output.mp3");
|
||||
audio.pipe(writeStream);
|
||||
```
|
||||
|
||||
### With Composition Plan
|
||||
|
||||
```python
|
||||
plan = client.music.composition_plan.create(
|
||||
prompt="A jazz ballad with piano and saxophone",
|
||||
music_length_ms=60000
|
||||
)
|
||||
|
||||
# Modify the plan as needed
|
||||
audio = client.music.compose(
|
||||
composition_plan=plan,
|
||||
music_length_ms=60000
|
||||
)
|
||||
```
|
||||
|
||||
## composition_plan.create
|
||||
|
||||
Generate a structured composition plan from a prompt for granular control before generating audio.
|
||||
|
||||
### Parameters
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `prompt` | string | Yes | Music description |
|
||||
| `music_length_ms` | integer | Yes | Duration in milliseconds |
|
||||
|
||||
### Response Structure
|
||||
|
||||
```json
|
||||
{
|
||||
"positiveGlobalStyles": ["jazz", "smooth", "warm"],
|
||||
"negativeGlobalStyles": ["aggressive", "distorted"],
|
||||
"sections": [
|
||||
{
|
||||
"name": "Intro",
|
||||
"localStyles": ["soft", "building"],
|
||||
"duration_ms": 15000,
|
||||
"lines": [
|
||||
{ "text": "Instrumental intro", "type": "instrumental" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
plan = client.music.composition_plan.create(
|
||||
prompt="A peaceful ambient track with nature sounds",
|
||||
music_length_ms=60000
|
||||
)
|
||||
|
||||
# Inspect and modify the plan
|
||||
print(plan.positiveGlobalStyles)
|
||||
for section in plan.sections:
|
||||
print(f"{section.name}: {section.duration_ms}ms")
|
||||
```
|
||||
|
||||
## compose_detailed
|
||||
|
||||
Generate music while returning both the composition plan and metadata alongside the audio.
|
||||
|
||||
### Returns
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `json` | Composition plan + song metadata (includes lyrics if applicable) |
|
||||
| `filename` | Output file identifier |
|
||||
| `audio` | Audio bytes |
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
result = client.music.compose_detailed(
|
||||
prompt="A pop song about summer adventures",
|
||||
music_length_ms=120000
|
||||
)
|
||||
|
||||
# Access the composition plan and metadata
|
||||
print(result.json)
|
||||
|
||||
# Save the audio
|
||||
with open(result.filename, "wb") as f:
|
||||
f.write(result.audio)
|
||||
```
|
||||
|
||||
## upload
|
||||
|
||||
Upload a music file for later inpainting workflows. This endpoint is available to enterprise clients with access to the inpainting feature.
|
||||
|
||||
### Parameters
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `file` | file | Yes | The audio file to upload |
|
||||
| `extract_composition_plan` | boolean | No | If `true`, the response includes an extracted composition plan and may take longer to return |
|
||||
|
||||
### Returns
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `song_id` | Unique identifier for the uploaded song |
|
||||
| `composition_plan` | Extracted composition plan, or `null` when `extract_composition_plan` is not enabled |
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
client.music.upload(
|
||||
file="example_file",
|
||||
)
|
||||
```
|
||||
|
||||
### cURL
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.elevenlabs.io/v1/music/upload" \
|
||||
-H "xi-api-key: $ELEVENLABS_API_KEY" \
|
||||
-F "file=@<file1>"
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### bad_prompt
|
||||
|
||||
Occurs when the prompt references copyrighted material (specific artists, bands, or copyrighted lyrics). The error response includes a `prompt_suggestion` with alternative phrasing.
|
||||
|
||||
```python
|
||||
try:
|
||||
audio = client.music.compose(
|
||||
prompt="A song like Beatles",
|
||||
music_length_ms=30000
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"Request failed: {e}")
|
||||
```
|
||||
|
||||
### bad_composition_plan
|
||||
|
||||
Returned when a composition plan contains copyrighted styles. The error includes a `composition_plan_suggestion` with corrected styles. No suggestion is provided for harmful content.
|
||||
|
||||
### Common HTTP Errors
|
||||
|
||||
| Code | Meaning |
|
||||
|------|---------|
|
||||
| 401 | Invalid API key |
|
||||
| 422 | Invalid parameters |
|
||||
| 429 | Rate limit exceeded |
|
||||
@@ -0,0 +1,65 @@
|
||||
# Installation
|
||||
|
||||
## JavaScript / TypeScript
|
||||
|
||||
```bash
|
||||
npm install @elevenlabs/elevenlabs-js
|
||||
```
|
||||
|
||||
> **Important:** Always use `@elevenlabs/elevenlabs-js`. The old `elevenlabs` npm package (v1.x) is deprecated and should not be used.
|
||||
|
||||
```javascript
|
||||
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
|
||||
|
||||
// Option 1: Environment variable (recommended)
|
||||
// Set ELEVENLABS_API_KEY in your environment
|
||||
const client = new ElevenLabsClient();
|
||||
|
||||
// Option 2: Pass directly
|
||||
const client = new ElevenLabsClient({ apiKey: "your-api-key" });
|
||||
```
|
||||
|
||||
## Python
|
||||
|
||||
```bash
|
||||
pip install elevenlabs
|
||||
```
|
||||
|
||||
```python
|
||||
from elevenlabs import ElevenLabs
|
||||
|
||||
# Option 1: Environment variable (recommended)
|
||||
# Set ELEVENLABS_API_KEY in your environment
|
||||
client = ElevenLabs()
|
||||
|
||||
# Option 2: Pass directly
|
||||
client = ElevenLabs(api_key="your-api-key")
|
||||
```
|
||||
|
||||
## cURL / REST API
|
||||
|
||||
Set your API key as an environment variable:
|
||||
|
||||
```bash
|
||||
export ELEVENLABS_API_KEY="your-api-key"
|
||||
```
|
||||
|
||||
Include in requests via the `xi-api-key` header:
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.elevenlabs.io/v1/music" \
|
||||
-H "xi-api-key: $ELEVENLABS_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"prompt": "A chill lo-fi beat", "music_length_ms": 30000}'
|
||||
```
|
||||
|
||||
## Getting an API Key
|
||||
|
||||
1. Sign up at [elevenlabs.io](https://elevenlabs.io)
|
||||
2. Go to [API Keys](https://elevenlabs.io/app/settings/api-keys)
|
||||
3. Click **Create API Key**
|
||||
4. Copy and store securely
|
||||
|
||||
Or use the `setup-api-key` skill for guided setup.
|
||||
|
||||
**Note:** Music generation requires a paid ElevenLabs plan.
|
||||
Reference in New Issue
Block a user