Initial release — OpenMontage: the first open-source agentic video production system
11 production pipelines, 47 tools, 124 agent skills. Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
calesthio
@@ -0,0 +1,52 @@
|
||||
# Asset Director - Localization Dub Pipeline
|
||||
|
||||
## When To Use
|
||||
|
||||
This stage produces the localized asset kit: translated subtitle files, dubbed audio, optional lip-sync renders, and any language-specific replacements needed for the final outputs.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Layer | Resource | Purpose |
|
||||
|-------|----------|---------|
|
||||
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
|
||||
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Language plan and transcript package |
|
||||
| Tools | `tts_selector`, `subtitle_gen`, `lip_sync`, `audio_enhance` — `tts_selector` auto-discovers all available TTS providers from the registry | Dubbed audio, subtitle, and optional lip-sync production |
|
||||
| Playbook | Active style playbook | Subtitle and replacement-text rules |
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Produce Subtitle Assets First
|
||||
|
||||
Create the subtitle or caption package for each language. This gives a reviewable fallback even if dubbed-audio generation or lip sync is blocked.
|
||||
|
||||
### 2. Generate Dubbed Audio Per Language
|
||||
|
||||
Use the approved translated script package, not raw machine output. Record which voice or synthesis path was used for each language.
|
||||
|
||||
### 3. Treat Lip Sync As Optional
|
||||
|
||||
Only generate lip-sync assets for scenes and languages that actually need it. If the tool path is blocked, record that and keep the dub-audio path alive.
|
||||
|
||||
### 4. Use Metadata For Localization Truth
|
||||
|
||||
Recommended metadata keys:
|
||||
|
||||
- `subtitle_assets_by_language`
|
||||
- `dub_audio_assets_by_language`
|
||||
- `lip_sync_assets_by_language`
|
||||
- `voice_map`
|
||||
- `pronunciation_warnings`
|
||||
- `blocked_assets`
|
||||
|
||||
### 5. Quality Gate
|
||||
|
||||
- subtitle assets exist,
|
||||
- dubbed audio assets exist for planned dub outputs,
|
||||
- lip-sync remains explicitly optional,
|
||||
- every referenced file exists.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Generating dubbed audio before finalizing translation review.
|
||||
- Treating lip sync as mandatory for every language.
|
||||
- Failing to record which language asset maps to which voice and subtitle set.
|
||||
@@ -0,0 +1,57 @@
|
||||
# Compose Director - Localization Dub Pipeline
|
||||
|
||||
## When To Use
|
||||
|
||||
Render the localized outputs. The quality bar is intelligibility, timing coherence, and clear version labeling across every language package.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Layer | Resource | Purpose |
|
||||
|-------|----------|---------|
|
||||
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
|
||||
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | Locale-specific render instructions |
|
||||
| Tools | `video_compose`, `audio_mixer`, `video_trimmer`, `audio_enhance` | Final render and audio finishing |
|
||||
| Playbook | Active style playbook | Subtitle placement and output quality |
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Render By Locale
|
||||
|
||||
Treat each target language as its own deliverable set. Keep names and output directories explicit.
|
||||
|
||||
### 2. Expect Timing Adjustments
|
||||
|
||||
Allow for:
|
||||
|
||||
- subtitle reflow,
|
||||
- dub-audio duration drift,
|
||||
- longer CTA holds,
|
||||
- optional trims or coverage sections.
|
||||
|
||||
### 3. Verify Every Locale
|
||||
|
||||
Record important findings in:
|
||||
|
||||
- `render_report.verification_notes`
|
||||
- `render_report.warnings`
|
||||
- `render_report.metadata.locale_notes`
|
||||
|
||||
Check:
|
||||
|
||||
- intelligibility,
|
||||
- subtitle fit,
|
||||
- obvious sync drift,
|
||||
- version labeling.
|
||||
|
||||
### 4. Quality Gate
|
||||
|
||||
- each locale output exists,
|
||||
- the dub and subtitle timing are acceptable,
|
||||
- labels and filenames are unambiguous,
|
||||
- warnings are preserved.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Rendering all locales as if they were timing-identical.
|
||||
- Forgetting to re-check subtitle line length after translation.
|
||||
- Naming outputs in ways that hide the locale or treatment mode.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Edit Director - Localization Dub Pipeline
|
||||
|
||||
## When To Use
|
||||
|
||||
Translate the scene plan and localized asset kit into concrete timeline decisions for each language output. The goal is to preserve the source structure where possible without pretending all languages land on the same timing.
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Preserve Structure By Default
|
||||
|
||||
Keep the original scene order and major timing unless the translated audio clearly requires extension, compression, or coverage.
|
||||
|
||||
### 2. Apply The Chosen Dub Mode
|
||||
|
||||
Per deliverable, decide where to:
|
||||
|
||||
- keep original picture with new subtitles,
|
||||
- replace only the audio,
|
||||
- use lip-sync output,
|
||||
- cover mismatch with graphics or B-roll.
|
||||
|
||||
### 3. Keep Language Variants Organized
|
||||
|
||||
Separate timeline decisions by locale so versioning stays clear all the way into compose and publish.
|
||||
|
||||
### 4. Use Metadata For Variant Control
|
||||
|
||||
Recommended metadata keys:
|
||||
|
||||
- `locale_timeline_map`
|
||||
- `timing_adjustments`
|
||||
- `coverage_sections`
|
||||
- `subtitle_strategy_by_locale`
|
||||
|
||||
### 5. Quality Gate
|
||||
|
||||
- language variants are explicit,
|
||||
- timing changes are recorded,
|
||||
- coverage decisions are deliberate,
|
||||
- the original structure is only changed where necessary.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Forcing every language to match source timing exactly.
|
||||
- Mixing locale-specific notes into one ambiguous edit list.
|
||||
- Hiding sections where the dub treatment is visually weak.
|
||||
@@ -0,0 +1,133 @@
|
||||
# Executive Producer — Localization Dub Pipeline
|
||||
|
||||
## When to Use
|
||||
|
||||
You are the **Executive Producer (EP)** for a localization/dubbing project. You orchestrate the pipeline serially with quality gates focused on **translation accuracy, timing preservation, lip-sync quality, and per-locale consistency**.
|
||||
|
||||
**No pre-production stages.** Source video exists in one language. The EP manages the complexity of producing multiple language variants while preserving the original's timing and quality.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Layer | Resource | Purpose |
|
||||
|-------|----------|---------|
|
||||
| Pipeline | `pipeline_defs/localization-dub.yaml` | Stage definitions |
|
||||
| Skills | All 7 director skills + `meta/reviewer` | Stage execution |
|
||||
| Schemas | All artifact schemas | Validation |
|
||||
| Playbook | Active style playbook | Quality constraints |
|
||||
|
||||
## Cumulative State
|
||||
|
||||
```
|
||||
EP_STATE:
|
||||
pipeline: localization-dub
|
||||
playbook: <selected>
|
||||
budget_total_usd: <configured>
|
||||
budget_spent_usd: 0.0
|
||||
|
||||
# Localization-specific
|
||||
source_language: null
|
||||
target_languages: []
|
||||
dub_mode_per_locale: {} # language → subtitle_only / dub / dub_with_lipsync
|
||||
glossary_terms: [] # protected terms that must not be translated
|
||||
timing_drift_tolerance: 0.5 # seconds
|
||||
|
||||
artifacts:
|
||||
idea: null
|
||||
script: null
|
||||
scene_plan: null
|
||||
assets: null
|
||||
edit: null
|
||||
compose: null
|
||||
publish: null
|
||||
|
||||
revision_counts: {}
|
||||
issues_log: []
|
||||
```
|
||||
|
||||
## EP-Specific Cross-Stage Checks
|
||||
|
||||
### After IDEA stage:
|
||||
```
|
||||
CHECK: Scope definition
|
||||
- Source and target languages explicit?
|
||||
- Deliverable mode clear per language (subtitle / dub / dub+lipsync)?
|
||||
- Glossary and protected terms captured?
|
||||
- Review requirements noted?
|
||||
```
|
||||
|
||||
### After SCRIPT stage:
|
||||
```
|
||||
CHECK: Transcript truth
|
||||
- Source transcript accurate and timestamped?
|
||||
- Glossary terms preserved in translations?
|
||||
- Translated scripts reviewable before synthesis?
|
||||
- Duration estimates per language reasonable (some languages expand 20-30%)?
|
||||
```
|
||||
|
||||
### After SCENE_PLAN stage:
|
||||
```
|
||||
CHECK: Dub mode feasibility
|
||||
- Is the chosen dub mode realistic per locale?
|
||||
- Lip-sync limited to shots that can support it (front-facing, clear mouth)?
|
||||
- Timing drift risks mapped (which languages will run long)?
|
||||
- On-screen text replacement planned if needed?
|
||||
```
|
||||
|
||||
### After ASSETS stage:
|
||||
```
|
||||
CHECK: Locale asset completeness
|
||||
- Subtitle files exist for every target language?
|
||||
- Dubbed audio generated for every dub-mode language?
|
||||
- TTS voice quality acceptable for each language?
|
||||
- Lip-sync applied only where planned?
|
||||
- Budget gate: 90% threshold (localization can be expensive with many languages)
|
||||
```
|
||||
|
||||
### After EDIT stage:
|
||||
```
|
||||
CHECK: Timing preservation
|
||||
- Source structure preserved unless timing forces change?
|
||||
- CTA and legal copy survive translation?
|
||||
- Language variants organized consistently?
|
||||
- Timing drift within tolerance per segment?
|
||||
```
|
||||
|
||||
### After COMPOSE stage:
|
||||
```
|
||||
CHECK: Per-locale validation
|
||||
- Each language output rendered and intelligible?
|
||||
- Subtitle timing matches speech in each locale?
|
||||
- Version labeling unambiguous (language code in filename)?
|
||||
- Audio quality consistent across locales?
|
||||
```
|
||||
|
||||
## Quality Gates Summary
|
||||
|
||||
| Gate | After Stage | What's Checked | Fail Action |
|
||||
|------|-------------|---------------|-------------|
|
||||
| G1 | idea | Scope, languages, dub modes | Revise |
|
||||
| G2 | script | Transcript accuracy, glossary, translations | Revise |
|
||||
| G3 | scene_plan | Dub mode feasibility, timing risks | Revise |
|
||||
| G4 | assets | Locale completeness, TTS quality, budget | Revise |
|
||||
| G5 | edit | Timing preservation, structure | Revise |
|
||||
| G6 | compose | Per-locale probe, subtitle timing | Revise or send-back |
|
||||
| G7 | publish | Locale packaging, metadata | Revise |
|
||||
| FINAL | all | Translation quality, timing, lip-sync | Send-back |
|
||||
|
||||
## Execution Limits
|
||||
|
||||
| Limit | Value |
|
||||
|-------|-------|
|
||||
| Max revisions per stage | 3 |
|
||||
| Max send-backs per stage pair | 1 |
|
||||
| Max total send-backs | 3 |
|
||||
| Max total budget | Configurable (default $3 — localization is costlier) |
|
||||
| Max total wall-time | 15 minutes |
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- **Ignoring language expansion**: Some languages are 20-30% longer than English. The dubbed audio won't fit the original timing without adjustments.
|
||||
- **Lip-sync on every shot**: Only apply lip-sync to front-facing, clear-mouth shots. Side angles and distant shots don't need it.
|
||||
- **Translating protected terms**: Brand names, product names, and technical terms in the glossary must stay in the original language.
|
||||
- **Inconsistent locale labeling**: Use ISO language codes in filenames. "Spanish" is ambiguous (es-ES vs es-MX).
|
||||
- **Degrading source video**: Re-encoding the source video for each locale should preserve quality. Never downgrade resolution.
|
||||
@@ -0,0 +1,70 @@
|
||||
# Idea Director - Localization Dub Pipeline
|
||||
|
||||
## When To Use
|
||||
|
||||
Use this pipeline when the user has a source video and wants translated deliverables: subtitles, dubbed audio, or localized videos in one or more target languages.
|
||||
|
||||
Your first responsibility is to define what kind of localization is actually required, because subtitle-only, dubbed-audio, and lip-synced translation are different jobs.
|
||||
|
||||
## Reference Inputs
|
||||
|
||||
- `docs/localization-dubbing-best-practices.md`
|
||||
- `skills/creative/short-form.md`
|
||||
- `skills/creative/long-form.md`
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Define The Localization Scope
|
||||
|
||||
Capture:
|
||||
|
||||
- source language,
|
||||
- target languages,
|
||||
- review owner,
|
||||
- whether glossary or legal review is required,
|
||||
- whether the user needs subtitles, dubbed audio, lip-sync, or a mix.
|
||||
|
||||
### 2. Classify The Source
|
||||
|
||||
Record the source mode:
|
||||
|
||||
- `single_speaker`
|
||||
- `multi_speaker`
|
||||
- `voiceover_led`
|
||||
- `speaker_led_on_camera`
|
||||
|
||||
Also record whether on-screen text or motion graphics will need manual replacement or coverage.
|
||||
|
||||
### 3. Pick Deliverables That Match Reality
|
||||
|
||||
Possible deliverables:
|
||||
|
||||
- subtitle package only,
|
||||
- dubbed video without lip sync,
|
||||
- lip-synced localized video,
|
||||
- per-language export bundle.
|
||||
|
||||
### 4. Build The Brief
|
||||
|
||||
Recommended metadata keys:
|
||||
|
||||
- `source_language`
|
||||
- `target_languages`
|
||||
- `deliverable_mode_map`
|
||||
- `glossary_terms`
|
||||
- `protected_terms`
|
||||
- `review_requirements`
|
||||
- `timing_risks`
|
||||
|
||||
### 5. Quality Gate
|
||||
|
||||
- localization scope is explicit,
|
||||
- target outputs are realistic,
|
||||
- glossary and review requirements are captured,
|
||||
- risk increases from speaker count or visible mouths are surfaced.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Calling every translation request a dubbing request.
|
||||
- Ignoring glossary control until after audio is generated.
|
||||
- Promising lip sync on visually difficult source footage without warning.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Publish Director - Localization Dub Pipeline
|
||||
|
||||
## When To Use
|
||||
|
||||
Package the completed localization outputs so downstream teams can find the right video, subtitle, and metadata bundle for each language without manual cleanup.
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Package By Locale
|
||||
|
||||
Each language package should clearly separate:
|
||||
|
||||
- video output,
|
||||
- subtitle files,
|
||||
- transcript or approved script copy,
|
||||
- review notes,
|
||||
- metadata.
|
||||
|
||||
### 2. Keep Naming Precise
|
||||
|
||||
Recommended metadata keys:
|
||||
|
||||
- `locale`
|
||||
- `language_name`
|
||||
- `deliverable_mode`
|
||||
- `subtitle_included`
|
||||
- `review_owner`
|
||||
|
||||
### 3. Preserve Review Context
|
||||
|
||||
If a language output has pronunciation caveats, timing warnings, or missing lip sync, keep that note in the published package.
|
||||
|
||||
### 4. Quality Gate
|
||||
|
||||
- locale packages are clearly labeled,
|
||||
- metadata matches the actual treatment,
|
||||
- supporting text assets are present,
|
||||
- warnings and review notes are not lost.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Shipping localized videos without the matching subtitle or transcript files.
|
||||
- Mixing audio-dub and subtitle-only variants under the same generic filename.
|
||||
- Removing the QA notes that explain known issues.
|
||||
@@ -0,0 +1,65 @@
|
||||
# Scene Director - Localization Dub Pipeline
|
||||
|
||||
## When To Use
|
||||
|
||||
Plan how each localized deliverable will handle timing, visible speech, subtitles, and on-screen text. This is where the pipeline decides whether to preserve the original cut, cover mouth-visible sections, or attempt lip sync.
|
||||
|
||||
## Reference Inputs
|
||||
|
||||
- `docs/localization-dubbing-best-practices.md`
|
||||
- `skills/creative/video-editing.md`
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Choose The Dub Mode Per Deliverable
|
||||
|
||||
Use one of:
|
||||
|
||||
- `subtitle_only`
|
||||
- `dub_audio_only`
|
||||
- `lip_synced`
|
||||
- `hybrid_covered`
|
||||
|
||||
`hybrid_covered` means using B-roll, graphics, or text coverage during sections where visible mouth mismatch would be distracting.
|
||||
|
||||
### 2. Map Timing Risk
|
||||
|
||||
Identify scenes likely to drift because of:
|
||||
|
||||
- fast speech,
|
||||
- dense legal copy,
|
||||
- multiple speakers,
|
||||
- fast cuts,
|
||||
- visible close-up mouths.
|
||||
|
||||
### 3. Note On-Screen Language Dependencies
|
||||
|
||||
Record scenes that contain:
|
||||
|
||||
- UI text,
|
||||
- lower thirds,
|
||||
- title cards,
|
||||
- baked-in subtitles,
|
||||
- charts or labels that may need replacement or coverage.
|
||||
|
||||
### 4. Use Metadata For Variant Planning
|
||||
|
||||
Recommended metadata keys:
|
||||
|
||||
- `dub_mode_map`
|
||||
- `timing_risk_map`
|
||||
- `on_screen_text_replacement_map`
|
||||
- `language_variant_notes`
|
||||
|
||||
### 5. Quality Gate
|
||||
|
||||
- every deliverable has a defined localization treatment,
|
||||
- timing risks are mapped,
|
||||
- lip-sync usage is selective,
|
||||
- text replacement needs are not hidden.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Assuming dubbed audio will fit the source timing exactly.
|
||||
- Choosing lip sync for every shot instead of only the shots that justify it.
|
||||
- Forgetting about baked-in text until compose time.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Script Director - Localization Dub Pipeline
|
||||
|
||||
## When To Use
|
||||
|
||||
Turn the approved localization brief into a transcript-backed, reviewable script package for every target language. This stage should create text truth before any dubbing audio is generated.
|
||||
|
||||
## Reference Inputs
|
||||
|
||||
- `docs/localization-dubbing-best-practices.md`
|
||||
- `skills/creative/storytelling.md`
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Build Source Transcript Truth
|
||||
|
||||
Start with the source transcript and fix obvious errors in:
|
||||
|
||||
- names,
|
||||
- terminology,
|
||||
- speaker allocation,
|
||||
- numbers,
|
||||
- CTA phrasing.
|
||||
|
||||
### 2. Produce Reviewable Target Copy
|
||||
|
||||
For each target language, generate text that can be reviewed before synthesis. Record where terms should remain unchanged.
|
||||
|
||||
### 3. Preserve Structure Where Practical
|
||||
|
||||
Keep section timing and sequence aligned to the source unless the translation clearly needs a different pacing strategy.
|
||||
|
||||
### 4. Use Metadata For Localization Control
|
||||
|
||||
Recommended metadata keys:
|
||||
|
||||
- `source_transcript_status`
|
||||
- `target_language_sections`
|
||||
- `glossary_terms`
|
||||
- `protected_terms`
|
||||
- `pronunciation_notes`
|
||||
- `review_status_by_language`
|
||||
|
||||
### 5. Quality Gate
|
||||
|
||||
- the source transcript is strong enough to trust,
|
||||
- target-language copy exists for every planned deliverable,
|
||||
- glossary terms are preserved,
|
||||
- the script package can be reviewed before audio generation.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Generating audio from an unreviewed transcript.
|
||||
- Letting product names drift across languages.
|
||||
- Treating translation text as final timing without acknowledging length drift.
|
||||
Reference in New Issue
Block a user