Initial release — OpenMontage: the first open-source agentic video production system

11 production pipelines, 47 tools, 124 agent skills.
Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and
free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
calesthio
2026-03-29 08:25:17 -07:00
commit a3e735cc7a
1147 changed files with 240221 additions and 0 deletions
@@ -0,0 +1,101 @@
# Asset Director - Avatar Spokesperson Pipeline
## When To Use
This stage prepares the actual spokesperson ingredients: narration, avatar or lip-sync footage, subtitle assets, branded backgrounds, and the minimal support graphics needed to complete the cut.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Presenter plan and narration needs |
| Tools | `talking_head`, `lip_sync`, `tts_selector`, `subtitle_gen`, `image_selector`, `audio_enhance` — selectors auto-discover all available providers from the registry | Avatar, narration, and support asset options |
| Playbook | Active style playbook | Background, type, and subtitle rules |
## Process
### 1. Lock The Avatar Generation Path
Use one primary path and record it clearly:
- `talking_head` from still image plus audio,
- `lip_sync` from existing presenter plate plus new audio,
- externally supplied avatar render if created outside the current runtime.
Do not hide a blocked avatar path. Record it.
### 1b. Sample Preview (Prevents Wasted Spend)
Before batch-generating assets, produce one sample of each expensive type and show the user:
1. **TTS sample** (if generating narration): Generate one section. Confirm voice, pace, and persona before batching the rest.
2. **Avatar sample** (if using `talking_head`): Generate a short test clip. Confirm the avatar quality is acceptable before committing to full generation.
If rejected, adjust parameters and retry (max 3 iterations). Do not batch until approved.
### 2. Resolve Narration Before Support Graphics
Spokesperson videos depend on speech. Determine whether narration is:
- supplied,
- TTS-generated,
- already embedded in a presenter plate.
If narration is missing and no TTS tool is available, mark the project blocked instead of pretending the stage succeeded.
### 3. Build The Minimal Support Kit
Prepare only what the scene plan actually needs:
- subtitle files,
- one lower-third system,
- CTA card,
- background or plate assets,
- optional still or product support images.
### 4. Use Metadata For Capability Truth
Recommended metadata keys:
- `avatar_generation_path`
- `narration_assets`
- `subtitle_assets`
- `background_assets`
- `scene_asset_index`
- `blocked_assets`
### 5. Quality Gate
- the avatar path is explicit,
- narration and avatar assets align,
- support graphics stay minimal,
- every referenced file exists.
## No-Avatar Path
When the EP has triggered a narration-over-graphics pivot (neither `talking_head` nor `lip_sync` available), skip avatar generation entirely and produce a graphics-driven asset kit instead:
### What to produce:
1. **Narration audio** — via `tts_selector` (mandatory; block the project if no TTS is available either).
2. **Scene visuals** — via `image_selector` or `video_selector`. One primary visual per scene that reinforces the spoken point (diagram, illustration, product shot, or stock footage).
3. **Subtitle files** — same as standard path.
4. **Text cards** — key-point overlays, stat cards, CTA end card.
5. **Backgrounds** — consistent family matching the playbook.
### What to skip:
- No `talking_head` or `lip_sync` calls.
- No presenter framing metadata.
- `avatar_generation_path` should be set to `"none — narration-over-graphics pivot"`.
### Metadata for this path:
- `avatar_generation_path`: `"narration_over_graphics"`
- `pivot_reason`: why the no-avatar path was chosen
- All other metadata keys remain the same.
## Common Pitfalls
- Building decorative assets before the narration path is solved.
- Mixing multiple avatar-generation strategies in one simple spokesperson video.
- Marking the stage complete when the core presenter asset is still hypothetical.
- (No-avatar path) Generating filler visuals with no connection to the narration — every image must reinforce the spoken point.
@@ -0,0 +1,60 @@
# Compose Director - Avatar Spokesperson Pipeline
## When To Use
Render the final spokesperson outputs. The bar is simple: the presenter must look stable, speech must be clear, and subtitles or support cards must not crowd the frame.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | What to render |
| Tools | `video_compose`, `audio_mixer`, `video_stitch`, `audio_enhance` | Render and audio finishing |
| Playbook | Active style playbook | Typography and layout rules |
## Process
### 1. Render The Hero Cut First
Prefer one strong master before derivatives. Compose:
- presenter video,
- subtitles,
- lower-thirds,
- CTA cards,
- mixed narration.
### 2. Keep The Frame Clean
Subtitle and CTA placement matter more here than flashy transitions. Leave the face and mouth region unobstructed.
### 3. Verify Mouth Timing And Audio
If the avatar path used lip sync or audio-driven talking head, check:
- mouth timing,
- face artifacts,
- drift on long sections,
- audio clarity.
### 4. Verify Every Output
Record important findings in:
- `render_report.verification_notes`
- `render_report.warnings`
- `render_report.metadata.variant_notes`
### 5. Quality Gate
- the output file is valid,
- speech is clear,
- subtitles stay readable,
- the presenter remains visually stable.
## Common Pitfalls
- Letting subtitles cover the chin or mouth area.
- Shipping a long lip-sync render without spot-checking drift.
- Making derivative crops that cut off the presenter or CTA.
@@ -0,0 +1,47 @@
# Edit Director - Avatar Spokesperson Pipeline
## When To Use
Turn the planned presenter scenes and produced assets into a coherent spokesperson timeline. The quality bar is steady delivery, readable support layers, and a clear CTA landing.
## Process
### 1. Cut The Presenter Track First
Assemble the core spokesperson performance before adding support layers. If the presenter cut is weak, extra graphics will not rescue it.
### 2. Add Support Layers Sparingly
Use overlays only where they help:
- short proof points,
- product names,
- pricing or feature cards,
- CTA reinforcement,
- subtitles.
### 3. Respect Spoken Rhythm
Keep pauses where they help emphasis. Do not trim so tightly that the avatar feels rushed or robotic.
### 4. Plan Deliverables Clearly
Recommended metadata keys:
- `hero_cut_order`
- `cta_frame_range`
- `overlay_timing_map`
- `variant_decisions`
### 5. Quality Gate
- the presenter remains the anchor,
- overlays are timed cleanly,
- scene transitions are calm and intentional,
- the CTA lands once and clearly.
## Common Pitfalls
- Overcutting to simulate energy.
- Letting captions, side panels, and lower thirds compete for the same area.
- Ending without a clean CTA hold.
@@ -0,0 +1,171 @@
# Executive Producer — Avatar Spokesperson Pipeline
## When to Use
You are the **Executive Producer (EP)** for an avatar spokesperson video. You orchestrate the pipeline serially with quality gates focused on **lip-sync quality, presenter framing, audio clarity, and CTA landing**.
**No pre-production stages.** The project is script-driven with a digital presenter as the anchor. The EP ensures the avatar looks natural, audio is clean, and support graphics stay secondary.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/avatar-spokesperson.yaml` | Stage definitions |
| Skills | All 7 director skills + `meta/reviewer` | Stage execution |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
## Cumulative State
```
EP_STATE:
pipeline: avatar-spokesperson
playbook: <selected>
target_duration_seconds: <from brief>
budget_total_usd: <configured>
budget_spent_usd: 0.0
# Avatar-specific
avatar_path: null # heygen_api / sadtalker / musetalk / stock
narration_source: null # tts / provided_audio
cta_type: null # what the viewer should do after watching
presenter_framing: null # layout: center, left-third, etc.
artifacts:
idea: null
script: null
scene_plan: null
assets: null
edit: null
compose: null
publish: null
revision_counts: {}
issues_log: []
```
## Pivot Decision Matrix
`talking_head` is the preferred tool but commonly unavailable (requires GPU or HeyGen API key). When blocked, the EP must route the project explicitly — not improvise.
```
IF talking_head AVAILABLE:
→ Standard avatar path. Proceed as normal.
IF talking_head UNAVAILABLE and lip_sync AVAILABLE:
→ Lip-sync path. User must supply a presenter plate (existing footage).
Script and scene plan stay the same.
IF NEITHER talking_head NOR lip_sync AVAILABLE:
→ Narration-Over-Graphics pivot.
Tell the user: "No avatar tool is configured. I can produce a
narration-over-graphics video instead — your script and CTA stay the same,
but the presenter is replaced with styled visuals, text overlays, and
voice-over narration."
If the user approves:
- Switch scene-director to narration-over-graphics layout (see its fallback section).
- Switch asset-director to no-avatar path (see its fallback section).
- CTA and script are unchanged.
If the user declines:
- Mark the project blocked. Do not proceed with a half-working avatar.
```
**The pivot decision happens at G1 (after IDEA).** Do not wait until the ASSETS stage to discover the tool is missing.
## EP-Specific Cross-Stage Checks
### After IDEA stage:
```
CHECK: Avatar path feasibility
- Is the avatar generation path explicit (which tool)?
- Is the required tool available in the registry?
- If tool unavailable: run the Pivot Decision Matrix above
- Are CTA and audience appropriate for spokesperson format?
```
### After SCRIPT stage:
```
CHECK: Spoken copy quality
- Is the script concise and natural-sounding when read aloud?
- Are scene breaks realistic for avatar delivery (no mid-sentence cuts)?
- Is on-screen text restrained (presenter is the focus, not graphics)?
CHECK: Duration fit
- Word count aligns with natural speaking pace (~140-160 WPM for spokesperson)
```
### After SCENE_PLAN stage:
```
CHECK: Presenter layout
- Is the speaker layout consistent and coherent?
- Are support overlays secondary to the presenter?
- Are background changes minimal (max 2-3 distinct backgrounds)?
CHECK: Subtitle safety
- Is subtitle placement planned to avoid overlapping the presenter's face?
```
### After ASSETS stage:
```
CHECK: Avatar generation
- Did the avatar tool produce a usable video?
- Is lip-sync timing acceptable?
- Is narration audio clear and natural?
- Budget gate: 90% threshold warning
CHECK: Support asset restraint
- Are support graphics (backgrounds, overlays) minimal?
- Do they match the playbook style?
```
### After EDIT stage:
```
CHECK: Presenter primacy
- Is the presenter visually primary in every scene?
- Are graphics and captions reinforcing, not crowding?
- Does CTA land clearly (dedicated end section)?
CHECK: Timeline completeness
- All cuts reference valid assets
- Audio ducking if background music present
```
### After COMPOSE stage:
```
CHECK: Output validation
- ffprobe: duration, resolution, codec
- Lip-sync or mouth timing acceptable for the chosen path
- Subtitle placement clean and non-overlapping
- Audio clear and presenter-focused
- No uncanny-valley artifacts that break immersion
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | idea | Avatar path feasibility, CTA fit | Revise |
| G2 | script | Spoken copy quality, duration | Revise |
| G3 | scene_plan | Presenter layout, subtitle safety | Revise |
| G4 | assets | Avatar quality, lip-sync, budget | Revise |
| G5 | edit | Presenter primacy, CTA landing | Revise |
| G6 | compose | Lip-sync, subtitle placement, audio | Revise or send-back |
| G7 | publish | Metadata, presenter thumbnail | Revise |
| FINAL | all | Avatar naturalness, audio, CTA | Send-back |
## Execution Limits
| Limit | Value |
|-------|-------|
| Max revisions per stage | 3 |
| Max send-backs per stage pair | 1 |
| Max total send-backs | 3 |
| Max total budget | Configurable (default $2) |
| Max total wall-time | 12 minutes |
## Common Pitfalls
- **Uncanny valley**: If avatar quality is low, it undermines the entire video. Be honest about tool capabilities.
- **Graphics overload**: The presenter IS the content. Support graphics should be minimal.
- **Unnatural script**: Spokesperson scripts must sound conversational, not robotic or essay-like.
- **Ignoring CTA**: Every spokesperson video has a purpose. The CTA must land clearly.
@@ -0,0 +1,73 @@
# Idea Director - Avatar Spokesperson Pipeline
## When To Use
Use this pipeline when the deliverable is a presenter-led avatar video: a spokesperson spot, product intro, onboarding message, internal comms update, or short scripted explainer where the speaker remains the visual anchor.
Your first job is to classify the avatar path honestly before anyone writes polished copy for an impossible production setup.
## Reference Inputs
- `docs/avatar-spokesperson-best-practices.md`
- `skills/creative/storytelling.md`
- `skills/creative/short-form.md`
## Process
### 1. Classify The Avatar Path
Record which production mode the project actually has:
- `platform_avatar`
- `photo_talking_head`
- `presenter_plate_lip_sync`
Also record whether the avatar already exists or still has to be created outside the current run.
### 2. Define The Message Shape
Capture:
- audience,
- core offer or CTA,
- runtime target,
- platform targets,
- whether the video is sales, onboarding, support, or announcement led.
Spokesperson videos work best when they have one clear job.
### 3. Capture Source Reality
The brief should explicitly state:
- whether clean narration is supplied,
- whether TTS is acceptable,
- whether brand backgrounds or overlays exist,
- whether subtitles are required,
- whether multilingual variants are expected.
### 4. Build The Brief
Recommended metadata keys:
- `avatar_path`
- `avatar_exists`
- `narration_source`
- `target_audience`
- `cta_type`
- `background_strategy`
- `deliverable_mix`
- `missing_capabilities`
### 5. Quality Gate
- the avatar path is explicit,
- the message is narrow enough for a spokesperson format,
- missing narration or avatar dependencies are visible early,
- deliverables fit the actual source setup.
## Common Pitfalls
- Treating a generic generated-video request as a deterministic avatar workflow.
- Writing the CTA before confirming the avatar and narration path.
- Planning multiple aspect ratios before the hero layout is proven.
@@ -0,0 +1,44 @@
# Publish Director - Avatar Spokesperson Pipeline
## When To Use
Package the finished spokesperson outputs for delivery. This stage should make it obvious which file is the hero cut, which are derivatives, and what message or audience each version serves.
## Process
### 1. Label Deliverables Clearly
Distinguish:
- hero cut,
- vertical cutdown,
- square cutdown,
- language variants,
- watermark or review versions.
### 2. Keep Metadata Message-Led
Recommended metadata keys:
- `audience_segment`
- `cta_copy`
- `offer_name`
- `locale`
- `thumbnail_concept`
### 3. Package Review Notes
If the avatar path has limitations such as visible lip-sync risk, retain that note in the package instead of hiding it.
### 4. Quality Gate
- exports are clearly named,
- metadata matches the intended message,
- poster frame or thumbnail concept features the presenter cleanly,
- review notes stay attached to the package.
## Common Pitfalls
- Mixing hero and derivative exports without clear naming.
- Reusing generic metadata that ignores the spokesperson offer.
- Dropping risk notes that matter for downstream publishing teams.
@@ -0,0 +1,82 @@
# Scene Director - Avatar Spokesperson Pipeline
## When To Use
Plan the visual system for the presenter-led cut. This stage decides how the speaker sits in frame, what support layers appear, and which aspect ratios are actually worth building.
## Reference Inputs
- `docs/avatar-spokesperson-best-practices.md`
- `skills/creative/typography.md`
## Process
### 1. Lock The Presenter Layout
Choose a primary layout:
- full presenter center frame,
- presenter plus side panel,
- presenter plus lower-third system,
- presenter over branded background.
Keep the hero layout simple and reusable.
### 2. Choose Background Discipline
Use one family:
- clean solid or gradient background,
- branded office or set,
- product UI or image plate,
- transparent-avatar composite plan.
Background changes should mark real section changes, not compensate for weak copy.
### 3. Map Support Layers
For each scene, decide whether the support layer is:
- none,
- subtitle only,
- lower third,
- product image,
- side-panel proof point,
- CTA card.
### 4. Plan Variants Late
Default to one hero cut first. Only plan vertical or square variants if the presenter layout and text system can survive the crop.
### 5. Quality Gate
- the presenter remains primary,
- support layers solve specific communication needs,
- the background system is consistent,
- variant ambition stays realistic.
## Narration-Over-Graphics Fallback
When the EP triggers a no-avatar pivot (no `talking_head` or `lip_sync` available), redesign the visual system to carry the story with graphics and voice-over instead of a presenter:
### Layout changes:
- **Primary frame**: full-screen visual (background, image, diagram, or motion graphic) with voice-over narration underneath.
- **Support layers**: key-point text cards, lower-third stats, illustrated diagrams, product screenshots.
- **CTA scene**: dedicated end card with clear CTA text, no presenter needed.
### What stays the same:
- The script and scene breaks are unchanged — the narration drives pacing.
- Subtitle planning is unchanged.
- Background discipline still applies (consistent family, not random).
### What changes:
- No presenter framing decisions (center, left-third, etc.) — replaced with full-frame visual compositions.
- Each scene needs a **primary visual** that reinforces the spoken content, since there's no face to anchor attention.
- Plan more visual variety per scene — without a presenter, static backgrounds feel empty.
## Common Pitfalls
- Switching backgrounds every few seconds.
- Filling empty space with decorative panels.
- Assuming a landscape presenter layout will survive a vertical crop untouched.
- (Fallback mode) Producing a wall of text on screen to compensate for no presenter — let the narration carry the content.
@@ -0,0 +1,65 @@
# Script Director - Avatar Spokesperson Pipeline
## When To Use
Turn the approved brief into scene-safe spoken copy for an avatar presenter. The quality bar is not literary flourish. It is spoken clarity, believable pacing, and one clean point per scene.
## Reference Inputs
- `docs/avatar-spokesperson-best-practices.md`
- `skills/creative/storytelling.md`
## Process
### 1. Write For Speech, Not For Slides
Prefer:
- short sentences,
- direct verbs,
- one idea per beat,
- explicit transitions,
- conversational emphasis.
If the copy sounds like a brochure when read aloud, rewrite it.
### 2. Break Into Scene-Safe Chunks
Avatar scenes are easier to manage when each section is compact. A useful starting point is:
- hook,
- value statement,
- proof or feature beat,
- CTA.
### 3. Keep On-Screen Text Light
The presenter is already carrying attention. Use on-screen text only for:
- product names,
- short proof points,
- CTA copy,
- legal or compliance text that must appear.
### 4. Use Metadata For Delivery Notes
Recommended metadata keys:
- `scene_copy_map`
- `cta_language`
- `pronunciation_notes`
- `supplied_script_source`
- `legal_text_requirements`
### 5. Quality Gate
- the copy sounds spoken,
- scene lengths are realistic,
- CTA placement is clear,
- text overlays are restrained.
## Common Pitfalls
- Overstuffing one scene because the script reads well on paper.
- Duplicating the same sentence in speech and large text overlays.
- Writing humor or improvisational beats the avatar path cannot sell.