freedak/OpenMontage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial release — OpenMontage: the first open-source agentic video production system

Browse Source 
11 production pipelines, 47 tools, 124 agent skills.
Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and
free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
calesthio
2026-03-29 08:25:17 -07:00
commit a3e735cc7a
1147 changed files with 240221 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/INDEX.md
+309
View File
@@ -0,0 +1,309 @@
# OpenMontage â€" Skill Index
> For the full agent onboarding guide, see [`AGENT_GUIDE.md`](../AGENT_GUIDE.md) in the project root.
This file lists all available Layer 2 skills and documents the 3-layer knowledge architecture.
## Knowledge Architecture
```
Layer 1: tools/tool_registry.py "What tools exist and what they can do"
tools/base_tool.py Each tool declares: capabilities, tier, status,
dependencies, cost, and agent_skills[]
â†" agent_skills[] points to â†"
Layer 2: skills/ "How OpenMontage uses these tools"
Project-specific conventions: Pipeline integration, artifact mappings,
{core,creative,meta,pipelines}/ enhancement chain order, quality checklists
â†" references underlying tech in â†"
Layer 3: .agents/skills/ "How the technology itself works"
Generic API knowledge from Correct import paths, code patterns,
skills.sh (47 installed skills) constraints, parameters â€" tech-agnostic
```
**How the agent uses this:**
1. The orchestrator queries Layer 1 (`tool_registry.support_envelope()`) to see what's available
2. Each tool's `agent_skills[]` field names the Layer 3 skills it relies on
3. Layer 2 skills (this directory) teach the agent OpenMontage-specific conventions
4. Layer 3 skills (`.agents/skills/`) provide generic API knowledge, loaded on-demand
## Capability Families & Tool Discovery
Every tool declares a `capability` (what it does) and a `provider` (who/what powers it). The registry groups tools by capability so agents can discover all options for a given task.
### Selector / Provider Pattern
For capability families with multiple providers (TTS, video generation), the architecture uses:
- **Selector tool** (`tts_selector`, `video_selector`, `image_selector`) — routes to the best available provider based on requirements, API key availability, and cost. Selectors auto-discover providers from the registry. Agents should default to selectors when the user hasn't specified a provider.
- **Provider tools** — call a specific provider directly. Agents use these when the user explicitly requests a provider or when the selector's routing isn't appropriate.
### Capability Family Reference
**Do not maintain a hardcoded tool list.** The registry is the single source of truth. Query it at runtime:
```bash
python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.capability_catalog(), indent=2))"
```
Key capability families to look for in the output:
| Capability | Selector | Discovery |
|---|---|---|
| `tts` | `tts_selector` | Auto-discovers all `capability="tts"` tools |
| `video_generation` | `video_selector` | Auto-discovers all `capability="video_generation"` tools |
| `image_generation` | `image_selector` | Auto-discovers all `capability="image_generation"` tools |
| `audio_processing` | — | FFmpeg-based local tools |
| `enhancement` | — | Mixed providers |
| `analysis` | — | Mixed providers |
| `graphics` | — | Local rendering tools |
| `music_generation` | — | Single-provider |
| `subtitle` | — | Pure Python |
| `avatar` | — | Local GPU models |
| `video_post` | — | FFmpeg-based local tools |
### Adding New Tools
1. Place the tool in the correct capability folder (or create a new one under `tools/`)
2. Set `capability` and `provider` in the class definition
3. If joining a multi-provider family, the existing selector discovers it automatically
4. Attach relevant Layer 2 and Layer 3 skills via `agent_skills[]`
5. The registry discovers tools automatically — no manual registration needed
6. **No other files need updating** — selectors, manifests, and instructions all derive from the registry
## Core Skills
| Skill | File | Trigger | Agent Skills (Layer 3) |
|-------|------|---------|----------------------|
| FFmpeg | `core/ffmpeg.md` | Video encoding, filtering, composition | `ffmpeg`, `video_toolkit` |
| Remotion | `core/remotion.md` | React-based composition, Phase 3+ | `remotion-best-practices`, `remotion` |
| WhisperX | `core/whisperx.md` | Transcription with word-level timestamps | `speech-to-text` |
| Subtitle Sync | `core/subtitle-sync.md` | Subtitle timing and alignment | `remotion-best-practices` |
| Color Grading | `core/color-grading.md` | FFmpeg color profiles, LUT workflow, accessibility | `ffmpeg` |
## Creative Skills
| Skill | File | Trigger | Agent Skills (Layer 3) |
|-------|------|---------|----------------------|
| Video Editing | `creative/video-editing.md` | Cut decisions, pacing, rhythm | `ffmpeg`, `video_toolkit` |
| Enhancement Strategy | `creative/enhancement-strategy.md` | Overlay placement and density | `ffmpeg` |
| Data Visualization | `creative/data-visualization.md` | Chart type selection, animation, label placement | `d3-viz`, `remotion-best-practices` |
| Video Stitching | `creative/video-stitching.md` | Multi-clip assembly, AI clip chaining, spatial composition | `ffmpeg`, `video_toolkit` |
| Video Gen Prompting | `creative/video-gen-prompting.md` | Universal video generation prompt vocabulary | `ai-video-gen`, `ltx2`, `create-video` |
| ↳ Sora Prompting | `creative/prompting/sora-prompting.md` | Sora 2 structured template, advanced fields | `ai-video-gen` |
| ↳ VEO Prompting | `creative/prompting/veo-prompting.md` | VEO 3.1 14-component structure, art movements | `ai-video-gen` |
| ↳ LTX Prompting | `creative/prompting/ltx-prompting.md` | LTX-2 6-element structure, audio prompting | `ltx2` |
| ↳ HunyuanVideo Prompting | `creative/prompting/hunyuan-prompting.md` | HunyuanVideo formula, I2V best practices | â€" |
| Storytelling | `creative/storytelling.md` | Narrative structure, hooks, pacing, Mayer's principles | â€" |
| Sound Design | `creative/sound-design.md` | Audio ducking, LUFS targets, SFX timing, AI TTS mixing | `elevenlabs` |
| Typography | `creative/typography.md` | Font selection, text sizing, safe zones, caption styling | â€" |
| ManimCE Usage | `creative/manim-usage.md` | Scene composition, animation timing, color usage | `manimce-best-practices` |
| Image Gen Usage | `creative/image-gen-usage.md` | Prompt consistency, hero reference, batch strategy | `flux-best-practices`, `bfl-api` |
| Image Provider Usage | `creative/image-provider-usage.md` | Provider selection (FLUX/OpenAI/Recraft/stock), cost-quality tradeoffs | `flux-best-practices`, `bfl-api` |
| B-Roll Planning | `creative/broll-planning.md` | Stock vs. generated decision, query construction, footage evaluation | — |
| Stock Sourcing Usage | `creative/stock-sourcing-usage.md` | Pexels/Pixabay usage, parameters, licensing, integration | — |
| Scene Detect Usage | `creative/scene-detect-usage.md` | Threshold tuning, algorithm selection, content presets | â€" |
| Diagram Gen Usage | `creative/diagram-gen-usage.md` | Complexity limits, progressive building, themes | `beautiful-mermaid` |
| Music Gen Usage | `creative/music-gen-usage.md` | BPM selection, prompt engineering, duration matching | `music`, `elevenlabs` |
| Background Removal | `creative/bg-remove-usage.md` | Model selection, alpha matting, compositing workflows | â€" |
| Upscaling | `creative/upscale-usage.md` | Scale factor, model selection, face-aware upscaling | â€" |
| Face Restoration | `creative/face-restore-usage.md` | CodeFormer/GFPGAN selection, fidelity tuning, vs face_enhance | â€" |
| Lip Sync | `creative/lip-sync-usage.md` | Wav2Lip model selection, dubbing workflows, input requirements | `faceswap` |
| Talking Head Gen | `creative/talking-head-gen-usage.md` | SadTalker/MuseTalk, photo-to-video, expression tuning | `avatar-video` |
| Video Understanding | `creative/video-understand-usage.md` | Visual QA, quality gating, scene classification | `video-understand` |
## Pipeline Type Skills
Pipeline type skills provide production guidance for specific video formats, independent of the animated-explainer or talking-head pipeline.
| Skill | File | When to Use |
|-------|------|-------------|
| Short-Form | `creative/short-form.md` | TikTok, Reels, Shorts â€" vertical 9:16, under 60s |
| Long-Form | `creative/long-form.md` | YouTube 10+ min â€" chapters, retention, end screens |
| Screen Recording | `creative/screen-recording.md` | Code walkthroughs, tutorials, software demos |
| Animation Pipeline | `creative/animation-pipeline.md` | Motion graphics, easing, transitions, composition |
| Cinematic | `creative/cinematic.md` | Letterbox, film pacing, layered audio, color grading |
## Pipeline Stage Director Skills
Stage director skills teach the agent HOW to execute each pipeline stage. Each skill is a detailed markdown file with process steps, quality rubrics, and self-evaluation criteria.
### Animated Explainer Pipeline (`pipelines/explainer/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/explainer/executive-producer.md` | `all` | **8-stage serial orchestration, quality gates, cross-stage checks, send-back** |
| **Research Director** | `pipelines/explainer/research-director.md` | `research` | **Web research methodology, 5 search batches, landscape/trending/data/audience/expert analysis** |
| **Proposal Director** | `pipelines/explainer/proposal-director.md` | `proposal` | **Concept options from research, production plan, cost estimate, approval gate** |
| Script Director | `pipelines/explainer/script-director.md` | `script` | Narrative architecture, timing, enhancement cues, research integration |
| Scene Director | `pipelines/explainer/scene-director.md` | `scene_plan` | Visual planning, technique library, feasibility |
| Asset Director | `pipelines/explainer/asset-director.md` | `assets` | TTS, image gen, diagram gen, music, budget |
| Edit Director | `pipelines/explainer/edit-director.md` | `edit` | Timeline assembly, subtitles, audio ducking |
| Compose Director | `pipelines/explainer/compose-director.md` | `compose` | FFmpeg/Remotion render, audio mixing |
| Publish Director | `pipelines/explainer/publish-director.md` | `publish` | SEO metadata, chapters, export packaging |
> **Note:** The old `idea-director.md` still exists for reference but is superseded by the research + proposal two-stage flow in v2.0. The talking-head pipeline continues to use its own `idea-director`.
### Talking Head Pipeline (`pipelines/talking-head/`)
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| Idea Director | `pipelines/talking-head/idea-director.md` | `idea` | Footage inspection, content assessment |
| Script Director | `pipelines/talking-head/script-director.md` | `script` | Transcription, section segmentation |
| Scene Director | `pipelines/talking-head/scene-director.md` | `scene_plan` | Enhancement planning, overlay placement |
| Asset Director | `pipelines/talking-head/asset-director.md` | `assets` | Subtitle gen, audio extraction |
| Edit Director | `pipelines/talking-head/edit-director.md` | `edit` | Cut assembly, subtitle config |
| Compose Director | `pipelines/talking-head/compose-director.md` | `compose` | Enhancement chain, render |
| Publish Director | `pipelines/talking-head/publish-director.md` | `publish` | Metadata, export packaging |
### Screen Demo Pipeline (`pipelines/screen-demo/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/screen-demo/executive-producer.md` | `all` | **7-stage serial orchestration, legibility gates, audio clarity, pacing checks** |
| Idea Director | `pipelines/screen-demo/idea-director.md` | `idea` | Workflow scoping, UI density assessment, output-shape choice |
| Script Director | `pipelines/screen-demo/script-director.md` | `script` | Action mapping, procedural narration, speed planning |
| Scene Director | `pipelines/screen-demo/scene-director.md` | `scene_plan` | Crop planning, callout restraint, aspect-ratio viability |
| Asset Director | `pipelines/screen-demo/asset-director.md` | `assets` | Subtitle-first asset kit, audio cleanup, reusable overlays |
| Edit Director | `pipelines/screen-demo/edit-director.md` | `edit` | Tight timeline planning, speed notes, subtitle zone control |
| Compose Director | `pipelines/screen-demo/compose-director.md` | `compose` | Legibility-first render, crisp screen output, verification |
| Publish Director | `pipelines/screen-demo/publish-director.md` | `publish` | Searchable metadata, chapter packaging, thumbnail concepts |
### Clip Factory Pipeline (`pipelines/clip-factory/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/clip-factory/executive-producer.md` | `all` | **7-stage serial orchestration, clip selection gates, batch consistency, hook placement** |
| Idea Director | `pipelines/clip-factory/idea-director.md` | `idea` | Batch strategy, clip families, yield planning |
| Script Director | `pipelines/clip-factory/script-director.md` | `script` | Transcript mining, ranking, standalone validation |
| Scene Director | `pipelines/clip-factory/scene-director.md` | `scene_plan` | Platform framing, safe zones, crop-viability planning |
| Asset Director | `pipelines/clip-factory/asset-director.md` | `assets` | Shared brand kit, rebased subtitles, batch audio consistency |
| Edit Director | `pipelines/clip-factory/edit-director.md` | `edit` | Hook-first mini-edits, series consistency |
| Compose Director | `pipelines/clip-factory/compose-director.md` | `compose` | Multi-job rendering, batch resilience, per-output verification |
| Publish Director | `pipelines/clip-factory/publish-director.md` | `publish` | Posting order, platform copy, batch cataloging |
### Podcast Repurpose Pipeline (`pipelines/podcast-repurpose/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/podcast-repurpose/executive-producer.md` | `all` | **7-stage serial orchestration, audio preservation gates, clip quality, multi-deliverable** |
| Idea Director | `pipelines/podcast-repurpose/idea-director.md` | `idea` | Deliverable mix by source mode, realistic long-form planning |
| Script Director | `pipelines/podcast-repurpose/script-director.md` | `script` | Diarized transcript truth, highlight ranking, chapter mapping |
| Scene Director | `pipelines/podcast-repurpose/scene-director.md` | `scene_plan` | Source-faithful treatments, audiogram vs quote vs companion planning |
| Asset Director | `pipelines/podcast-repurpose/asset-director.md` | `assets` | Subtitle-first packaging, speaker assets, optional topic art |
| Edit Director | `pipelines/podcast-repurpose/edit-director.md` | `edit` | Hook-led podcast clips, quote hold time, companion simplicity |
| Compose Director | `pipelines/podcast-repurpose/compose-director.md` | `compose` | Audio-first rendering, deliverable prioritization |
| Publish Director | `pipelines/podcast-repurpose/publish-director.md` | `publish` | Episode cross-linking, guest attribution, staggered release logic |
### Cinematic Pipeline (`pipelines/cinematic/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/cinematic/executive-producer.md` | `all` | **7-stage serial orchestration, emotional pacing gates, color consistency, audio dynamics** |
| Idea Director | `pipelines/cinematic/idea-director.md` | `idea` | Emotional arc selection, source truth, delivery-shape planning |
| Script Director | `pipelines/cinematic/script-director.md` | `script` | Beat mapping, dialogue selects, title-card restraint |
| Scene Director | `pipelines/cinematic/scene-director.md` | `scene_plan` | Hero-frame planning, reveal structure, transition limits |
| Asset Director | `pipelines/cinematic/asset-director.md` | `assets` | Source selects, support-insert discipline, music/ambience planning |
| Edit Director | `pipelines/cinematic/edit-director.md` | `edit` | Emotion-first pacing, reveal timing, audio-driven rhythm |
| Compose Director | `pipelines/cinematic/compose-director.md` | `compose` | Grade and mix finishing, frame-treatment judgment |
| Publish Director | `pipelines/cinematic/publish-director.md` | `publish` | Hero vs teaser packaging, poster-frame concepts |
### Animation Pipeline (`pipelines/animation/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/animation/executive-producer.md` | `all` | **8-stage serial orchestration, quality gates, motion consistency, math accuracy checks** |
| **Research Director** | `pipelines/animation/research-director.md` | `research` | **Topic + animation technique research, visual reference scan, mode-informed angles** |
| **Proposal Director** | `pipelines/animation/proposal-director.md` | `proposal` | **Animation mode selection (Manim/Remotion/AI/diagram), reuse strategy, cost estimate, approval gate** |
| Script Director | `pipelines/animation/script-director.md` | `script` | Animation-ready beats, text restraint, research integration, mode-aware writing |
| Scene Director | `pipelines/animation/scene-director.md` | `scene_plan` | Animatic planning, transition systems, tool-path mapping |
| Asset Director | `pipelines/animation/asset-director.md` | `assets` | Deterministic asset choice, reusable motifs, feasibility truth |
| Edit Director | `pipelines/animation/edit-director.md` | `edit` | Hold timing, stagger rules, readable motion planning |
| Compose Director | `pipelines/animation/compose-director.md` | `compose` | Sharp render output, timing integrity, safe-zone checks |
| Publish Director | `pipelines/animation/publish-director.md` | `publish` | Animation-mode packaging, thumbnail-system alignment |
> **Note:** The old `idea-director.md` still exists for reference but is superseded by the research + proposal two-stage flow in v2.0.
### Hybrid Pipeline (`pipelines/hybrid/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/hybrid/executive-producer.md` | `all` | **7-stage serial orchestration, source/support balance gates, overlay density, coherence** |
| Idea Director | `pipelines/hybrid/idea-director.md` | `idea` | Anchor-medium selection, support-layer planning, fallback visibility |
| Script Director | `pipelines/hybrid/script-director.md` | `script` | Source-vs-support beat mapping, dialogue retention, support justification |
| Scene Director | `pipelines/hybrid/scene-director.md` | `scene_plan` | Source-primary layout rules, overlay density control, variant-safe planning |
| Asset Director | `pipelines/hybrid/asset-director.md` | `assets` | Shared support kits, source-vs-generated asset tracking |
| Edit Director | `pipelines/hybrid/edit-director.md` | `edit` | Anchor-cut-first workflow, layered support timing, readable variants |
| Compose Director | `pipelines/hybrid/compose-director.md` | `compose` | Source/support balance checks, variant verification, coherent mix |
| Publish Director | `pipelines/hybrid/publish-director.md` | `publish` | Master-vs-derivative packaging, source-mix metadata |
### Avatar Spokesperson Pipeline (`pipelines/avatar-spokesperson/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/avatar-spokesperson/executive-producer.md` | `all` | **7-stage serial orchestration, lip-sync quality gates, presenter framing, CTA landing** |
| Idea Director | `pipelines/avatar-spokesperson/idea-director.md` | `idea` | Avatar-path classification, CTA scoping, capability truth |
| Script Director | `pipelines/avatar-spokesperson/script-director.md` | `script` | Spoken-copy shaping, scene-safe pacing, text restraint |
| Scene Director | `pipelines/avatar-spokesperson/scene-director.md` | `scene_plan` | Presenter layout, background discipline, variant realism |
| Asset Director | `pipelines/avatar-spokesperson/asset-director.md` | `assets` | Avatar-path locking, narration resolution, minimal support kits |
| Edit Director | `pipelines/avatar-spokesperson/edit-director.md` | `edit` | Presenter-first cut planning, overlay timing, CTA landing |
| Compose Director | `pipelines/avatar-spokesperson/compose-director.md` | `compose` | Lip-sync verification, subtitle-safe framing, clean render checks |
| Publish Director | `pipelines/avatar-spokesperson/publish-director.md` | `publish` | Audience-led packaging, presenter-first thumbnail concepts |
### Localization Dub Pipeline (`pipelines/localization-dub/`) — v2.0
| Skill | File | Stage | Key Capabilities |
|-------|------|-------|-----------------|
| **Executive Producer** | `pipelines/localization-dub/executive-producer.md` | `all` | **7-stage serial orchestration, translation accuracy gates, timing preservation, per-locale QA** |
| Idea Director | `pipelines/localization-dub/idea-director.md` | `idea` | Scope definition, locale planning, glossary and review capture |
| Script Director | `pipelines/localization-dub/script-director.md` | `script` | Transcript truth, translated script packaging, term preservation |
| Scene Director | `pipelines/localization-dub/scene-director.md` | `scene_plan` | Dub-mode selection, timing-risk mapping, on-screen text planning |
| Asset Director | `pipelines/localization-dub/asset-director.md` | `assets` | Subtitle-first localization kit, dubbed audio generation, optional lip sync |
| Edit Director | `pipelines/localization-dub/edit-director.md` | `edit` | Locale-specific timelines, coverage planning, timing adjustments |
| Compose Director | `pipelines/localization-dub/compose-director.md` | `compose` | Per-locale rendering, subtitle-fit checks, output labeling |
| Publish Director | `pipelines/localization-dub/publish-director.md` | `publish` | Locale packaging, metadata precision, QA-note retention |
## Meta Skills
Cross-cutting skills that apply to all pipelines:
| Skill | File | Purpose |
|-------|------|---------|
| Reviewer | `meta/reviewer.md` | Self-review protocol after every stage |
| Checkpoint Protocol | `meta/checkpoint-protocol.md` | When/how to checkpoint and request human approval |
| Skill Creator | `meta/skill-creator.md` | Dynamically create new skills during pipeline runs |
## Style Playbooks
Style playbooks (`styles/*.yaml`) define visual language, typography, motion, audio, and asset generation constraints. They are validated against `schemas/styles/playbook.schema.json`.
| Playbook | Category | Mood | Best For |
|----------|----------|------|----------|
| `clean-professional` | motion-graphics | polished, trustworthy | Corporate, educational, SaaS |
| `flat-motion-graphics` | motion-graphics | energetic, bold | Social media, TikTok, startups |
| `minimalist-diagram` | whiteboard | focused, technical | Technical deep-dives, architecture |
Load via `styles/playbook_loader.py`: `load_playbook("clean-professional")`
## Installed Agent Skills (Layer 3)
All agent skills live in `.agents/skills/` and are managed via `npx skills add`.
Claude Code accesses them via symlinks in `.claude/skills/`.
| Category | Installed Skills | Source |
|----------|-----------------|--------|
| **Video Composition** | `remotion-best-practices`, `remotion` | `remotion-dev/skills`, `digitalsamba/claude-code-video-toolkit` |
| **Video Processing** | `ffmpeg`, `video_toolkit` | `digitalsamba/claude-code-video-toolkit` |
| **TTS & Audio** | `text-to-speech`, `speech-to-text`, `music`, `sound-effects`, `elevenlabs`, `agents`, `setup-api-key` | `elevenlabs/skills`, `digitalsamba/claude-code-video-toolkit` |
| **Image Generation** | `flux-best-practices`, `bfl-api` | `black-forest-labs/skills` |
| **Math Animation** | `manimce-best-practices`, `manimgl-best-practices`, `manim-composer` | `adithya-s-k/manim_skill` |
| **3D Graphics** | `threejs-animation`, `threejs-fundamentals`, `threejs-geometry`, `threejs-interaction`, `threejs-lighting`, `threejs-loaders`, `threejs-materials`, `threejs-postprocessing`, `threejs-shaders`, `threejs-textures` | `cloudai-x/threejs-skills` |
| **Diagrams** | `beautiful-mermaid`, `d3-viz` | `intellectronica/agent-skills`, `davila7/claude-code-templates` |
| **Animation** | `framer-motion`, `lottie-bodymovin` | `pproenca/dot-skills`, `dylantarre/animation-principles` |
| **Design** | `tailwind-design-system`, `web-design-guidelines`, `vercel-react-best-practices`, `vercel-composition-patterns` | `wshobson/agents`, `vercel-labs/agent-skills` |
| **AI Video (HeyGen)** | `heygen`, `avatar-video`, `create-video`, `faceswap`, `ai-video-gen`, `video-download`, `video-edit`, `video-translate`, `video-understand`, `visual-style` | `heygen-com/skills` |
| **Infrastructure** | `acestep`, `ltx2`, `playwright-recording` | `digitalsamba/claude-code-video-toolkit` |
skills/core/color-grading.md
+161
View File
@@ -0,0 +1,161 @@
# Color Grading for Video Production
> Sources: BBC Academy color standards, Filmmaker IQ color science series, DaVinci Resolve
> color theory (Blackmagic documentation), WCAG 2.1 contrast guidelines, FFmpeg filter
> documentation, Adobe color accessibility research, Wong (2011) colorblind-safe palette
## Quick Reference Card
```
PROFILES: cinematic_warm | cinematic_cool | moody_dark | bright_clean | vintage_film | high_contrast | neutral
LUT FORMAT: .cube (3D LUT) — industry standard, FFmpeg lut3d filter
INTENSITY: 0.6-0.85 for subtle grades, 1.0 for full effect
SKIN TONE: Vectorscope should fall on the "skin tone line" (~123° on I-line)
COLOR SPACE: BT.709 for web delivery, BT.2020 for HDR only
BIT DEPTH: Grade in 10-bit when possible, deliver in 8-bit for web
```
## FFmpeg Filter Reference
The `color_grade` tool uses these FFmpeg filters. Understanding them helps you craft `custom_vf` chains.
### Core Filters
| Filter | Purpose | Key Parameters |
|--------|---------|----------------|
| `eq` | Brightness, contrast, saturation, gamma | `contrast=1.0:saturation=1.0:brightness=0.0:gamma=1.0` |
| `colorbalance` | RGB adjustments in shadows/mids/highlights | `rs/gs/bs` (shadows), `rm/gm/bm` (mids), `rh/gh/bh` (highlights) — range -1.0 to 1.0 |
| `curves` | Tone curves per channel | `all='0/0 0.5/0.5 1/1'` or per-channel `red=`, `green=`, `blue=` |
| `colortemperature` | White balance shift | `temperature=6500` (neutral) — lower = cooler, higher = warmer |
| `lut3d` | Apply external .cube LUT | `lut3d='path/to/file.cube'` |
| `hue` | Hue rotation and saturation | `h=0:s=1` — h in degrees, s as multiplier |
| `normalize` | Auto-stretch histogram to full range | `blackpt=black:whitept=white:smoothing=0` |
### Filter Chain Order
Apply filters in this order for predictable results:
```
1. normalize (auto-levels if source is flat/log)
2. colortemperature (white balance correction)
3. colorbalance (shadow/mid/highlight color shifts)
4. curves (contrast and tone shaping)
5. eq (final contrast/saturation/brightness tweak)
6. lut3d (creative LUT — applied LAST, on corrected footage)
```
## Profile Selection by Content Type
| Content Type | Recommended Profile | Intensity | Why |
|-------------|-------------------|-----------|-----|
| Corporate / SaaS explainer | `bright_clean` | 0.8 | Clean, professional, approachable |
| Science / educational | `neutral` | 1.0 | Accurate color representation matters |
| Storytelling / narrative | `cinematic_warm` | 0.85 | Warmth builds emotional connection |
| Tech / dark theme | `cinematic_cool` | 0.7 | Complements dark UI screenshots |
| Drama / serious topic | `moody_dark` | 0.6-0.7 | Atmosphere without crushing detail |
| Lifestyle / social media | `high_contrast` | 0.8 | Punchy, attention-grabbing on mobile |
| Retro / nostalgic | `vintage_film` | 0.7 | Subtle faded look, not overdone |
## Mood-Specific Parameter Recipes
When the built-in profiles don't match, use these as starting points for `custom_vf`:
### Warm / Inviting
```
colorbalance=rs=0.06:gs=0.02:bs=-0.04:rh=0.05:gh=0.01:bh=-0.03,
eq=contrast=1.05:saturation=1.08:brightness=0.01
```
### Cool / Technical
```
colorbalance=rs=-0.03:gs=-0.01:bs=0.06:rh=-0.02:gh=0.01:bh=0.04,
eq=contrast=1.06:saturation=0.95
```
### High Energy
```
curves=all='0/0 0.15/0.08 0.5/0.52 0.85/0.92 1/1',
eq=contrast=1.15:saturation=1.2
```
### Subdued / Serious
```
curves=all='0/0.04 0.25/0.22 0.5/0.47 0.75/0.73 1/0.94',
eq=contrast=1.03:saturation=0.75:brightness=-0.02
```
## LUT Workflow
### When to Use LUTs
- Matching footage across different cameras/sources
- Applying a specific film stock emulation
- Maintaining brand consistency across multiple videos
- Converting from LOG/flat camera profiles to display color
### LUT Application Best Practices
1. **Always correct before grading** — normalize/white-balance the footage first, then apply creative LUT
2. **Use intensity < 1.0** — a LUT at full strength usually looks overdone; 0.6-0.8 is typical
3. **Test on skin tones first** — if people appear in the video, skin must look natural
4. **One LUT per project** — switching LUTs between scenes creates visual inconsistency
5. **LUT file location** — store in `assets/luts/` relative to the project, reference with `lut_path`
### FFmpeg LUT Application
```bash
# Apply LUT at 70% intensity (blend with original)
ffmpeg -i input.mp4 -vf "split[a][b];[b]lut3d='my_lut.cube'[graded];[a][graded]blend=all_mode=normal:all_opacity=0.7" output.mp4
```
## Skin Tone Protection
Skin tones are the most critical element in color grading — viewers instantly notice unnatural skin.
**The Skin Tone Line:**
- On a vectorscope, healthy skin (all ethnicities) falls on a narrow line at approximately 123 degrees (between red and yellow)
- If your grade pushes skin away from this line, reduce saturation or adjust hue
**Rules:**
- Never push saturation above 1.2 on footage with people
- After grading, check a frame with visible skin — if it looks orange, green, or magenta, pull back
- The `cinematic_warm` profile at intensity 0.85 is pre-tuned to keep skin natural
- For `moody_dark`, keep intensity at 0.6-0.7 to avoid making skin look grey
## Accessibility
### Colorblind-Safe Design (Wong Palette)
When generating graphics, overlays, or diagrams that accompany graded video, use this palette verified safe for all common types of color vision deficiency:
| Color | Hex | Use For |
|-------|-----|---------|
| Black | `#000000` | Text, outlines |
| Orange | `#E69F00` | Primary accent |
| Sky Blue | `#56B4E9` | Secondary accent |
| Bluish Green | `#009E73` | Positive/success |
| Yellow | `#F0E442` | Highlight/warning |
| Blue | `#0072B2` | Links, info |
| Vermillion | `#D55E00` | Error/danger |
| Reddish Purple | `#CC79A7` | Tertiary accent |
### WCAG Contrast Requirements
| Element | Minimum Ratio | Standard |
|---------|--------------|----------|
| Body text on background | 4.5:1 | WCAG AA |
| Large text (>18pt) on background | 3:1 | WCAG AA |
| Body text (enhanced) | 7:1 | WCAG AAA |
| UI components / graphical objects | 3:1 | WCAG 2.1 |
**Practical rule:** After color grading, any text overlays or subtitles burned into the video must still meet 4.5:1 contrast against the graded background. Test with a contrast checker on a representative frame.
## Applying to OpenMontage
When using the `color_grade` tool:
1. **Select profile by content type** using the table above — don't default to `cinematic_warm` for everything
2. **Set intensity to 0.8** as a starting point, not 1.0 — subtlety reads better on mobile screens
3. **Test on a single frame first** before grading the full video — saves render time
4. **Grade after face enhancement** — the enhancement chain order in `skills/creative/enhancement-strategy.md` is: subtitle → face → color → audio → final
5. **Use the same profile across all clips in a video** — visual consistency is critical
6. **For generated visuals** (image_selector, math_animate), apply a lighter grade (0.5-0.6) since they're already stylized
7. **Use the Wong palette** for any generated graphics (diagrams, code snippets, overlays) to ensure colorblind accessibility
8. **For custom grades**, follow the filter chain order above and keep parameter changes small — ±0.05 per adjustment, then review
skills/core/ffmpeg.md
+91
View File
@@ -0,0 +1,91 @@
# FFmpeg Skill
## When to Use
Use FFmpeg-backed tools for any video/audio processing that does not require AI inference:
cutting, trimming, speed adjustment, concatenation, audio extraction, mixing,
subtitle burn-in, overlay compositing, encoding, face enhancement, color grading,
and audio cleanup.
## Tools That Use FFmpeg
### Core Pipeline
| Tool | Capability |
|------|-----------|
| `video_trimmer` | Cut, trim, speed adjust, concat video segments |
| `video_compose` | Full composition: cuts + subtitles + overlays + encode |
| `audio_mixer` | Mix speech/music/SFX, ducking, fades, extract audio |
| `frame_sampler` | Extract representative frames from video |
### Enhancement Layer
| Tool | Capability | Key Presets |
|------|-----------|-------------|
| `face_enhance` | Skin smoothing, sharpening, warm/cool tones | `talking_head_standard`, `soft_skin`, `sharpen` |
| `color_grade` | Cinematic color grading with intensity control | `cinematic_warm`, `cinematic_cool`, `moody_dark` |
| `audio_enhance` | Noise reduction, loudness normalization, EQ | `clean_speech`, `voice_clarity`, `podcast` |
## Key Patterns
### Enhancement Chain Order
Apply enhancements in this order to avoid filter interactions:
1. **Subtitles first** — burn into the base video
2. **Face enhance** — smoothing/sharpening works best on ungraded footage
3. **Color grade** — applies look after face is already enhanced
4. **Audio enhance** — independent of video, apply last
Each step is optional and gracefully skipped if the tool is unavailable.
### Lossless vs Re-encode
- Use `-c copy` (codec copy) when you only need to cut or concat without altering frames. This is instant and lossless.
- Re-encode (`-c:v libx264`) when applying filters (speed change, subtitles, overlays, scaling).
- Default CRF is 23. Use 18-20 for higher quality when the output is the final deliverable.
### Subtitle Burn-in
- Prefer SRT format for simple word subtitles.
- Use `force_style` with full ASS color format: `&H00FFFFFF` (not `&HFFFFFF`).
- Always escape Windows paths (`C\:` not `C:`) in the subtitles filter.
- Vertical video: `font_size: 18`, `max 3 words/cue`, `margin_v: 50`.
- Horizontal video: `font_size: 22`, `max 6 words/cue`, `margin_v: 40`.
### Audio Enhancement Targets
| Platform | Target LUFS | Loudness Range |
|----------|-------------|----------------|
| Social media (TikTok, Reels) | -14 LUFS | 5-7 LU |
| YouTube | -14 to -16 LUFS | 7-11 LU |
| Podcast | -16 LUFS | 7-11 LU |
| Broadcast | -24 LUFS | 7 LU |
The `clean_speech` preset targets -16 LUFS with 11 LU range — good for YouTube/social media.
### Color Grade Intensity
- `intensity: 0.85` — recommended default for cinematic_warm on talking heads
- `intensity: 0.5` — subtle, barely noticeable
- `intensity: 1.0` — full effect, may look over-processed on some footage
### Audio Ducking
- Use `sidechaincompress` with speech as the key signal to lower music volume during dialogue.
- Typical settings: threshold=0.02, ratio=9, attack=200ms, release=500ms.
### Concatenation
- Use the concat demuxer (`-f concat -safe 0`) for same-codec segments.
- For mixed codecs or different resolutions, re-encode all segments first.
## Quality Checklist
- [ ] Output plays without artifacts on desktop and mobile
- [ ] Audio and video remain in sync after processing
- [ ] Subtitles are in the bottom 20% of frame, never covering the face
- [ ] Audio loudness is within target range for the platform
- [ ] Enhancement is visible but natural — skin tones look healthy, not orange
- [ ] No audio clipping or silence gaps at cut points
- [ ] File size is reasonable for the target platform
skills/core/remotion.md
+210
View File
@@ -0,0 +1,210 @@
# Remotion Skill
## When to Use
Use Remotion for advanced video composition from Phase 3 onward — anywhere that requires
React-based scene assembly, parametric templates, animated overlays, transitions, or
data-driven batch rendering. For simple cuts, burns, and encodes, prefer FFmpeg directly.
## Relationship to Remotion Agent Skills
The **installed agent skills** (`.agents/skills/remotion-best-practices/`) teach correct
Remotion API usage — imports, timing, animation constraints, code patterns.
**This file** teaches how OpenMontage uses Remotion — which compositions map to pipeline
stages, how artifacts flow in, and how renders are triggered.
## When to Use Remotion vs FFmpeg
| Use Case | Backend | Why |
|----------|---------|-----|
| Simple cuts, trims, concat | FFmpeg | Instant, no Node dependency |
| Subtitle burn-in | FFmpeg | Proven, fast |
| Face enhance, color grade | FFmpeg | Filter-based, deterministic |
| Multi-layer overlays + transitions | Remotion | React composability |
| Animated diagrams/text cards | Remotion | Frame-by-frame control |
| Data-driven batch videos | Remotion | Zod props + parametric renders |
| Generated explainer pipeline | Remotion | Full scene graph needed |
| Talking-head (video-only cuts) | FFmpeg | No images/animations needed |
**Note:** The `render` operation auto-routes — if any cut contains images,
animations, transitions, or component types (text_card, stat_card, etc.),
it delegates to Remotion automatically. No need to manually select backend.
## Architecture
```
remotion-composer/
├── src/
│ ├── Root.tsx # Composition registry
│ ├── compositions/ # One file per pipeline type
│ │ ├── Explainer.tsx # Generated explainer composition
│ │ ├── AnimatedScene.tsx # Individual animated scene
│ │ └── TitleCard.tsx # Standalone title card
│ ├── components/ # Reusable visual building blocks
│ │ ├── Caption.tsx # Subtitle/caption renderer
│ │ ├── DiagramOverlay.tsx
│ │ ├── ProgressBar.tsx
│ │ └── TransitionWrapper.tsx
│ └── styles/ # Tailwind + playbook-derived styles
├── public/ # Static assets (fonts, LUTs)
├── package.json
├── remotion.config.ts
└── tsconfig.json
```
## Pipeline Integration
### How Artifacts Map to Remotion Props
| OpenMontage Artifact | Remotion Prop | Maps To |
|---------------------|---------------|---------|
| `scene_plan.json` → `scenes[]` | `scenes` prop | `<TransitionSeries>` children |
| `scene.type` | Component selector | `talking_head` → `<Video>`, `diagram` → `<DiagramOverlay>`, etc. |
| `scene.start_seconds` / `end_seconds` | `from` / `durationInFrames` | `fps * seconds` conversion |
| `scene.transition_in` / `transition_out` | `<TransitionSeries.Transition>` | `fade`, `slide`, `wipe` |
| `asset_manifest.json` → assets | `assets` prop | `staticFile()` or absolute paths |
| `style_playbook` | `theme` prop | Colors, fonts, animation curves |
| `edit_decisions.json` → cuts | `cuts` prop | `<Series>` with trimmed `<Video>` segments |
| `media_profile` | Composition dimensions | `width`, `height`, `fps` from profile |
### Render Invocation
The orchestrator calls Remotion renders via CLI:
```bash
# Standard render
npx remotion render src/index.ts ExplainerVideo \
--props='{"scenes": [...], "theme": "clean_professional"}' \
--output=pipeline/<project>/output/final_output.mp4 \
--codec=h264
# With specific media profile
npx remotion render src/index.ts ExplainerVideo \
--width=1080 --height=1920 --fps=30 \
--props=props.json \
--output=output.mp4
```
In Python, invoke via `subprocess` from `video_compose.py` when `backend="remotion"`.
### Media Profile Mapping
| OpenMontage Profile | Remotion Config |
|--------------------|-----------------|
| `youtube_landscape` | `width: 1920, height: 1080, fps: 30` |
| `youtube_shorts` | `width: 1080, height: 1920, fps: 30` |
| `tiktok_vertical` | `width: 1080, height: 1920, fps: 30` |
| `instagram_reels` | `width: 1080, height: 1920, fps: 30` |
| `instagram_square` | `width: 1080, height: 1080, fps: 30` |
| `cinematic_wide` | `width: 2560, height: 1080, fps: 24` |
## Key Patterns
### Scene Plan to Composition
Each scene in `scene_plan.json` becomes a child of `<TransitionSeries>`:
```tsx
// Pseudocode — actual component in remotion-composer/src/compositions/Explainer.tsx
const Explainer: React.FC<ExplainerProps> = ({ scenes, theme, assets }) => {
return (
<TransitionSeries>
{scenes.map((scene, i) => (
<React.Fragment key={scene.id}>
{scene.transition_in && (
<TransitionSeries.Transition
presentation={mapTransition(scene.transition_in)}
timing={timing({ durationInFrames: 15 })}
/>
)}
<TransitionSeries.Sequence durationInFrames={secondsToFrames(scene)}>
<SceneRenderer scene={scene} theme={theme} assets={assets} />
</TransitionSeries.Sequence>
</React.Fragment>
))}
</TransitionSeries>
);
};
```
### Dynamic Duration with calculateMetadata
When TTS audio determines video length (generated explainers), use `calculateMetadata`:
```tsx
export const ExplainerVideo = {
component: Explainer,
calculateMetadata: async ({ props }) => {
const totalDuration = props.scenes.reduce(
(sum, s) => sum + (s.end_seconds - s.start_seconds), 0
);
return {
durationInFrames: Math.ceil(totalDuration * props.fps),
fps: props.fps,
width: props.width,
height: props.height,
};
},
};
```
### Style Playbook to Theme
Style playbooks (`skills/styles/`) define visual parameters. Map them to Remotion themes:
```tsx
// Derived from the style playbook YAML
const cleanProfessional = {
background: "#FFFFFF",
text: "#1A1A1A",
accent: "#2563EB",
fontFamily: "Inter",
headingWeight: 600,
transitionType: "fade",
transitionDuration: 15, // frames
animationEasing: "easeInOutCubic",
};
```
### Audio Layering
Narration + background music + SFX as parallel `<Audio>` components:
```tsx
<AbsoluteFill>
<Audio src={narrationUrl} />
<Audio src={musicUrl} volume={0.06} />
{sfxCues.map(cue => (
<Sequence key={cue.id} from={secondsToFrames(cue.time)}>
<Audio src={cue.url} volume={cue.volume} />
</Sequence>
))}
{/* Visual layers */}
</AbsoluteFill>
```
### Cost Tracking
Remotion renders are CPU-intensive but $0 API cost. Track via cost_tracker:
- `estimate`: based on composition duration × resolution tier
- `reserve`: 0 (no API spend)
- `reconcile`: wall-clock render time for benchmarking
## Critical Constraints
- **No CSS animations or transitions** — they don't render correctly. Use `useCurrentFrame()` + `interpolate()` for all motion.
- **No Tailwind animation classes** — `animate-*` classes break frame-based rendering. Static Tailwind utilities are fine.
- **Always clamp interpolate()** — use `extrapolateLeft: 'clamp', extrapolateRight: 'clamp'` to prevent values shooting past endpoints.
- **Node.js 18+ required** — listed as optional in minimum system, required in recommended.
- **Render in series, not parallel** — unless the machine has enough RAM. Each render spawns a Chromium instance.
## Quality Checklist
- [ ] Composition duration matches sum of scene durations minus transition overlaps
- [ ] All `staticFile()` references resolve to existing assets
- [ ] Transitions don't cut off content (account for overlap in timing)
- [ ] Audio layers are in sync with visual scenes
- [ ] Theme colors match the active style playbook
- [ ] Output resolution and FPS match the target media profile
- [ ] Render completes without Chromium timeout errors
- [ ] Final output plays correctly on target platform
skills/core/subtitle-sync.md
+104
View File
@@ -0,0 +1,104 @@
# Subtitle Sync Skill
## When to Use
Use the `subtitle_gen` tool to convert transcript data (from `transcriber`)
into properly timed subtitle files. This skill covers timing strategy,
formatting, and readability for both vertical and horizontal video.
## Tool
| Tool | Capability |
|------|-----------|
| `subtitle_gen` | Generate SRT, VTT, or caption JSON from word-level timestamps |
## Output Formats
| Format | Extension | Use Case |
|--------|-----------|----------|
| SRT | `.srt` | Universal — works with FFmpeg, players, YouTube upload |
| VTT | `.vtt` | Web-native — HTML5 video, browser playback |
| Caption JSON | `.caption.json` | Programmatic — word-level data for custom renderers |
## Cue Length by Format
### Vertical Short-form (TikTok, Reels, Shorts)
- **Max 3-4 words per cue** — screen is narrow, text must be large enough to read
- **Max 20 characters per line** — prevents wrapping on narrow screens
- Subtitles are **mandatory** (most viewers watch muted)
### Horizontal Standard (YouTube, web)
- **Max 6-8 words per cue** — wider screen accommodates more text
- **Max 42 characters per line** — standard broadcast limit
### General Rules
- Average viewer reads ~15 characters/second
- Minimum display time: 0.5 seconds per cue
- Maximum display time: 5 seconds per cue
## Styling for Burn-in (ASS force_style)
When burning subtitles via `video_compose`, these parameters are passed as ASS
`force_style`. Use the correct ASS color format: `&HAABBGGRR` (not hex RGB).
### Vertical Video (1080x1920)
```
font: Arial
font_size: 18
bold: true
primary_color: &H00FFFFFF # white (ASS format: alpha=00, BGR=FFFFFF)
outline_color: &H00000000 # black
outline_width: 3 # thick outline for readability on varied backgrounds
shadow: 2
margin_v: 50 # pixels from bottom edge
alignment: 2 # bottom center
```
### Horizontal Video (1920x1080)
```
font: Arial
font_size: 22
bold: true
primary_color: &H00FFFFFF
outline_color: &H00000000
outline_width: 2
shadow: 1
margin_v: 40
alignment: 2
```
### Common Mistakes
- **Wrong color format:** `&HFFFFFF` breaks positioning. Always use full 8-char `&H00FFFFFF`.
- **Font too large on vertical:** `font_size: 28` fills the center of a 9:16 frame. Use 18 max.
- **Too many words per cue on vertical:** 5+ words creates multi-line blocks that cover the face.
- **MarginV too large:** Values over 200 push text off-screen. Stay under 100 for most cases.
## Timing Best Practices
### Alignment with Speech
- Cue start must match word onset (not before the speaker starts)
- Cue end should extend ~200ms past the last word for comfortable reading
- Never let a cue linger into the next speaker's turn
### Word Boundary Grouping
The `subtitle_gen` tool groups words respecting `max_words_per_cue` and
`max_chars_per_line`. When word timestamps are unavailable, it falls back
to segment-level timing with even distribution.
## Quality Checklist
- [ ] Every spoken word appears in a subtitle cue
- [ ] No cue exceeds the character limit for the target format
- [ ] Subtitles are in the bottom 20% of frame — never covering the face
- [ ] Text is readable on mobile at native resolution
- [ ] Timing matches speech — no early or late cues
- [ ] Cues don't overlap each other
- [ ] Outline/shadow provides sufficient contrast against all backgrounds
skills/core/whisperx.md
+63
View File
@@ -0,0 +1,63 @@
# WhisperX / Transcription Skill
## When to Use
Use the `transcriber` tool whenever you need to convert speech to text from
audio or video files. This is the entry point for all transcript-dependent
workflows: subtitle generation, edit decisions based on spoken content, and
scene analysis from dialogue.
## Tool
| Tool | Capability |
|------|-----------|
| `transcriber` | Speech-to-text with word timestamps, language detection, optional diarization |
## How It Works
1. **Model loading:** faster-whisper loads the specified model size (tiny through large-v3). Defaults to `base` for speed. Use `large-v3` for production quality.
2. **Transcription:** VAD filter removes silence. Word-level timestamps are always enabled.
3. **Diarization (optional):** WhisperX alignment + pyannote speaker diarization assigns speaker labels. Requires `HF_TOKEN` environment variable.
## Model Size Guide
| Model | RAM | Speed (CPU) | Quality | When to Use |
|-------|-----|-------------|---------|-------------|
| `tiny` | ~1 GB | ~10x real-time | Low | Quick drafts, iteration |
| `base` | ~1 GB | ~5x real-time | Good | Default for development |
| `small` | ~2 GB | ~3x real-time | Better | Short content |
| `medium` | ~5 GB | ~1.5x real-time | High | Important content |
| `large-v3` | ~10 GB | ~0.5x real-time | Best | Final production |
## Key Patterns
### Choosing When to Diarize
- **Single speaker (talking head):** Skip diarization — it adds latency with no benefit.
- **Multiple speakers (interview, podcast):** Enable diarization to label who said what.
- **Diarization requires** `whisperx` and `HF_TOKEN`. If unavailable, the tool proceeds without speaker labels.
### Word Timestamps for Subtitles
The transcriber produces word-level timestamps with confidence scores. The `subtitle_gen` tool consumes these directly:
```
word_timestamps: [
{"word": "Hello", "start": 0.5, "end": 0.8, "probability": 0.95},
{"word": "world", "start": 0.9, "end": 1.2, "probability": 0.92},
...
]
```
### Language Detection
- Pass `language: null` to auto-detect (adds ~1s overhead).
- Pass an explicit ISO 639-1 code (`en`, `es`, `ja`, etc.) when you know the language.
## Quality Checklist
- [ ] Transcript text is accurate (spot-check 3-5 segments)
- [ ] Word timestamps align with actual speech when played back
- [ ] No missing segments or large gaps in the transcript
- [ ] Language was correctly detected (if auto)
- [ ] Speaker labels are correct (if diarization was used)
skills/creative/animation-pipeline.md
+127
View File
@@ -0,0 +1,127 @@
# Animation & Motion Graphics Pipeline
> Sources: School of Motion curriculum, After Effects documentation, Remotion documentation,
> Disney's 12 Principles of Animation (Frank Thomas & Ollie Johnston), Motion Design School,
> The Animator's Survival Kit (Richard Williams)
## Quick Reference Card
```
FRAME RATE: 30fps for web video | 24fps for cinematic feel | 60fps for UI/smooth motion
EASE DEFAULT: easeInOutCubic (0.65, 0, 0.35, 1) — never use linear
TRANSITION: 0.5-1.0s between scenes
ANTICIPATION: 2-3 frames before main action
OVERSHOOT: 10-15% past target, settle back in 3-5 frames
HOLD FRAMES: 6-12 frames (0.2-0.4s) on key poses
COLOR: Max 5 colors from playbook palette
EXPORT: H.264 CRF 18-20 for web, ProRes 422 for editing
```
## Frame Rate Selection
| Style | FPS | When to Use |
|-------|-----|-------------|
| **Cinematic animation** | 24 | Film-like feel, character animation, organic motion |
| **Web/explainer standard** | 30 | Default for YouTube/web video. OpenMontage default. |
| **Smooth UI animation** | 60 | Software demos, UI transitions, scrolling |
| **Stylized/limited** | 12-15 on 2s/3s | Deliberately choppy, artistic choice |
**OpenMontage default:** 30fps. Render Manim at 60fps and transcode to 30fps for smoother motion at delivery frame rate.
## Timing Principles (Applied to Motion Graphics)
### The 4 Most Important Principles
| Principle | Application | Timing |
|-----------|------------|--------|
| **Ease In/Out** | Every movement starts slow, ends slow | Use cubic or quart easing, never linear |
| **Anticipation** | Brief movement opposite to the main action | 2-3 frames (66-100ms at 30fps) |
| **Overshoot** | Object passes target, bounces back | 10-15% past target, settle in 3-5 frames |
| **Staging** | Only one thing moves at a time | Stagger animations by 3-6 frames |
### Easing Curves
| Curve | Cubic Bezier | Use For |
|-------|-------------|---------|
| **easeOutCubic** | `(0.33, 1, 0.68, 1)` | Objects entering the scene |
| **easeInCubic** | `(0.32, 0, 0.67, 0)` | Objects leaving the scene |
| **easeInOutCubic** | `(0.65, 0, 0.35, 1)` | Position changes within scene |
| **easeOutBack** | `(0.34, 1.56, 0.64, 1)` | Bouncy pop-in (playful) |
| **easeOutElastic** | spring simulation | Attention-grabbing reveals |
| **linear** | `(0, 0, 1, 1)` | **NEVER for motion** — only for opacity or color |
### Hold Frames
After a movement completes, **hold the pose** before the next animation:
| Context | Hold Duration |
|---------|--------------|
| Key information on screen | 1.0-2.0s (narration dependent) |
| Between animation beats | 0.3-0.5s (8-15 frames at 30fps) |
| After a reveal | 1.5-3.0s (let it register) |
| Quick transition | 0.1-0.2s (3-6 frames) |
## Scene Transitions
| Transition | Duration | When to Use |
|-----------|----------|-------------|
| **Hard cut** | Instant | Same topic, different angle/zoom |
| **Crossfade** | 0.5-1.0s | Topic change, gentle shift |
| **Wipe/slide** | 0.5-0.8s | Sequential steps, progression |
| **Zoom in** | 0.8-1.2s | Diving deeper into detail |
| **Zoom out** | 0.8-1.2s | Revealing bigger picture |
| **Match cut** | Instant | Same shape/position, different content |
| **Morph/transform** | 1.0-2.0s | Concept evolution, before/after |
### Transition Rules
1. **Consistent transitions** — pick 2-3 types and stick with them throughout the video
2. **Transition = meaning** — a wipe means "next step," a zoom means "deeper detail"
3. **Don't over-transition** — a hard cut is the most invisible and most professional transition
4. **Audio leads visual** — start transition sound 10-20ms before the visual change
## Composition for Motion Graphics
### Layout
- **Rule of thirds** — place focal elements on intersection points
- **Visual hierarchy** — largest/brightest element = most important
- **White space** — minimum 10% margin on all sides (within title-safe)
- **Direction of motion** — left-to-right = forward/progress, right-to-left = reverse/back
### Color
- **Max 5 colors** from the style playbook palette
- **1 accent color** for emphasis — used sparingly
- **Background** should be the least saturated color
- **Contrast** between foreground elements and background: minimum 3:1
### Stagger and Choreography
When multiple elements enter:
- Stagger entry by **3-6 frames** (100-200ms) between elements
- Enter from the same direction for grouped elements
- Use `LaggedStart` (Manim) or staggered `delay` (Remotion) with `lag_ratio=0.1-0.2`
## Export Settings
| Target | Codec | Settings |
|--------|-------|----------|
| YouTube/web final | H.264 | CRF 18-20, `-pix_fmt yuv420p`, `-movflags +faststart` |
| Editing intermediate | ProRes 422 | For further editing/compositing |
| Transparent overlay | ProRes 4444 | When compositing over other footage |
| GIF preview | GIF | 480px wide, 15fps, 256 colors |
## Applying to OpenMontage
When building animation/motion graphics content:
1. **Render at 30fps** (OpenMontage default) — Manim at 60fps, transcode down
2. **Never use linear easing** — default to `easeInOutCubic` for all motion
3. **Stagger multi-element entrances** by 100-200ms — don't reveal everything at once
4. **Hold key frames** for 1.0-2.0s after reveals (synced to narration)
5. **Use 2-3 transition types** consistently — hard cut + crossfade covers most needs
6. **Audio leads visual** — SFX starts 10-20ms before transition (see sound-design.md)
7. **Max 5 palette colors** — enforce from the style playbook
8. **Anticipation + overshoot** on important movements for polish
9. **Export H.264 CRF 18-20** for final output via `video_compose`
skills/creative/bg-remove-usage.md
+113
View File
@@ -0,0 +1,113 @@
# Background Removal Usage for OpenMontage
> Sources: rembg library documentation, U2Net paper (Qin et al. 2020), IS-Net paper
> (Qin et al. 2022), OpenMontage `tools/bg_remove.py` implementation
## Quick Reference Card
```
DEFAULT MODEL: u2net (general purpose, fast)
FOR PEOPLE: u2net_human_seg (optimized for human silhouettes)
FINE EDGES: Enable alpha_matting (hair, fur, leaves)
OUTPUT: Transparent PNG by default; set bg_color for solid replacement
RUNTIME: ~1-3s per image (CPU), <0.5s (GPU with onnxruntime-gpu)
INSTALL: pip install rembg (CPU) | pip install rembg[gpu] (CUDA)
```
## When to Use bg_remove
Background removal is an **asset-prep** step. Use it before the compose stage.
- **Product demos / e-commerce videos** -- isolate a product on a clean background
- **Compositing** -- layer a speaker over generated backgrounds or diagrams
- **Thumbnail generation** -- clean cutouts for YouTube thumbnails
- **Green-screen replacement** -- achieve green-screen results without an actual green screen
- **B-roll preparation** -- clean up raw photos for overlay use
## Model Selection Guide
| Model | Best For | Speed | Notes |
|-------|----------|-------|-------|
| `u2net` | General objects, products, scenes | Fast | Default; good all-rounder |
| `u2net_human_seg` | People, portraits, speakers | Fast | More accurate masks for human silhouettes |
| `isnet-general-use` | Complex edges, hair, fur | Slower | Higher detail on fine boundaries |
**Decision rule:** If the subject is a person, use `u2net_human_seg`. If the subject has intricate edges (hair, fur, foliage) and you need maximum quality, use `isnet-general-use`. Otherwise, use the default `u2net`.
## Alpha Matting
Alpha matting refines the edge mask by computing soft transparency at boundaries. It produces more natural edges but costs approximately 2x processing time.
| Subject Type | Alpha Matting | Reason |
|-------------|---------------|--------|
| Hair, fur, feathers | Enable | Fine semi-transparent strands need soft edges |
| Leaves, trees, grass | Enable | Irregular organic boundaries benefit from matting |
| Products, devices | Disable | Clean geometric edges; matting adds no value |
| Text, logos, shapes | Disable | Hard edges are correct for these subjects |
## Common Workflows
### 1. Speaker Cutout for Compositing
Extract a speaker from their background and layer over a diagram or slide.
```
bg_remove(input_path="speaker.png", model="u2net_human_seg")
--> speaker_nobg.png (transparent)
--> compose over diagram/slide in compose stage
```
### 2. Product Isolation
Isolate a product and optionally place on a brand-colored background.
```
bg_remove(input_path="product.jpg", model="u2net")
--> product_nobg.png (transparent)
# Or with brand background:
bg_remove(input_path="product.jpg", model="u2net", bg_color="#FFFFFF")
--> product_nobg.png (white background)
```
### 3. Thumbnail Prep
Remove background, upscale, then compose with text overlays.
```
bg_remove(input_path="subject.png", model="u2net_human_seg", alpha_matting=True)
--> subject_nobg.png
--> upscale --> compose with text overlays in compose stage
```
### 4. Batch Frame Processing
When preparing multiple frames for a compositing sequence, process all source frames before entering the compose stage.
```
for each source frame:
bg_remove(input_path=frame, model="u2net_human_seg")
--> frame_nobg.png
then: compose all transparent frames over background sequence
```
## Quality Checklist
Before moving to the compose stage, verify each bg_remove output:
- [ ] **Edge quality is clean** -- no halo artifacts around the subject
- [ ] **Fine details preserved** -- hair, fingers, and thin features are intact
- [ ] **Transparency is complete** -- no residual background bleed in transparent areas
- [ ] **Subject integrity** -- no parts of the subject were incorrectly removed
- [ ] **Compositing test** -- when layered over the target background, the subject blends naturally
## Applying to OpenMontage
When using the `bg_remove` tool in asset preparation:
1. **Use `u2net_human_seg` for any frame containing people** -- it produces tighter masks around human silhouettes than the general model
2. **Enable `alpha_matting` only for subjects with complex edges** like hair, fur, or foliage -- skip it for clean-edged subjects to save processing time
3. **For compositing workflows, output transparent PNG** (omit `bg_color`) and layer in the compose stage -- this preserves maximum flexibility
4. **For solid-background replacements, set `bg_color`** to match the playbook's background color token -- keeps outputs consistent with the project style
5. **Process source frames BEFORE the compose stage** -- bg_remove is an asset-prep step, not a compose-time operation
6. **Check output edges at full resolution before compositing** -- halo artifacts and edge bleed are visible in final video and must be caught early
skills/creative/broll-planning.md
+140
View File
@@ -0,0 +1,140 @@
# B-Roll Planning for OpenMontage
> How to plan B-roll needs from a script, decide between stock and generated footage,
> construct effective search queries, and evaluate footage quality.
## When to Use
You are planning visual assets for a video and need supplementary footage (B-roll) to accompany
narration, establish context, or add visual variety. This skill teaches you when to reach for
stock footage vs. AI generation, and how to get good results from each.
## The Decision Matrix: Stock vs. Generated
| Scene Need | Prefer Stock | Prefer Generated |
|------------|-------------|-----------------|
| Real-world establishing shot (city, office, nature) | **Yes** — stock excels here | Only if no good stock match |
| People in realistic settings | **Yes** — generated humans often look uncanny | Only with high-quality models |
| Abstract concept visualization | No | **Yes** — AI can create what doesn't exist |
| Custom diagrams/infographics | No | **Yes** — use `diagram_gen` or `image_selector` |
| Branded/stylized imagery | No | **Yes** — AI matches your playbook style |
| Historical/archival footage | **Yes** — stock libraries have archives | No |
| Specific technical equipment | **Yes** — real photos are more credible | Only if equipment doesn't exist |
| Motion/action clips (waves, traffic, clouds) | **Yes** — stock video is perfect for this | AI video is catching up |
| Metaphorical imagery (growth, connection) | Either works | **Yes** — more creative control |
**Rule of thumb:** If the scene needs to look _real_, use stock. If it needs to look _specific to your concept_, generate it.
## Extracting B-Roll Needs from a Script
Walk the script section by section. For each section, ask:
1. **What is the narrator talking about?** — The subject suggests the visual.
2. **Is there an enhancement cue?** — The script writer may have embedded `[B-ROLL: ...]` cues.
3. **Does this section reference something concrete?** — "servers in a data center" → stock footage of servers.
4. **Does this section explain an abstract concept?** — "the algorithm weighs each factor" → generated diagram.
5. **How long is this section?** — Determines clip duration needed.
### Output: B-Roll Brief
For each identified need, create an entry:
```
Scene: s3 (15s-22s)
Need: Establishing shot of a modern data center
Source: stock
Keywords: ["data center", "server room", "rack servers blue light"]
Duration: 4-6 seconds
Orientation: landscape
Mood: cool, technological, clean
Fallback: AI-generated image of server racks
```
## Constructing Effective Stock Search Queries
### Query Construction Rules
1. **Be specific but not too specific.** "aerial city skyline sunset" works. "aerial shot of downtown San Francisco financial district at 6:47pm golden hour" returns nothing.
2. **Use 2-4 keywords.** Stock search is keyword-based, not semantic. More words = fewer results.
3. **Lead with the subject.** "ocean waves" not "beautiful calm serene ocean waves at dawn."
4. **Include the visual quality you need:**
- Add "aerial" or "drone" for overhead shots
- Add "close-up" or "macro" for detail shots
- Add "timelapse" for time-lapse footage
- Add "slow motion" for slow-mo clips
5. **Try synonyms on failure.** If "programmer coding" returns poor results, try "developer laptop" or "software engineer workspace."
### Query Templates by Scene Type
| Scene Type | Query Template | Example |
|-----------|---------------|---------|
| Establishing | `[place] [time of day]` | "tokyo skyline night" |
| Activity | `[person] [action]` | "scientist microscope" |
| Object | `[object] [style]` | "circuit board closeup" |
| Nature | `[element] [quality]` | "ocean waves aerial" |
| Abstract motion | `[movement] [style]` | "light trails timelapse" |
| Workplace | `[setting] [activity]` | "modern office meeting" |
## Evaluating Stock Footage Quality
When the stock tool returns results, evaluate before using:
### Image Criteria
- **Resolution:** Meets target (1080p minimum for video frames)
- **Relevance:** Actually depicts what the scene needs (not just keyword match)
- **Style compatibility:** Doesn't clash with the playbook's visual style
- **No watermarks:** Pexels/Pixabay are license-free, but verify
- **Composition:** Subject is well-framed, not cut off awkwardly
### Video Criteria (all image criteria plus)
- **Duration:** At least as long as the scene needs (can trim, can't extend)
- **Motion:** Smooth, no jarring camera movement (unless that's the intent)
- **Frame rate:** Matches target output (24/30fps standard)
- **Audio:** Stock video audio is usually discarded — don't factor it in
### Scoring Heuristic
Rate each result 1-5:
- **5:** Perfect match, use immediately
- **4:** Good match, minor crop or trim needed
- **3:** Acceptable, would benefit from color grading to match playbook
- **2:** Marginal — try different keywords first
- **1:** Wrong — doesn't match the scene at all
**Threshold:** Use results scoring 3+. Below 3, refine the query or switch to generated.
## Failure Escalation
When stock search fails (no results or all score below 3):
1. **Retry with different keywords** — try synonyms, broader terms, or different angles
2. **Try the other stock provider** — Pexels and Pixabay have different libraries
3. **Switch to AI generation** — use `flux_image` or `openai_image` with the scene description
4. **Escalate to user** — "I couldn't find good stock footage for [scene]. Here are the best options: [show results]. Or I can generate an image instead. What do you prefer?"
The agent should only ask the user when both stock search AND generation fallback would produce suboptimal results. For most cases, the fallback chain handles it silently.
## Attribution Tracking
Both Pexels and Pixabay are free for commercial use with no required attribution.
However, best practice is to track sources in the asset manifest:
```json
{
"id": "broll-scene-3",
"type": "image",
"source_tool": "pexels_image",
"provider": "pexels",
"attribution": {
"photographer": "Joey Farina",
"source_url": "https://www.pexels.com/photo/...",
"license": "Pexels License"
}
}
```
This data is available in the tool's response (`photographer`, `pexels_url` / `page_url`). Include it in the asset manifest for transparency.
skills/creative/cinematic.md
+129
View File
@@ -0,0 +1,129 @@
# Cinematic Video Pipeline
> Sources: No Film School editorial guides, StudioBinder filmmaking resources, Film Riot
> production tutorials, CinematographyDB shot databases, Walter Murch "In the Blink of an Eye"
## Quick Reference Card
```
ASPECT RATIO: 2.39:1 (widescreen cinematic) or 16:9 with letterbox
LETTERBOX: Black bars at top/bottom — 1920x800 active area in 1920x1080 frame
FRAME RATE: 24fps (cinematic standard)
SHOT DURATION: 4-8 seconds average (longer than explainer, shorter than documentary)
COLOR GRADE: cinematic_warm or cinematic_cool profile
AUDIO: Layered: dialogue + ambient + Foley + score
MUSIC: 60-90 BPM, orchestral or ambient, dynamic (not loop-based)
TARGET LUFS: -14 LUFS integrated, -24 LUFS for quiet moments
```
## Aspect Ratios
| Ratio | Resolution (in 1080p frame) | Feel | When to Use |
|-------|---------------------------|------|-------------|
| **2.39:1** (anamorphic) | 1920x803 (138px bars each) | Epic, cinematic, grand | Cinematic explainers, brand films |
| **2.35:1** (scope) | 1920x817 (131px bars each) | Classic film | Similar to 2.39:1, slightly taller |
| **1.85:1** (flat) | 1920x1038 (21px bars each) | Moderate cinematic | Subtle letterbox, less dramatic |
| **16:9** (no letterbox) | 1920x1080 | Standard | Default, no cinematic treatment |
### Implementing Letterbox in FFmpeg
```bash
# Add 2.39:1 letterbox (138px black bars top and bottom)
ffmpeg -i input.mp4 -vf "pad=1920:1080:0:138:black,crop=1920:1080:0:0" output.mp4
# Or render at native ratio and pad:
ffmpeg -i input.mp4 -vf "scale=1920:803,pad=1920:1080:0:138:black" output.mp4
```
**Rule:** Only use letterbox when the content genuinely benefits from cinematic framing. Don't letterbox a screen recording or talking head — it just wastes pixels.
## Shot Duration and Pacing
### Average Shot Length by Style
| Style | Average Shot | Cuts/Minute |
|-------|-------------|-------------|
| Action/intense | 2-4s | 15-30 |
| Standard cinematic | 4-8s | 8-15 |
| Documentary | 6-12s | 5-10 |
| Contemplative | 10-20s | 3-6 |
| Montage sequence | 1-3s | 20-40 |
### Pacing Rhythm
Cinematic pacing follows a **breathing rhythm** — vary shot length deliberately:
```
Long (8s) → Medium (5s) → Short (3s) → Short (2s) → LONG (10s) → Medium (6s)
```
**Never use the same shot length 3 times in a row** — it creates monotony.
### The Murch Rule
Walter Murch's editing priorities (in order of importance):
1. **Emotion** — does the cut serve the emotional arc?
2. **Story** — does the cut advance the narrative?
3. **Rhythm** — does the cut feel right in the pacing?
4. **Eye trace** — where is the viewer looking?
5. **2D plane** — screen geography (180-degree rule)
6. **3D space** — spatial continuity
For OpenMontage explainers using cinematic style: prioritize rhythm and story over spatial concerns (since we're often cutting between generated images, not continuous footage).
## Audio Layering
Cinematic audio has **4 layers** (not just voiceover + music):
| Layer | Level | Content |
|-------|-------|---------|
| **Dialogue/narration** | -12 dB peak | Primary voice |
| **Music/score** | -24 to -18 dB | Orchestral, ambient, dynamic |
| **Ambient/room tone** | -30 to -24 dB | Environmental sound bed |
| **Foley/SFX** | -18 to -12 dB | Specific action sounds |
### Music for Cinematic
| Characteristic | Value |
|---------------|-------|
| BPM | 60-90 (slower than standard explainer) |
| Genre | Orchestral, ambient, piano, cinematic electronic |
| Dynamics | Dynamic (crescendos, swells, quiet moments) — NOT loop-based |
| Key changes | At narrative turning points |
| Silence | Deliberately remove music for 3-5s at key reveals |
### Ambient Sound
Add a subtle ambient layer to fill silence and create depth:
- Room tone / air conditioning hum (very low, -35 dB)
- Environmental sounds matching the topic (city, nature, lab)
- Generates "presence" even during narration pauses
## Color Grading for Cinematic
| Look | Profile | Intensity | Characteristics |
|------|---------|-----------|----------------|
| **Warm cinematic** | `cinematic_warm` | 0.85 | Orange highlights, lifted shadows |
| **Teal & orange** | `cinematic_cool` | 0.7 | Classic Hollywood blockbuster look |
| **Moody dark** | `moody_dark` | 0.6 | Crushed blacks, low saturation |
| **Vintage film** | `vintage_film` | 0.7 | Faded, warm tint, reduced contrast |
**Cinematic grading rules:**
- Shadows should be slightly lifted (never pure black)
- Highlights should be slightly rolled off (never pure white)
- Skin tones must stay on the vectorscope skin tone line
- Consistency across all clips — one LUT/profile for the entire video
## Applying to OpenMontage
When building cinematic-style content:
1. **Set aspect ratio** — use 2.39:1 letterbox for true cinematic, or 16:9 with `cinematic_warm` grade for subtle
2. **Render at 24fps** if the content is purely generated/animated (set in `video_compose`)
3. **Shot duration 4-8 seconds average** — vary deliberately, never same length 3x
4. **Layer audio** — narration + music + ambient minimum; add Foley SFX at key moments
5. **Music at 60-90 BPM**, dynamic (not looping) — use `music_gen` with "cinematic orchestral" prompt
6. **Remove music for 3-5 seconds** at key reveals — silence is powerful
7. **Color grade with `cinematic_warm` or `cinematic_cool`** at 0.7-0.85 intensity
8. **Image prompts** should include "cinematic lighting, shallow depth of field, film grain" for matching aesthetic
9. **Slower narration** — 140-150 WPM (slower than standard 155 WPM explainer pace)
skills/creative/data-visualization.md
+344
View File
@@ -0,0 +1,344 @@
# Data Visualization Strategy Skill
## When to Use
Apply this skill when a scene requires presenting data visually: statistics, comparisons,
trends, compositions, or key metrics. This skill guides chart type selection, animation
sequencing, label placement, data density, and color usage to produce charts that are
clear, accurate, and effective in video.
## Tools
| Tool | Role |
|------|------|
| `diagram_gen` | Generate charts via Mermaid or D3 |
| `image_selector` | Generate stylized chart illustrations (FLUX/DALL-E) |
| Remotion | Animated chart components (bar grow, line draw, pie fill) |
| Manim | Mathematical plots, coordinate systems, function graphs |
## Chart Type Decision Tree
Follow this tree top-to-bottom. Stop at the first match.
```
Is there data to visualize?
NO -> Use a text card or stat card instead
YES -> How many data points?
< 3 -> Use text or stat card (charts look empty with 1-2 points)
3-9 -> Continue to "What story does the data tell?"
> 12 -> Simplify first: aggregate into top-N + "Other", then continue
What story does the data tell?
|
|-- Comparing quantities across categories?
| -> BAR CHART (horizontal if labels are long)
|
|-- Showing a trend over time?
| -> LINE CHART (area chart if showing volume)
|
|-- Showing parts of a whole?
| -> PIE / DONUT CHART (max 5-6 slices)
| (If > 6 categories, aggregate smallest into "Other")
|
|-- Showing key metrics / KPIs?
| -> KPI GRID (3-6 stat cards in a grid layout)
|
|-- Showing ranking or ordered list?
| -> HORIZONTAL BAR CHART (sorted descending)
|
|-- Showing before/after or change?
| -> PAIRED BAR CHART or STAT CARD with delta arrow
|
|-- Showing correlation between two variables?
| -> LINE CHART with dual series (avoid scatter in video -- too dense)
|
|-- None of the above?
| -> Default to BAR CHART (most universally readable)
```
### When NOT to Use a Chart
| Situation | Do This Instead |
|-----------|----------------|
| Fewer than 3 data points | Stat card or text overlay: "Revenue grew 40% to $2.1M" |
| More than 12 categories | Aggregate into top 5-7 + "Other", then chart |
| Single number to emphasize | Full-screen stat card with impact animation |
| Qualitative comparison | Side-by-side images or text table |
| Data requires 30+ seconds to read | Split into multiple simpler charts across scenes |
## Animation Sequencing
Every chart in video should be animated. Static charts feel like slides, not video.
### Pattern: Build-Up (Default)
Show empty axes/frame, then animate data in.
```
Frame 0.0s: Empty chart frame (axes, title, gridlines visible)
Frame 0.3s: First data element begins animating in
Frame 2.0s: All data elements fully rendered
Frame 2.0-5.0s: Hold for readability
```
- **Bar charts:** Bars grow upward from baseline (stagger left-to-right, 0.1s delay each)
- **Line charts:** Line draws left-to-right following the data path
- **Pie/donut charts:** Slices fill clockwise from 12 o'clock, largest slice first
- **KPI grid:** Numbers count up from 0 to final value (odometer effect)
### Pattern: Narrative Highlight
Highlight one element at a time as narration mentions it.
```
Frame 0.0s: Full chart visible but all elements at 30% opacity (desaturated)
Frame 0.5s: First highlighted element goes full color + slight scale-up
Frame 3.0s: First element returns to normal, second element highlights
...continue for each narrated point
```
Use when the narrator walks through specific data points. Keeps viewer focus synchronized
with the voiceover.
### Pattern: Comparison Reveal
Show baseline, then animate the change.
```
Frame 0.0s: Baseline data visible (e.g., "Before" bars)
Frame 2.0s: Hold baseline for comprehension
Frame 2.5s: Animate change (bars grow/shrink to "After" values)
Frame 3.5s: Delta labels appear (+40%, -15%, etc.)
Frame 3.5-7.0s: Hold for readability
```
Use for before/after, year-over-year, or A/B comparisons.
### Timing Rules
| Element | Animation Duration | Hold Duration |
|---------|-------------------|---------------|
| Chart build-up | 2-4 seconds | 3-5 seconds |
| Single element highlight | 0.3-0.5 seconds | 2-3 seconds |
| Comparison transition | 1-2 seconds | 3-5 seconds |
| KPI counter | 1.5-2 seconds | 2-3 seconds |
| Label/annotation appear | 0.2-0.3 seconds | Remains on screen |
**Critical rule:** The chart must be fully built and held for at least 3 seconds before the
scene transitions. Viewers need time to read. If the narration moves on before the chart is
readable, either extend the scene or simplify the chart.
## Label Placement Rules
### Bar Charts
```
Vertical bars:
- Value labels: ABOVE each bar (or INSIDE if bar is tall enough for legible text)
- Category labels: Below on x-axis, horizontal text
- If labels overlap: rotate 45 degrees or use horizontal bars instead
- Y-axis: include gridlines, omit axis label if title makes it obvious
Horizontal bars:
- Value labels: TO THE RIGHT of each bar
- Category labels: Left-aligned on y-axis
- Preferred when category names are longer than 2 words
```
### Line Charts
```
- Endpoint labels: Show value at the last data point (right end)
- Start label: Show value at the first data point (left end) for context
- Dense data (>7 points): Label only start, end, and notable peaks/valleys
- Avoid: Labels on every point (creates clutter in video)
- Legend: Top-right or inline (label next to the line) for multi-series
```
### Pie / Donut Charts
```
- Large slices (>= 10%): Label INSIDE the slice (percentage + category)
- Small slices (< 10%): Label OUTSIDE with leader line connecting to slice
- Center of donut: Use for total value or key metric label
- Maximum: 5-6 slices. Combine anything under 5% into "Other"
- Always show percentages, not just raw values
```
### KPI Grid
```
- Large number: Center of each card, using stat_card font (3-4x body size)
- Label: Below the number, smaller font, describes the metric
- Delta indicator: Small arrow + percentage showing change (green up, red down)
- Grid: 2x2 or 3x2 layout, evenly spaced, consistent card sizing
```
### Universal Label Rules
- **Title visible:** Every chart must have a clear title (top-left or top-center)
- **Source citation:** If data is from an external source, show "Source: [name]" in small text at bottom
- **Units:** Always show units (%, $, seconds, etc.) either in the title or on the axis
- **No orphan labels:** Every visual element must be labeled or explained by the narration
## Data Density vs Readability
### The Video Rule: Less Is More
Video is not a spreadsheet. The viewer cannot pause, scroll, or zoom. Every data point
competes for attention in a 5-7 second window.
```
Ideal data points per chart type:
Bar chart: 5-7 bars (max 9)
Line chart: 5-12 points (max 15, but label sparsely)
Pie chart: 3-5 slices (max 6)
KPI grid: 3-6 metrics (max 6)
```
### Simplification Strategies
| Problem | Solution |
|---------|----------|
| Too many categories (>9) | Show top 5-7, aggregate rest into "Other" |
| Too many time periods | Aggregate (monthly -> quarterly, daily -> weekly) |
| Multiple metrics to show | Split into separate charts across scenes |
| Wide value ranges | Use normalized/percentage view instead of absolute |
| Decimal precision | Round aggressively: $1,234,567 -> $1.2M |
### Font Size Minimums (at 1080p)
These are non-negotiable for readability on screens including mobile:
| Element | Minimum Size | Recommended |
|---------|-------------|-------------|
| Chart title | 32px | 36-40px |
| Axis labels | 24px | 28px |
| Value labels | 24px | 28px |
| Annotations | 20px | 24px |
| Source citation | 16px | 18px |
**Scaling rule:** For 4K output, multiply by 2x. For 720p, these minimums still apply
(they are the floor).
## Color Usage
### Deriving Chart Colors from the Playbook
Charts must look like they belong to the video. Always derive colors from the active
style playbook.
```
Color derivation priority:
1. playbook.visual_language.color_palette.chart_palette (if the playbook defines one)
2. Derive from primary + accent colors:
- Bar/slice 1: primary[0]
- Bar/slice 2: accent[0]
- Bar/slice 3: primary[1]
- Bar/slice 4: accent[1]
- Bar/slice 5+: generate by adjusting lightness of primary[0]
3. Background: use playbook background color
4. Text/labels: use playbook text color
5. Gridlines: use playbook muted color at 50% opacity
```
### Highlight and Focus
```
Highlighting strategy:
- KEY data point: Full saturation of accent[0], slight scale-up (1.05x)
- FOCUS data points: Full saturation of their assigned color
- NON-FOCUS points: Desaturate to 30% opacity or use muted color
- BASELINE/CONTEXT: Dashed lines using muted color
```
### Accessibility Rules
Never rely on color alone to convey meaning:
- **Add patterns:** Use hatching, dots, or stripes on bars/slices in addition to color
- **Add labels:** Every bar/slice/line must have a text label, not just a legend
- **Contrast:** Minimum 3:1 contrast ratio between adjacent chart elements
- **Colorblind-safe:** Avoid red-green as the only differentiator. Prefer blue-orange or
blue-yellow pairings when showing positive/negative
## Common Pitfalls
### Misleading Charts
| Pitfall | Why It Misleads | Fix |
|---------|----------------|-----|
| Truncated y-axis (not starting at 0) | Small differences look enormous | Always start bar chart y-axis at 0 |
| 3D charts | Perspective distorts size perception | Always use 2D flat charts |
| Dual y-axes with different scales | Implies false correlation | Use two separate charts side by side |
| Cherry-picked time range | Hides broader context | Show full relevant range or acknowledge truncation |
| Pie chart with too many slices | Impossible to compare small angles | Max 5-6 slices, aggregate rest |
### Animation Mistakes
| Pitfall | Fix |
|---------|-----|
| Animation too fast (< 1.5s) | Viewers cannot track what appeared. Minimum 2s build-up |
| No hold time after animation | Scene cuts away before chart is readable. Hold 3-5s minimum |
| All elements appear at once | Loses the narrative. Stagger element entrance |
| Gratuitous bouncing/spinning | Distracts from data. Use clean ease-in-out per playbook |
### Design Mistakes
| Pitfall | Fix |
|---------|-----|
| Too many colors (>5 in one chart) | Limit to 4-5 distinct colors. Aggregate or split charts |
| Missing title | Every chart needs a title. Viewers have no other context |
| Tiny font on mobile | Enforce minimums: 32px title, 24px labels at 1080p |
| Decorative gridlines | Use light gridlines or none. They should aid reading, not decorate |
| Dark text on dark background | Use playbook text color on playbook background. Check contrast |
## Integration with Scene Director
When the Scene Director identifies a data visualization need, apply this skill as follows:
1. **Determine chart type** using the decision tree above
2. **Specify animation pattern** in the scene's `movement` field (e.g., "build-up: bars grow from baseline over 2s, hold 4s")
3. **Include label specifications** in `overlay_notes` (e.g., "value labels above bars, title top-left, source bottom-right")
4. **Reference playbook colors** in `required_assets` description (e.g., "bar chart using primary[0] #2563EB for main bars, accent[0] #F59E0B for highlight bar")
5. **Set scene duration** to accommodate animation (2-4s) + hold (3-5s) = minimum 5s per chart scene
### Example Scene Specification
```json
{
"id": "scene-7",
"type": "animation",
"description": "Horizontal bar chart comparing response times: Traditional DB 450ms, Vector DB 12ms, Cached 3ms. Bars grow left-to-right with stagger. Vector DB bar highlighted in accent color. Hold for readability.",
"start_seconds": 32,
"end_seconds": 40,
"script_section_id": "s5",
"framing": "full-screen chart, centered with generous padding",
"movement": "build-up: bars grow from left over 2.5s with 0.3s stagger, hold 5s",
"transition_in": "fade",
"transition_out": "dissolve",
"overlay_notes": "Title: 'Query Response Time Comparison'. Value labels right of bars (ms units). Source: 'Benchmark 2024' bottom-right 16px. Vector DB bar uses accent[0], others use primary[0] at 50% opacity.",
"required_assets": [
{
"type": "chart_data",
"description": "Horizontal bar chart data: Traditional DB 450ms, Vector DB 12ms, Cached 3ms. Use playbook primary #2563EB at 50% for context bars, accent #F59E0B for Vector DB highlight bar.",
"source": "generate"
}
]
}
```
## Quality Checklist
- [ ] Chart type matches the data story (not just "default to bar chart")
- [ ] Data points within limits: 5-7 bars, 3-5 pie slices, 5-12 line points
- [ ] Animation duration: 2-4s build, 3-5s hold minimum
- [ ] All text meets size minimums: 32px title, 24px labels at 1080p
- [ ] Colors derived from active playbook palette
- [ ] Key data point has visual emphasis (highlight color, scale, or annotation)
- [ ] No reliance on color alone for meaning (labels + patterns for accessibility)
- [ ] Y-axis starts at 0 for bar charts
- [ ] No 3D effects or perspective distortion
- [ ] Title is visible and descriptive
- [ ] Source cited if using external data
- [ ] Chart is readable when paused at any frame during the hold period
skills/creative/diagram-gen-usage.md
+111
View File
@@ -0,0 +1,111 @@
# Diagram Generation Usage for OpenMontage
> Sources: Mermaid.js documentation, existing Layer 3 skill at `.agents/skills/beautiful-mermaid/`,
> Mermaid-Sonar complexity analysis research, Mermaid GitHub issues #651 (scaling), #3029 (animation)
## Quick Reference Card
```
MAX NODES (1080p): 15-20 nodes, 20-25 edges
MAX NODES (4K): 25-35 nodes, 35-45 edges
MAX NODES (vert): 10-12 nodes, 12-15 edges
MIN FONT SIZE: 16px at 1080p, 14px at 4K
RENDER WIDTH: Minimum 1200px
RENDER VIEWPORT: 3840x2160 (4K) for high-res PNG export
THEME (dark bg): tokyo-night or dracula
THEME (light bg): github-light or catppuccin-latte
```
## Diagram Type Selection
| Type | Video Suitability | Best For |
|------|------------------|----------|
| **Flowchart (TD)** | Excellent | Process flows, decision trees, algorithms |
| **Sequence diagram** | Good | API calls, user interactions, message flows |
| **State diagram** | Good | State machines, lifecycle, workflow status |
| **Class diagram** | Fair | Architecture (limit to 3-5 classes) |
| **ER diagram** | Poor for video | Too dense — simplify to key entities only |
| **Gantt chart** | Fair | Timelines, project phases |
| **Mindmap** | Good | Concept overviews, topic breakdowns |
**Default:** Use flowcharts (top-down `TD`) unless the content specifically requires another type. They read naturally and build well step by step.
## Complexity Limits for Video
Video is transient — viewers can't zoom or scroll. Cut complexity in half compared to static documentation.
| Target Resolution | Max Nodes | Max Edges | Min Font Size (CSS) |
|------------------|-----------|-----------|---------------------|
| 1920x1080 (HD) | 15-20 | 20-25 | 16px |
| 3840x2160 (4K) | 25-35 | 35-45 | 14px |
| 1080x1920 (vertical) | 10-12 | 12-15 | 18px |
**If your diagram exceeds these limits:** Split it into multiple frames, each showing a subset. This also creates a natural "building" animation for the video.
## Color Themes for Video
| Use Case | Theme | Why |
|----------|-------|-----|
| Dark video background | `tokyo-night` or `dracula` | High contrast, readable |
| Light video background | `github-light` or `catppuccin-latte` | Soft, professional |
| Code/developer content | `one-dark` | Familiar to dev audience |
| Maximum contrast | `zinc-dark` | Neutral, no color bias |
| Corporate/presentation | `nord-light` | Calm, professional |
**Match the playbook:** The diagram theme should complement the style playbook's color palette.
## Progressive Building for Video
Mermaid doesn't animate natively. Use progressive rendering to create a "building" effect:
### Approach: Multi-Stage Renders
1. Render diagram in stages — first 2 nodes, then 4, then full diagram
2. Each stage is a separate Mermaid render → SVG → PNG
3. Crossfade or cut between stages in FFmpeg
4. Viewers follow the logic step by step
### Highlighting Current Step
Use `classDef` to highlight the active node and dim completed ones:
```mermaid
graph TD
A[Input Data]:::completed --> B[Process]:::highlight
B --> C[Output]:::dimmed
classDef highlight fill:#f96,stroke:#333,stroke-width:3px
classDef completed fill:#6c6,stroke:#333,stroke-width:1px
classDef dimmed fill:#555,stroke:#333,opacity:0.5
```
Generate one PNG per step with different `classDef` assignments, then sequence them in the compose stage.
## Styling for Video Readability
### Node Sizing
- Minimum node width: 150px at 1080p
- Padding inside nodes: 15-20px
- Keep text to 3-5 words per node — use abbreviations if needed
### Edge Labels
- Keep to 1-2 words maximum
- Use edge labels only when the relationship isn't obvious from context
- Prefer labeled nodes over labeled edges
### Layout Direction
- **Top-down (TD):** Best for processes, hierarchies, flows
- **Left-right (LR):** Best for timelines, sequences, pipelines
- Avoid bottom-up (BT) — counterintuitive for most viewers
## Applying to OpenMontage
When using the `diagram_gen` tool:
1. **Check complexity** — max 15-20 nodes at 1080p. Split larger diagrams into multiple frames
2. **Choose theme** to match the video's style playbook and background
3. **Use progressive building** — render stages and crossfade for "building" effect in video
4. **Highlight with classDef** — show the current step in orange/red, completed in green, upcoming in grey
5. **Keep text minimal** — 3-5 words per node, 1-2 words per edge label
6. **Default to flowchart TD** unless the content specifically requires another diagram type
7. **Render at 4K viewport** (3840x2160) even for 1080p output — ensures crisp text when scaled
8. **Test readability** — view the rendered PNG at actual video frame size before composing
skills/creative/enhancement-strategy.md
+111
View File
@@ -0,0 +1,111 @@
# Enhancement Strategy Skill
## When to Use
Apply this skill when deciding how to enhance talking-head footage: which
face/color/audio presets to use, what overlays to add, and how to balance
enhancement visibility with naturalness.
## Enhancement Tools
| Tool | What It Does | Recommended Preset |
|------|-------------|-------------------|
| `face_enhance` | Skin smoothing, sharpening, tone correction | `talking_head_standard` |
| `color_grade` | Cinematic color look with intensity control | `cinematic_warm` at 0.85 |
| `audio_enhance` | Loudness normalization, noise reduction, EQ | `clean_speech` |
| `code_snippet` | Render code as styled overlay image | `monokai` theme |
| `diagram_gen` | Generate box/flow diagrams as overlay images | `dark` theme |
| `image_selector` | AI-generated illustrations (requires API key) | — |
## Enhancement Chain
Apply in this order — each step is optional and gracefully skipped on failure:
```
raw footage
→ subtitle burn (video_compose)
→ face enhance (face_enhance)
→ color grade (color_grade)
→ audio enhance (audio_enhance)
→ final encode (video_compose)
```
### Face Enhancement Presets
| Preset | When to Use |
|--------|-------------|
| `talking_head_standard` | Default for any talking head — smoothing + sharpening + warm |
| `soft_skin` | Webcam footage with visible pores — gentle smoothing |
| `sharpen` | Soft/blurry camera — adds edge definition |
| `brighten` | Dark/underlit footage — lifts shadows and midtones |
| `denoise` | Grainy footage (low light, high ISO) — temporal noise reduction |
### Color Grade Profiles
| Profile | Look | Intensity |
|---------|------|-----------|
| `cinematic_warm` | Warm highlights, lifted shadows, slight saturation | 0.85 |
| `cinematic_cool` | Teal shadows, orange highlights | 0.7 |
| `bright_clean` | Vivid, lifted, YouTube-style | 0.8 |
| `moody_dark` | Crushed blacks, desaturated — dramatic | 0.6 |
| `neutral` | Minimal correction — just normalizes levels | 1.0 |
### Audio Enhancement Presets
| Preset | When to Use | Target |
|--------|-------------|--------|
| `clean_speech` | Default talking head — full processing chain | -16 LUFS |
| `voice_clarity` | Speaker sounds muddy — boosts 3kHz/5kHz presence | -16 LUFS |
| `podcast` | Interview/podcast — heavier compression | -16 LUFS |
| `noise_reduce` | Noisy environment — aggressive FFT denoising | -16 LUFS |
| `normalize_only` | Clean source that just needs loudness matching | -16 LUFS |
## Overlay Enhancement Types
| Type | When to Use | Tool | Placement |
|------|-------------|------|-----------|
| **Text overlay** | Key terms, statistics, quotes | video_compose overlay | Upper or lower third |
| **Code snippet** | Technical content, API examples | code_snippet → overlay | Side of frame or full-screen |
| **Diagram** | Explaining a concept visually | diagram_gen → overlay | Side of frame or full-screen |
| **Lower third** | Speaker name, topic label | video_compose overlay | Bottom 20% of frame |
## Overlay Density Guidelines
### Short-form (< 60 seconds)
- High density: overlay every 3-5 seconds
- Quick visual changes, bold text
- Subtitles **mandatory** (most viewers watch muted)
### Medium-form (1-10 minutes)
- Moderate density: overlay every 10-20 seconds
- Let the speaker carry sections without visual competition
### Long-form (> 10 minutes)
- Low density: overlay every 30-60 seconds
- Only enhance when the content benefits (key points, complex topics)
## Placement Rules
1. **Never cover the speaker's face** — eyes, nose, mouth must remain visible
2. **Subtitles go in the bottom 20%** — margin_v: 50 for vertical, 40 for horizontal
3. **Consistent positioning** — once you place overlays on the left, keep them there
4. **Text overlays: 2-5 seconds on screen** — long enough to read, short enough to not feel stuck
## Deciding What to Enhance
For each section of the script, ask:
1. Is the speaker explaining something visual? → Add a diagram (`diagram_gen`)
2. Is there a key statistic or quote? → Add a text overlay
3. Is there code or technical content? → Add a code screenshot (`code_snippet`)
4. Has the speaker been on camera > 30 seconds straight? → Consider B-roll or overlay
5. Is this the intro or conclusion? → Bold text overlay with the key message
## Quality Checklist
- [ ] Face enhancement looks natural — not over-smoothed or orange
- [ ] Color grade is visible but subtle — skin tones look healthy
- [ ] Audio is normalized to target LUFS — consistent volume throughout
- [ ] Subtitles are readable on mobile, positioned below the face
- [ ] Overlays add value (not just decoration)
- [ ] Enhancement density matches content length and platform
skills/creative/face-restore-usage.md
+96
View File
@@ -0,0 +1,96 @@
# Face Restoration Usage for OpenMontage
> Sources: CodeFormer paper (Zhou et al. 2022), GFPGAN documentation, Real-ESRGAN upsampling docs,
> existing Layer 2 skill at `skills/creative/enhancement-strategy.md`
## Quick Reference Card
```
DEFAULT MODEL: CodeFormer with fidelity 0.5
ALTERNATIVE: GFPGAN (faster, less controllable)
FIDELITY RANGE: 0 = max quality enhancement, 1 = max faithfulness to input
BG UPSAMPLER: Enable to also upscale the background (Real-ESRGAN)
PROCESSING ORDER: face_restore BEFORE face_enhance — restore first, polish second
```
## CRITICAL DISTINCTION — face_restore vs face_enhance
| Tool | What It Does | When to Use |
|------|-------------|-------------|
| `face_enhance` | FFmpeg filter chains — skin smoothing, color balance, sharpening | Good-quality footage that needs polish |
| `face_restore` | AI model reconstruction — rebuilds degraded face detail | Bad-quality footage with blur, compression, low-res faces |
**Decision rule:** If the face is recognizable and just needs polish, use `face_enhance`. If the face is degraded, blurry, or compressed beyond recognition, use `face_restore`.
## Model Selection
| Model | Strengths | Fidelity Control | Speed |
|-------|----------|------------------|-------|
| CodeFormer | Better quality, identity preservation, controllable | Yes (0-1 slider) | Slower |
| GFPGAN | Good baseline, simpler | No | Faster |
### Fidelity Tuning (CodeFormer Only)
| Fidelity | Effect | Use Case |
|----------|--------|----------|
| 0.0 | Maximum enhancement — best visual quality but may alter identity | Unrecognizable faces, artistic use |
| 0.3 | Strong restoration — good for very degraded faces | Old footage, heavy compression artifacts |
| 0.5 | Balanced (default) — restoration + identity preservation | General-purpose restoration |
| 0.7 | Conservative — mild cleanup, strong identity preservation | Webcam footage, light degradation |
| 1.0 | Minimal change — essentially passthrough | Testing, comparison baseline |
## Common Workflows
### 1. Old Footage Restoration
```
face_restore (fidelity 0.3) → color_grade → compose
```
Heavy restoration for archival/vintage footage where faces are significantly degraded.
### 2. Webcam Cleanup
```
face_restore (fidelity 0.7) → face_enhance (talking_head_standard) → compose
```
Light restoration followed by polish — best for modern but low-quality webcam footage.
### 3. Low-Res Face + Background Upscale
```
face_restore (bg_upsampler=true) → compose
```
Single-step restoration when both face and background need improvement.
### 4. Archival Photo for Talking Head
```
face_restore → talking_head tool (SadTalker)
```
Restore the source face image before feeding into the talking-head animation pipeline.
## Quality Checklist
Before accepting face_restore output, verify:
- [ ] Restored face is sharper and cleaner than input
- [ ] Identity is preserved — the person is still recognizable
- [ ] No hallucinated features (extra eyes, wrong skin texture, teeth artifacts)
- [ ] Skin texture looks natural, not plastic/over-smoothed
- [ ] Consistent across frames (for video) — no flickering between restored/unrestored quality
## Applying to OpenMontage
When using the `face_restore` tool:
1. **Use face_restore BEFORE face_enhance** in the processing chain — restore first, polish second
2. **Start with fidelity 0.5** and adjust based on visual inspection
3. **For talking-head pipelines with poor source footage**, apply face_restore in the assets stage
4. **Enable `bg_upsampler` only when both face AND background need improvement**
5. **NEVER use face_restore on already-good footage** — it can introduce subtle artifacts
6. **Compare input and output side-by-side** — the face should be recognizably the same person
7. **For video, extract key frames and test face_restore settings** before processing full video
skills/creative/image-gen-usage.md
+110
View File
@@ -0,0 +1,110 @@
# Image Generation Usage for OpenMontage
> Sources: OpenAI DALL-E 3 documentation, FLUX/BFL API documentation, existing Layer 3 skills
> at `.agents/skills/flux-best-practices/` and `.agents/skills/bfl-api/`
## Quick Reference Card
```
FLUX RESOLUTION: 1920x1088 (16:9) | 1088x1920 (9:16) — must be multiples of 16
MAX TOTAL: 4 megapixels (width x height)
CONSISTENCY: Use hero image as input_image for subsequent frames
STYLE PREFIX: Set in playbook, prepend to every prompt
BATCH STRATEGY: Hero at max quality → iterate with klein → final pass with pro
```
## Resolution for Video Frames
All FLUX dimensions **must be multiples of 16**. Maximum total is 4MP.
| Target | FLUX Resolution | Cost (FLUX.2 pro) |
|--------|----------------|-------------------|
| YouTube 16:9 | `1920x1088` | $0.03/image |
| YouTube 4K | `3840x2160` | Requires pro/max |
| TikTok/Reels 9:16 | `1088x1920` | $0.03/image |
| Square 1:1 | `1024x1024` | $0.03/image |
| Thumbnail | `1280x720` | $0.03/image |
## Maintaining Visual Consistency
The biggest challenge: making 8-12 generated images look like they belong in the same video.
### Strategy 1 — Style Prefix (Always Use)
Prepend the playbook's `image_prompt_prefix` to every prompt. Example from `clean-professional`:
```
"Clean, minimal illustration with soft shadows, muted color palette,
white background, professional vector art style. [YOUR SCENE DESCRIPTION]"
```
### Strategy 2 — Hero Reference Image (Recommended)
1. Generate one "hero" image at maximum quality (`FLUX.2 [max]`, $0.07)
2. Use it as `input_image` for all subsequent frames:
```
Frame 1: T2I with detailed prompt → hero.png
Frame 2: I2I with hero.png + "Same style, camera pans right to show..."
Frame 3: I2I with hero.png + "Same style, zoomed in on..."
```
FLUX.2 supports up to 4 references (klein) or 8 references (pro/max/flex). Reference by number: "The character from image 1 in the environment from image 2."
### Strategy 3 — Seed Locking
Use the same `seed` parameter across generations with similar prompts. Produces similar compositions but is fragile to prompt changes — use as supplement, not primary strategy.
## Prompt Template
```
[STYLE PREFIX from playbook].
[SCENE DESCRIPTION: subject, action, environment].
[LIGHTING: golden hour / overcast / studio softbox / dramatic side-light].
[COMPOSITION: wide shot / medium shot / close-up / overhead / isometric].
[CAMERA: Shot on [camera] with [lens] at [aperture]] (for photorealistic only).
16:9 aspect ratio.
```
### Style-Specific Prompt Patterns
| Style | Prompt Pattern |
|-------|---------------|
| **Flat illustration** | "Flat vector illustration, bold colors, clean edges, no gradients, white background" |
| **Isometric** | "Isometric 3D illustration, 30-degree angle, clean geometric shapes, soft shadows" |
| **Photorealistic** | "Photorealistic, shot on Canon EOS R5 with 85mm f/1.4, shallow depth of field" |
| **Diagram-style** | "Technical diagram, labeled components, clean lines, minimal color, white background" |
| **Watercolor** | "Soft watercolor illustration, muted tones, visible brush strokes, paper texture" |
## Batch Generation Strategy
| Phase | Model | Cost/Image | Purpose |
|-------|-------|-----------|---------|
| 1. Style guide | FLUX.2 [max] | $0.07 | One hero image, maximum quality |
| 2. Storyboard iteration | FLUX.2 [klein] 9B | $0.015 | Rapid variations during planning |
| 3. Final frames | FLUX.2 [pro] | $0.03 | Re-generate finals with hero as reference |
**Rate limit:** 24 concurrent requests max. Pipeline accordingly.
**Budget for 8-image explainer:** $0.07 (hero) + $0.12 (8x klein iterations) + $0.24 (8x pro finals) = ~$0.43
## Common Pitfalls
1. **Text in images** — AI image generators are unreliable with text. Never include text in prompts; add text as overlays in the compose stage
2. **Hands and fingers** — DALL-E 3 and FLUX still struggle. Avoid prompts requiring detailed hand poses
3. **Inconsistent characters** — Without reference images, the same character will look different each time. Always use the hero reference strategy
4. **Over-prompting** — Long, complex prompts produce unpredictable results. Keep to 2-3 sentences
5. **Ignoring the playbook** — Every image must match the style playbook. The style prefix is not optional
## Applying to OpenMontage
When using the `image_selector` tool in the asset stage:
1. **Always prepend the playbook's style prefix** to every prompt
2. **Generate a hero image first** at highest quality, use as reference for all others
3. **Use `1920x1088`** for 16:9 video frames (FLUX multiple-of-16 requirement)
4. **Never request text in images** — add text overlays in the compose stage
5. **Budget check** — estimate total image cost before generating; switch to local diffusers if over budget
6. **Iterate with klein** during planning, finalize with pro
7. **Keep prompts to 2-3 sentences** — style prefix + scene description + composition
8. **Match the scene plan** — each image maps to a specific scene in the script
skills/creative/image-provider-usage.md
+111
View File
@@ -0,0 +1,111 @@
# Image Provider Usage for OpenMontage
> How to choose between image generation and stock providers, and how to use each effectively.
> Supplements the existing `image-gen-usage.md` (which covers FLUX prompting in depth).
## Provider Landscape
### Generation Providers (AI creates the image)
| Tool | Provider | Cost | Speed | Best For |
|------|----------|------|-------|----------|
| `flux_image` | FLUX 2 Pro via fal.ai | ~$0.03-0.05 | ~5-10s | Photorealism, general purpose, workhorse |
| `openai_image` | GPT Image 1 (OpenAI) | ~$0.01-0.17 | ~5-15s | Complex instructions, text in images, multi-element |
| `recraft_image` | Recraft V4 via fal.ai | ~$0.04-0.25 | ~5-10s | Logos, SVG vectors, brand assets, text rendering |
| `local_diffusion` | Stable Diffusion (local) | Free | ~30s+ | Offline, privacy, free |
| `image_gen` | Multi (legacy, deprecated) | Varies | Varies | **Deprecated** — use `image_selector` or per-provider tools |
### Stock Providers (search and download existing images)
| Tool | Provider | Cost | Speed | Best For |
|------|----------|------|-------|----------|
| `pexels_image` | Pexels | Free | ~2-5s | High-quality photography, color filtering |
| `pixabay_image` | Pixabay | Free | ~2-5s | Large library, category filtering, illustrations |
### Selector
| Tool | Purpose |
|------|---------|
| `image_selector` | Routes to the best available provider based on preference and availability |
## Provider Selection by Scene Type
| Scene Type | Primary Provider | Why | Fallback |
|-----------|-----------------|-----|----------|
| **Real-world photo** (city, nature, people) | `pexels_image` | Real photos > AI for realism | `pixabay_image` → `flux_image` |
| **Technical diagram** | `diagram_gen` | Structured, editable | `flux_image` with diagram prompt |
| **Abstract/conceptual illustration** | `flux_image` | AI excels at custom concepts | `openai_image` |
| **Logo or brand asset** | `recraft_image` | SVG support, text accuracy | `openai_image` |
| **Image with text/labels** | `openai_image` | Best text rendering (GPT Image 1) | `recraft_image` |
| **Complex multi-element composition** | `openai_image` | Best instruction following | `flux_image` |
| **Hero image (key visual)** | `flux_image` | Highest visual quality | `openai_image` |
| **Thumbnail** | `flux_image` or `recraft_image` | Needs to be eye-catching | — |
| **Budget/free project** | `pexels_image` or `pixabay_image` | Free, immediate | `local_diffusion` |
| **Offline/air-gapped** | `local_diffusion` | No network needed | — |
## Cost-Quality Tradeoff
```
PRODUCTION PATH: Premium
├── Hero images: flux_image ($0.05/img)
├── Supporting visuals: flux_image ($0.03/img)
├── Text overlays: openai_image ($0.04/img)
├── B-roll stills: pexels_image ($0.00)
└── Total for 10 images: ~$0.35
PRODUCTION PATH: Standard
├── All generated: flux_image ($0.03/img)
├── B-roll stills: pexels_image ($0.00)
└── Total for 10 images: ~$0.25
PRODUCTION PATH: Budget
├── All stock: pexels_image + pixabay_image ($0.00)
├── Diagrams: diagram_gen ($0.00)
└── Total: $0.00
PRODUCTION PATH: Offline
├── All generated: local_diffusion ($0.00)
├── Diagrams: diagram_gen ($0.00)
└── Total: $0.00 (but slower, lower quality)
```
## Using the Image Selector
For most cases, use `image_selector` and let it route:
```python
# The selector finds the best available provider
result = image_selector.execute({
"prompt": "aerial view of a modern data center",
"preferred_provider": "auto", # or "flux", "pexels", etc.
"output_path": "assets/images/scene-3.png"
})
```
Override with `preferred_provider` when you know which provider is best for the scene type.
Use `allowed_providers` to restrict to free or local options:
```python
# Budget mode: only free providers
result = image_selector.execute({
"prompt": "server room interior",
"allowed_providers": ["pexels", "pixabay", "local_diffusion"],
"output_path": "assets/images/scene-3.jpg"
})
```
## Consistency Across Mixed Sources
When mixing stock and generated images in the same video, visual consistency is the challenge.
### Strategy: Color Grade Everything
Apply the playbook's color grading LUT to both stock and generated images in the compose stage.
This unifies the look. The `color_grade` enhancement tool handles this.
### Strategy: Match Playbook Style in Prompts
When generating images, always prepend the playbook's `image_prompt_prefix`. When searching stock,
use the playbook's color names in search filters (Pexels supports color filtering).
### Strategy: Avoid Mixing Styles Within a Scene
Don't use a stock photo for one element and an AI illustration for another in the same scene.
Keep each scene internally consistent — all stock or all generated.
skills/creative/lip-sync-usage.md
+133
View File
@@ -0,0 +1,133 @@
# Lip Sync Usage for OpenMontage
> Sources: Wav2Lip paper (Prajwal et al. 2020), Wav2Lip-GAN documentation, OpenMontage
> `tools/lip_sync.py` implementation
## Quick Reference Card
```
DEFAULT MODEL: wav2lip (faster, good sync accuracy)
HIGHER QUALITY: wav2lip_gan (better visual quality, slower)
FACE PADDING: [0, 10, 0, 0] (top, bottom, left, right)
INPUT: Video with visible face + audio to sync to
RESIZE FACTOR: 1 = full res (best), 2 = half res (recommended for drafts)
KEY RULE: Use lip_sync for VIDEO input; use talking_head for PHOTO input
```
## When to Use lip_sync
Lip sync is a **post-production** step. Use it after generating replacement audio.
- **Dubbing / localization** -- replace original speech with translated audio and match lips
- **Audio replacement** -- re-record narration and sync to existing video
- **Voice-over correction** -- fix mismatched audio/video timing
- **NOT for photo-to-video** -- use the `talking_head` tool instead
## CRITICAL DISTINCTION -- lip_sync vs talking_head
| | `lip_sync` | `talking_head` |
|---|---|---|
| **Input** | Existing VIDEO + new audio | Still PHOTO + audio |
| **Output** | Video with synced lips | New video from photo |
| **Use Case** | Dubbing, audio replacement | Avatar generation, spokesperson |
**Decision rule:** If you already have video footage of the person speaking, use `lip_sync`. If you only have a photograph and want to make it talk, use `talking_head`.
## Model Selection Guide
| Model | Quality | Speed | Best For |
|-------|---------|-------|----------|
| `wav2lip` | Good lip sync, may blur chin | Faster | Quick dubbing, drafts |
| `wav2lip_gan` | Better visual quality around mouth | Slower | Final renders, close-ups |
**Decision rule:** Use `wav2lip` for iteration and drafts. Switch to `wav2lip_gan` for final renders or any shot where the face is prominent (close-ups, medium shots). The quality difference is most visible in the mouth and chin region.
## Input Requirements
- Video must contain a clearly visible face throughout
- Face should be front-facing or at most 30-degree angle
- Minimum face size: ~100px across
- Audio should be clean speech (not music or noise)
- Audio length should roughly match video length (within 10%)
## Face Padding
Face padding controls how much area around the detected face is included in the sync region. Format: `[top, bottom, left, right]`.
| Scenario | Padding | Reason |
|----------|---------|--------|
| Default talking-head shot | `[0, 10, 0, 0]` | Works for 90% of footage |
| Chin being cut off | Increase index 1 (bottom) | Extends the mask below the chin |
| Forehead getting cropped | Increase index 0 (top) | Extends the mask above the forehead |
| Face off-center in frame | Adjust indices 2, 3 (left, right) | Compensates for lateral offset |
Keep left/right at 0 unless the face is noticeably off-center in the frame.
## Resize Factor
| Value | Resolution | Quality | Speed | Use Case |
|-------|-----------|---------|-------|----------|
| 1 | Full | Best | Slowest | Final renders |
| 2 | Half | Good | Faster | Drafts, iteration |
| 3+ | Reduced | Degraded | Fastest | Quick previews only |
**Recommendation:** Use `resize_factor=2` during iteration, `resize_factor=1` for final output.
## Common Workflows
### 1. Localization Dubbing
Translate a video into another language with matched lip movements.
```
transcriber(video) --> transcript
--> translate script to target language
--> tts_selector(translated_script, target_language_voice)
--> lip_sync(original_video, translated_audio)
```
### 2. Audio Re-Record
Replace narration audio and re-sync the speaker's lips.
```
new_audio_recording
--> lip_sync(original_video, new_audio)
--> face_enhance (post-sync cleanup)
--> compose
```
### 3. Multi-Language Output
Produce multiple language versions from a single source video.
```
source_video
--> lip_sync(source_video, english_audio) --> english_output
--> lip_sync(source_video, spanish_audio) --> spanish_output
--> lip_sync(source_video, french_audio) --> french_output
```
Keep the original video as the source for each language -- do not chain lip_sync outputs.
## Quality Checklist
Before moving to the compose stage, verify each lip_sync output:
- [ ] **Lip movements match the new audio naturally** -- no desync or lag
- [ ] **No visual artifacts around the mouth/chin area** -- no blurring, smearing, or color mismatch
- [ ] **Face region blends seamlessly with the rest of the frame** -- no visible boundary
- [ ] **No temporal flickering between frames** -- smooth frame-to-frame transitions
- [ ] **Audio-visual sync is tight** -- no perceptible delay between mouth movement and sound
## Applying to OpenMontage
When using the `lip_sync` tool in post-production:
1. **Generate the replacement audio FIRST** (`tts_selector`, `elevenlabs_tts`, `openai_tts`, or `piper_tts`), then lip sync -- lip_sync requires finished audio as input
2. **Use `wav2lip` for drafts and iteration, `wav2lip_gan` for final renders** -- save processing time during the creative loop
3. **Apply `face_enhance` AFTER lip_sync, not before** -- lip_sync modifies the face region, so enhancing before sync is wasted work
4. **For localization workflows, keep the original video as source** and sync each language separately -- never chain lip_sync outputs
5. **Check that audio length matches video length before syncing** -- trim or pad audio if needed to stay within 10% of video duration
6. **Face padding `[0, 10, 0, 0]` works for 90% of talking-head footage** -- only adjust if you see cropping artifacts
7. **For close-up shots, always use `wav2lip_gan`** -- the quality difference is visible at this framing
skills/creative/long-form.md
+239
View File
@@ -0,0 +1,239 @@
# Long-Form Video Pipeline (10+ Minutes)
> Sources: YouTube Creator Academy, VidIQ analytics research, Think Media production guides,
> Paddy Galloway retention analytics, Retention Rabbit 2025 Benchmark Report, AIR Media-Tech
> retention editing guide, Epidemic Sound mixing guide, Sweetwater YouTube mastering
## Quick Reference Card
```
DURATION: 8-15 min (sweet spot for most topics)
HOOK: Complete by 0:30 — survive the 30-second cliff
PATTERN INTERRUPT: Every 45-90 seconds
RETENTION TARGET: 40-60% average view duration
CHAPTER LENGTH: 2-4 minutes per chapter
NARRATION: 150-160 WPM
MUSIC BED: Continuous, ducked 18-20 dB below speech
TARGET LUFS: -14 LUFS integrated
END SCREEN: Last 20 seconds (YouTube end screen cards)
```
## Retention Benchmarks (2025-2026 Data)
| Video Duration | Good Retention | Excellent Retention |
|---------------|---------------|-------------------|
| 1-3 min | 60%+ | 75%+ |
| 3-5 min | 50%+ | 65%+ |
| 5-10 min | 45%+ | 60%+ |
| **10-20 min** | **40%+** | **55%+** |
| 20-60 min | 35%+ | 50%+ |
- Platform average: **23.7%** across all YouTube videos
- Only **16.8%** of videos exceed 50% retention
- Only **16%** of viewers reach the final 10 seconds
- **Improving retention by 10 percentage points** correlates with 25%+ increase in impressions
### AI-Generated Content Warning
- AI-generated video shows **70% lower retention** vs human-fronted content
- AI narration triggers **35% viewer drop-off** within the first 45 seconds vs human narration
- **Implication for OpenMontage:** Prioritize natural-sounding TTS (ElevenLabs over Piper), and avoid detectable AI visual artifacts. The processing chain in `sound-design.md` is essential.
## Retention Curve Management
### The Critical Points
| Timestamp | What Happens | How to Survive |
|-----------|-------------|----------------|
| 0:00-0:03 | Thumbnail-to-video match | First frame must match thumbnail promise |
| 0:00-0:30 | **55%+ leave in first 60s** | Hook + tension must be complete by 0:30. Must retain 70%+ here. |
| 2:00-3:00 | **Retention valley** — initial curiosity spent | Deliver first major payoff BEFORE 2:00, pattern interrupt at 1:45 |
| 55-65% mark | **Secondary exodus** in long-form | Re-engage with burst sequence + open loop resolution |
| Last 20s | End screen opportunity | CTA + end screen cards |
### Survival Tactics for the 2-3 Minute Valley
1. **Open loops in first 60 seconds** — raise a question early, hold the answer until later
2. **First major payoff before 2:00** — the hook's promise must have a down-payment
3. **Pattern interrupt at 1:45-2:00** — camera angle shift, B-roll burst, music change
4. **"Burst sequence" at the valley** — 5-10 quick cuts lasting 10-15 seconds, then return to calm
5. **Foreshadowing cue** — "But the really surprising part is coming up in a minute"
### Pattern Interrupts
Deploy **major interrupts** every **60-90 seconds** and **minor interrupts** every **20-30 seconds**:
| Technique | Type | When to Use |
|-----------|------|-------------|
| B-roll cut | Minor | Every 30-60s of talking head |
| Visual style change | Major | New section, new concept |
| On-screen text/graphic | Minor | Key stat, definition, emphasis |
| Music energy shift | Major | Section transitions |
| Direct address | Minor | "Now here's what's interesting..." |
| Burst sequence (5-10 rapid cuts) | Major | Every 2-3 minutes |
| Sound effect | Minor | Transition whoosh, pop for text |
**Impact:** Videos using pattern interrupts in the first 5 seconds achieve **23% higher average retention**.
### Re-Engagement Hooks
Place a **re-hook** at the 2-minute mark and every 3-4 minutes after:
```
"But that's not even the interesting part..."
"Now here's where it gets weird..."
"Most people stop here, but if you keep watching..."
"This next part changes everything..."
```
These verbal signposts give viewers a reason to stay through the next segment.
## Content Structure
### Chapter Template
```
[INTRO] 0:00 - 0:30 Hook + stakes + preview
[CHAPTER 1] 0:30 - 3:00 Foundation concept
[RE-HOOK] 3:00 - 3:15 Curiosity gap for next section
[CHAPTER 2] 3:15 - 6:00 Complication / deeper layer
[PALETTE CLEANSER] 6:00 - 6:15 Visual break, humor, or "let that sink in"
[CHAPTER 3] 6:15 - 9:00 Key insight / "aha" moment
[PROOF] 9:00 - 10:30 Demonstration / example
[CONCLUSION] 10:30 - 11:30 Implications + reframe
[OUTRO] 11:30 - 12:00 CTA + end screen
```
### Chapter Length Rules
| Chapter Content | Ideal Length | Notes |
|----------------|-------------|-------|
| Simple concept | 2-3 minutes | One idea, one visual set |
| Complex concept | 3-4 minutes | Multi-step, needs examples |
| Demonstration | 2-3 minutes | Show, don't just tell |
| Story / narrative | 3-5 minutes | Needs setup + payoff |
**Max 5-6 chapters** for a 10-15 minute video. More chapters = too fragmented.
### YouTube Chapters (Timestamps)
Add chapter markers in the description:
```
0:00 Introduction
0:30 Why This Matters
3:15 The Key Mechanism
6:15 The Breakthrough
9:00 Real-World Example
10:30 What This Means For You
```
Chapters improve navigation and can boost retention by letting viewers skip to relevant sections.
## Audio Consistency
### Music Bed Management
| Rule | Value |
|------|-------|
| Music presence | Continuous throughout (no silent gaps) |
| Ducking during speech | -18 to -20 dB below narration |
| Music transitions | 2-3 second crossfade between sections |
| Energy matching | Shift music energy at chapter boundaries |
| BPM consistency | Stay within ±10 BPM across the video |
### LUFS Over Long Duration
- Target: **-14 LUFS integrated** (YouTube standard)
- Dynamic range: **6-12 dB** for speech-heavy content
- Check LUFS per chapter — variation between chapters should be < 2 LUFS
- Use a limiter at **-1.5 dBTP** on the final mix
### Narration Pacing
| Section | WPM | Energy |
|---------|-----|--------|
| Hook | 160-170 | High energy, urgent |
| Explanation | 150-160 | Steady, clear |
| Key insight | 140-150 | Slower, deliberate |
| Silence after reveal | 0 WPM (1-3s pause) | Let it land |
| Conclusion | 155-165 | Energized, resolved |
## Visual Pacing
### Cut Frequency by Video Phase
| Phase | Timing | Cut Interval | Notes |
|-------|--------|-------------|-------|
| Hook | 0:00-0:30 | Every 3-5s | Rapid changes signal momentum |
| Early body | 0:30-3:00 | Every 10-15s | High energy, frequent B-roll |
| Mid body | 3:00-7:00 | Every 15-25s | Stabilize; fewer cuts, more contextual B-roll |
| Late body | 8:00+ | 15-25s calm + burst sequences | Alternate calm with 5-10 quick-cut bursts every 2-3 min |
### B-Roll Strategy
- **Individual B-roll clip length:** 5-8 seconds
- **B-roll as percentage of total video:** 35-50% for educational content
- **Watch time impact:** Strategic B-roll at 35-50% increases watch time by **15-25%**
- **Shot absorption time:** Viewers need ~3 seconds; beyond 5 seconds without change, attention fades
### The "Something Must Happen" Rule
| Rule | Value |
|------|-------|
| Visual/audio change | Every 3-5 seconds |
| Substantive frame change | Every 20-30 seconds |
| Max without any change | 15 seconds (expect drop-off beyond this) |
## End Screen & Cards
### End Screen (Last 20 Seconds)
- YouTube allows end screen elements in the **last 5-20 seconds**
- Include: subscribe button, next video recommendation, playlist link
- **Do NOT put critical content in the last 20 seconds** — it gets covered
- Verbal CTA: "If you found this helpful, check out this next video on..."
### Info Cards
- Place at moments when a related topic is mentioned
- Max 1 card per 2 minutes — too many feels spammy
- Best placement: when you reference a concept covered in another video
## Applying to OpenMontage
When building long-form content:
1. **Structure with chapters** — 2-4 minutes each, max 5-6 chapters
2. **Complete the hook by 0:30** — follow the storytelling.md Explainer Arc template
3. **Re-hook at 2:00-3:00** — this is the retention valley
4. **Pattern interrupt every 45-90 seconds** — B-roll, text overlay, visual change
5. **Continuous music bed** — use `music_gen` for full-length track, duck 18-20 dB
6. **Narrate at 150-160 WPM** — slower than short-form, clearer for learning
7. **Check LUFS per chapter** — should be consistent (< 2 LUFS variation)
8. **Reserve last 20 seconds** for end screen — no essential content there
9. **Add chapter timestamps** — include in publish stage metadata
10. **Target 40-60% average view duration** — if retention drops below 30% at any point, that section needs a pattern interrupt
## Timing Cheat Sheet (12-Minute Video)
```
0:00-0:03 Visual hook (most compelling shot)
0:03-0:08 Verbal hook (promise/question)
0:08-0:15 Stakes ("here's why this matters")
0:15-0:30 Value preview + open loop planted
0:30-0:35 Branded intro (5 sec max)
0:35-1:45 Body segment 1 (high energy, cuts every 10-15s)
1:45-2:00 Pattern interrupt to bridge retention valley
2:00-3:00 First major payoff delivered
3:00-3:05 Chapter 2 mini-hook + bridging sentence
3:00-5:30 Body segment 2 (stabilized pacing, 15-25s cuts)
~5:00 Mid-roll CTA (subscribe ask, after earning value)
5:30-8:00 Body segment 3 (B-roll heavy, callbacks)
7:00-7:15 Burst sequence (5-10 quick cuts to re-engage)
8:00-10:00 Body segment 4 (mix calm + energy bursts)
9:30 Open loop resolution / major callback payoff
10:00-11:20 Final segment + main reveal
11:00 Card placement (last 20% of video)
11:20-11:40 Outro: tease next content, do NOT say goodbye
11:40-12:00 End screen (last 20 seconds), 1-2 elements
```
skills/creative/manim-usage.md
+102
View File
@@ -0,0 +1,102 @@
# ManimCE Usage for OpenMontage
> Sources: ManimCE documentation, 3Blue1Brown FAQ/conventions, Theorem of Beethoven tutorials,
> existing Layer 3 skill at `.agents/skills/manimce-best-practices/`
## Quick Reference Card
```
RENDER QUALITY: -qh (1080p60) for YouTube | -qm (720p30) for drafts
BACKGROUND: Dark (#1a1a2e or BLACK)
MAX ELEMENTS: 3-4 new visual elements revealed simultaneously
PACING: One concept per scene, build incrementally
EQUATION WRITE: 1.5-2.0s run_time
SHAPE CREATE: 0.8-1.2s run_time
WAIT AFTER: 1.0-2.0s (longer for complex equations)
2D vs 3D: Default to 2D. 3D only when spatial relationship IS the concept.
```
## Render Settings for OpenMontage
| Flag | Resolution | FPS | Use Case |
|------|-----------|-----|----------|
| `-ql` | 480x360 | 15 | Development/testing |
| `-qm` | 1280x720 | 30 | Draft review |
| `-qh` | 1920x1080 | 60 | Standard YouTube upload |
| `-qp` | 2560x1440 | 60 | High-quality export |
| `-qk` | 3840x2160 | 60 | 4K archival/premium |
For OpenMontage's YouTube landscape profile (1920x1080/30fps), render at `-qh` and transcode to 30fps, or set custom config:
```ini
[CLI]
pixel_width = 1920
pixel_height = 1080
frame_rate = 30
```
## Animation Timing
| Animation Type | `run_time` | Rate Function | Notes |
|---------------|-----------|---------------|-------|
| Equation write (`Write`) | 1.5-2.0s | `smooth` (default) | Give viewers time to parse LaTeX |
| Equation transform | 1.5s | `smooth` | Use `TransformMatchingTex` for derivations |
| Shape creation (`Create`) | 0.8-1.2s | `smooth` | `Create()` or `DrawBorderThenFill()` |
| Color highlight | 0.5s | `smooth` | Brief attention call |
| Camera zoom | 1.5-2.0s | `ease_in_out_cubic` | Smooth entry/exit |
| Staggered reveals | `lag_ratio=0.1-0.2` | — | `LaggedStart` for grid/list reveals |
| Wait after reveal | 1.0-2.0s | — | Longer for complex equations |
| Fast cut / punctuation | 0.3-0.5s | `rush_from` | Between concepts |
## Scene Composition
### Pacing Rule (3Blue1Brown Convention)
- **One concept per scene** — build incrementally
- Show the simple version first, then `Transform` it into the complex version
- Never reveal more than **3-4 new visual elements** simultaneously
- Use `self.wait(1.5)` after every major reveal
### 2D vs 3D Decision
**Use 2D** (`Scene` or `MovingCameraScene`) for:
- Equation derivations, graph plots, number lines, matrices
- 2D vector spaces (even for "high dimensions" — project down)
- State diagrams, flowcharts, timelines
**Use 3D** (`ThreeDScene`) only when:
- Visualizing surfaces (`z = f(x,y)`), volumes, or 3D vector fields
- The spatial relationship IS the concept (cross products, surface normals)
- You need camera orbit to reveal hidden structure
**Performance:** 3D uses CPU-only Cairo rendering — 5-10x slower than 2D.
## Color Usage
| Semantic Role | Color | Manim Constant |
|--------------|-------|----------------|
| Variable being solved | Yellow | `YELLOW` |
| Matrix / operator | Red | `RED` |
| Eigenvector / result | Teal | `TEAL` |
| Known constant | Blue | `BLUE_C` |
| Annotation / label | Green | `GREEN` |
| De-emphasis / background | Grey 50% | `GREY`, `opacity=0.5` |
| Error / wrong path | Dark red | `RED_E` |
**Accessibility:** Avoid red-green only distinctions. Use brightness variation (`_A` through `_E` shades) alongside hue changes.
**Background:** Always use dark backgrounds (`BLACK` or `#1a1a2e`) for video output.
## Applying to OpenMontage
When using the `math_animate` tool:
1. **Render at `-qh`** (1080p60) for final output, `-qm` for drafts
2. **One concept per scene** — break complex proofs into multiple Manim scenes
3. **Use timing table above** — don't rush equations (1.5-2.0s for writes)
4. **Wait after reveals** — `self.wait(1.5)` minimum after key insights
5. **Dark background** — set `background_color=BLACK` in config
6. **Use color semantically** — yellow for unknowns, blue for knowns, red for operators
7. **Default to 2D** — only use `ThreeDScene` when 3D is essential to understanding
8. **Stagger complex reveals** — `LaggedStart` with `lag_ratio=0.15` for lists/grids
9. **Sync to narration** — the scene's total duration should match the narration segment timing from the script
skills/creative/music-gen-usage.md
+135
View File
@@ -0,0 +1,135 @@
# Music Generation Usage for OpenMontage
> Sources: ElevenLabs Music API documentation, ElevenLabs best practices guide, Artlist BPM
> guide, existing Layer 3 skills at `.agents/skills/music/` and `.agents/skills/elevenlabs/`
## Quick Reference Card
```
API MODEL: music_v1
MIN DURATION: 3,000ms (3s)
MAX DURATION: 600,000ms (10 min)
INSTRUMENTAL: Always set force_instrumental=true for video background
COST: ~$0.05 per 30 seconds
KEY RULE: Music must be 18-20 dB below narration (see sound-design.md)
```
## BPM Selection by Video Type
| Video Type | BPM Range | Prompt Fragment |
|-----------|-----------|-----------------|
| Educational explainer | 80-100 | "gentle ambient electronic, 90 BPM" |
| Corporate / tech | 100-120 | "upbeat corporate pop, 110 BPM, positive" |
| Epic / dramatic reveal | 60-80 | "cinematic orchestral, 70 BPM, building tension" |
| Fast-paced montage | 120-140 | "energetic electronic, 130 BPM, driving beat" |
| Meditation / calm | 50-70 | "ambient drone, 60 BPM, peaceful" |
| Comedy / lighthearted | 100-130 | "playful ukulele pop, 120 BPM, whimsical" |
| Sad / reflective | 60-80 | "melancholic piano, 65 BPM, minor key" |
| Action / hype | 140-170 | "high-intensity drum and bass, 160 BPM" |
## Key and Mood Mapping
| Mood | Key | Musical Characteristics |
|------|-----|----------------------|
| Happy / upbeat | C major, G major | Bright, resolved, energetic |
| Serious / professional | D minor, A minor | Grounded, authoritative |
| Mysterious / curious | E minor, B minor | Tension, anticipation |
| Triumphant / inspiring | D major, Bb major | Expansive, climactic |
| Melancholic / thoughtful | F minor, C minor | Reflective, emotional |
| Neutral / ambient | C major, Am (no strong key) | Unobtrusive, background |
## Prompt Engineering
### Structure
```
[GENRE/STYLE], [BPM], [KEY/MOOD], [INSTRUMENTS], [ENERGY LEVEL], [PURPOSE]
```
### Examples
**Educational explainer:**
```
Gentle lo-fi ambient electronic, 90 BPM, C major, soft synth pads and light
percussion, calm and steady energy, background music for narration
```
**Corporate product demo:**
```
Modern upbeat corporate pop, 110 BPM, G major, acoustic guitar and light drums,
positive energy building gradually, underscore for product walkthrough
```
**Technical deep-dive:**
```
Minimal ambient electronic, 80 BPM, A minor, soft Rhodes piano and subtle
bass, contemplative and focused, background music for technical explanation
```
### Key Prompting Rules
1. **Always include "background" or "underscore"** — tells the model to stay dynamically even
2. **Always use `force_instrumental=true`** — lyrics compete with narration
3. **Specify BPM explicitly** — don't rely on genre to set tempo
4. **Avoid "bright hi-hats" or "prominent vocals"** — high-frequency busy elements compete with speech in the 2-4 kHz intelligibility band
5. **Include energy direction** — "steady energy" for explainers, "building gradually" for reveals
## Duration Matching
### Exact Duration
```python
result = music_gen.execute({
"prompt": "Gentle ambient, 90 BPM, background underscore",
"duration_seconds": 150, # Match video length
"output_path": "assets/music/background.mp3"
})
```
### Section-Mapped (Advanced)
For videos with distinct acts, generate sections separately:
| Video Section | Duration | Music Style |
|--------------|----------|-------------|
| Intro / hook | 8-10s | Soft, building |
| Main explanation | 90-120s | Steady, neutral |
| Key reveal | 20-30s | Intensified, fuller |
| Outro | 10-15s | Fading, gentle |
Generate each as a separate track and crossfade in the `audio_mixer`.
## Looping for Long Videos
For videos longer than the generated track:
1. Generate a track 30-60% of video length
2. Use FFmpeg to create a seamless loop:
```bash
ffmpeg -stream_loop 2 -i music.mp3 -c copy music_looped.mp3
```
3. Add a 2-3 second crossfade at loop points in `audio_mixer`
**Better approach:** Generate at the exact video duration. ElevenLabs supports up to 10 minutes per generation.
## Stem Isolation
For cleaner ducking control, generate isolated stems:
- `"solo electric guitar in E minor, 90 BPM"` — guitar-only track
- `"soft ambient pad in C major, 80 BPM"` — synth pad only
- Layer stems in FFmpeg during composition for precise ducking control
## Applying to OpenMontage
When using the `music_gen` tool:
1. **Match BPM to content type** using the table above — don't default to a generic prompt
2. **Always set `force_instrumental=true`** — no lyrics under narration
3. **Include "background" or "underscore"** in every prompt
4. **Set duration to match video length** — avoid looping when possible
5. **Budget check** — at $0.05/30s, a 3-minute video costs ~$0.30 for music
6. **Duck music 18-20 dB below narration** — see `skills/creative/sound-design.md` for ducking rules
7. **Cut 2-4 kHz on the music bed** in `audio_mixer` to clear the speech intelligibility band
8. **Test on phone speakers** — if narration disappears behind music, duck more aggressively
9. **One track per video** — avoid switching music styles mid-video unless there's a clear narrative shift
skills/creative/prompting/hunyuan-prompting.md
+88
View File
@@ -0,0 +1,88 @@
# HunyuanVideo 1.5 — Prompting Guide
> Source: [Tencent Prompt Handbook](https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/main/assets/HunyuanVideo_1_5_Prompt_Handbook_EN.md)
> For universal vocabulary, see: `skills/creative/video-gen-prompting.md`
## HunyuanVideo Prompt Formula
### Text-to-Video
```
Subject + Motion + Scene + [Shot Type] + [Camera Movement] + [Lighting] + [Style] + [Atmosphere]
```
### Image-to-Video
```
Subject Motion Dynamics + Scene Motion Dynamics + [Camera Movement]
```
For I2V, focus on describing MOTION, not appearance (the image provides appearance).
## HunyuanVideo-Specific Strengths
### Lighting as Atmosphere
Tencent emphasizes: **"Light is the soul of atmosphere."**
Describe lighting with multiple dimensions:
- **Style**: soft, hard, neon, ambient
- **Direction**: side-lit, backlit, overhead, underlighting
- **Quality**: harsh spotlight, diffuse glow
- **Shadows**: long dramatic shadows, soft shadow edges
- **Color temperature**: golden hour warmth, cool daylight blue
- **Reflections**: wet surface reflections, metallic glints
### Camera Movement Library
| Movement | Type | HunyuanVideo Prompt |
|----------|------|-------------------|
| Crane / Pedestal | Vertical | "camera rises vertically" |
| Truck / Tracking | Horizontal | "camera tracks left alongside subject" |
| Dolly In | Push | "camera pushes forward toward subject" |
| Dolly Out | Pull | "camera pulls back from subject" |
| Pan | Rotation | "camera pans right across the scene" |
| Orbit | Circular | "camera orbits around subject" |
| Follow | Lock-on | "camera follows subject from behind" |
| Static | Fixed | "static camera, no movement" |
### Style Keywords
**Photorealistic / Cinematic**:
- Film noir, hard sci-fi, cinematic photography
- Period drama, war documentary, nature documentary
**Animation / Illustration**:
- 2D animation, Japanese anime
- Watercolor painting, Chinese ink wash
- Low-poly 3D, pixel art
## I2V Best Practice
When using image-to-video, the input image defines appearance. Your prompt should ONLY describe:
1. How the subject moves
2. How the environment changes
3. Camera motion
**Good I2V prompt**: "The woman's hair blows in the wind as she turns to face the camera. Leaves scatter across the path. Camera slowly dollies in."
**Bad I2V prompt**: "A beautiful woman in a red dress standing in a forest" — this repeats what the image already shows.
## Example (T2V)
```
A young woman in a flowing white dress walks barefoot along
a deserted beach at golden hour. She trails her hand through
the shallow surf, leaving ripples. Her hair catches the warm
side-light from the setting sun. Medium tracking shot, camera
follows alongside at knee height. Soft golden lighting with
long shadows stretching toward the camera. Cinematic
photography style, shallow depth of field. Peaceful,
contemplative atmosphere.
```
## Example (I2V)
```
The cat stretches lazily, then leaps from the windowsill
to the floor. Dust motes scatter in the shaft of light.
Camera remains static, slight rack focus from window to
landing spot.
```
skills/creative/prompting/ltx-prompting.md
+77
View File
@@ -0,0 +1,77 @@
# LTX-2 — Prompting Guide
> Source: [LTX Official Prompting Guide](https://docs.ltx.video/api-documentation/prompting-guide)
> For universal vocabulary, see: `skills/creative/video-gen-prompting.md`
## LTX-Specific 6-Element Structure
LTX-2 uses a clean, focused prompt structure:
1. **Establish the shot** — cinematography terms matching your genre
2. **Set the scene** — lighting, color palette, textures, atmosphere
3. **Describe the action** — natural sequence flowing from beginning to end
4. **Define the character(s)** — physical cues (age, hair, clothes), not abstract labels
5. **Camera movement(s)** — specify how and when; describe what appears AFTER the movement
6. **Describe the audio** — ambient sound, music, speech, or singing
## LTX-Specific Tips
### Post-Movement Description
LTX renders camera movements more accurately when you describe the result:
- Instead of: "Camera pans left"
- Write: "Camera pans left to reveal a bustling market square"
### Audio Prompting (Unique to LTX-2)
LTX-2 generates synchronized audio. Use specific descriptors:
| Category | Examples |
|----------|---------|
| **Ambient** | "coffeeshop noise", "wind and rain", "forest with birdsong" |
| **Voice style** | "energetic announcer", "resonant voice with gravitas", "childlike curiosity" |
| **Volume** | "whisper", "mutter", "shout", "scream" |
| **Music** | "soft acoustic guitar", "electronic beat building" |
Dialogue goes in quotes: `The narrator says: "Welcome to the future."`
Specify language/accent: `speaks in British English with a warm tone`
### Style Categories
LTX organizes styles into three families:
**Animation**: stop-motion, 2D animation, 3D animation, claymation, hand-drawn
**Stylized**: comic book, cyberpunk, 8-bit pixel, surreal, minimalist, painterly
**Cinematic**: period drama, film noir, fantasy, thriller, documentary, arthouse
## What to Avoid (LTX-Specific)
| Avoid | Reason |
|-------|--------|
| Internal emotional states ("sad", "confused") | Use visual cues: tears, slumped posture, furrowed brow |
| Readable text and logos | Not reliably rendered |
| Complex physics (explosions, splashing) | Causes artifacts; simple motion is fine |
| Overloaded scenes | Many characters/actions reduces coherence |
| Conflicting lighting descriptions | Pick one setup, commit to it |
| Starting complex | Build up: simple prompt first, add layers |
## LTX Technical Notes
- **Duration**: ~5-8 seconds per generation
- **Audio**: Generated automatically; describe what you want to hear
- **~30% of outputs have artifacts** — re-run with a different seed
- **Cannot render readable text** — don't include signs or titles
- **Frame count must satisfy** `(n-1) % 8 == 0`: valid counts are 25, 49, 73, 97, 121, 161, 193
## Example
```
A wide establishing shot captures a misty morning harbor.
Weathered fishing boats bob gently, their paint peeling in
patches of red and blue. A grey-haired fisherman in a dark
wool peacoat steps onto the dock, carrying a heavy net over
one shoulder. He pauses, looks out at the fog bank, then
walks toward the nearest boat with steady, deliberate steps.
The camera tracks alongside him at waist height, slowly
pushing in as he reaches the boat and tosses the net aboard.
Soft overcast light with a warm break in the clouds near
the horizon. Ambient sound of water lapping, rope creaking,
and distant foghorn.
```
skills/creative/prompting/sora-prompting.md
+91
View File
@@ -0,0 +1,91 @@
# Sora 2 — Prompting Guide
> Source: [OpenAI Sora 2 Cookbook](https://developers.openai.com/cookbook/examples/sora/sora2_prompting_guide)
> For universal vocabulary, see: `skills/creative/video-gen-prompting.md`
## Sora-Specific Prompt Template
Sora responds best to a structured format with prose + cinematography block + action beats:
```
[Prose scene description — characters, costumes, scenery, weather, details.
Be as descriptive as possible to match your vision.]
Cinematography:
Camera shot: [framing and angle]
Lens: [focal length, type]
Lighting: [key, fill, rim, practical sources with color temp]
Mood: [overall tone]
Actions:
- [Beat 1: specific gesture or movement]
- [Beat 2: another distinct beat]
- [Beat 3: reaction or dialogue]
Dialogue:
[Short natural lines, kept brief for clip length]
```
## Advanced Optional Fields
Sora uniquely responds to these production-level details that most models ignore:
| Field | Example |
|-------|---------|
| **Lens spec** | "40mm spherical", "85mm", "Anamorphic 2.0x" |
| **Filtration** | "Black Pro-Mist 1/4", "slight CPL rotation" |
| **Grade / palette** | "Warm Kodak-inspired grade", "teal-and-orange LUT" |
| **Film stock emulation** | "16mm black-and-white", "35mm photochemical contrast" |
| **Diegetic sound** | "faint rail screech, rain patters window, clock ticks" |
| **Wardrobe** | "navy coat, sleeves rolled, suspenders loose" |
| **Finishing** | "fine-grain overlay, mild halation, gate weave, soft vignette" |
| **Shutter** | "180° shutter angle" |
## What Sora Does Differently
- **Prose-first**: Write a rich paragraph, then add technical blocks. Don't lead with camera specs.
- **Character references**: Can lock onto up to 2 uploaded character IDs via API.
- **Dialogue sync**: Short lines work. Complex multi-character dialogue does not.
- **Edit commands**: "Same shot, switch to 85mm" or "Same lighting, new palette: teal, sand, rust" — Sora supports iterative refinement on existing generations.
- **Creative freedom**: Shorter prompts → more creative latitude. Longer → more control.
## Color Palette Technique
Name 3-5 anchor colors instead of vague "warm tones":
- "Amber, cream, walnut brown" (vintage warmth)
- "Teal, sand, rust" (coastal desert)
- "Cool blues with warm tungsten accents" (noir)
## Sora API Parameters (cannot be set in prompt)
- `model`: `sora-2` or `sora-2-pro`
- `size`: 720x1280, 1280x720, 1080x1920, 1920x1080, 1024x1792, 1792x1024
- `seconds`: 4, 8, 12, 16, 20
## Example
```
Style: Hand-painted 2D/3D hybrid animation with soft brush textures,
warm tungsten lighting, tactile stop-motion feel. Subtle watercolor wash;
warm-cool balance; filmic motion blur.
Inside a cluttered workshop, shelves overflow with gears and yellowing
blueprints. Small round robot sits on wooden bench, dented body patched
with mismatched plates. Large glowing blue eyes flicker as it fiddles
with a humming light bulb.
Cinematography:
Camera: medium close-up, slow push-in with gentle parallax from hanging tools
Lens: 35mm virtual; shallow depth of field
Lighting: warm key from overhead practical; cool spill from window
Mood: gentle, whimsical, touch of suspense
Actions:
- Robot taps bulb; sparks crackle
- Flinches, dropping bulb, eyes widening
- Bulb tumbles in slow motion; catches it just in time
- Puff of steam escapes chest — relief and pride
Background Sound:
Rain, ticking clock, soft mechanical hum, faint bulb sizzle
```
skills/creative/prompting/veo-prompting.md
+73
View File
@@ -0,0 +1,73 @@
# VEO 3.1 / VEO 3 — Prompting Guide
> Source: [Vertex AI Video Gen Prompt Guide](https://cloud.google.com/vertex-ai/generative-ai/docs/video/video-gen-prompt-guide)
> For universal vocabulary, see: `skills/creative/video-gen-prompting.md`
## VEO-Specific 14-Component Structure
VEO responds to the most comprehensive prompt structure of any model:
1. **Subject** — who/what the action revolves around
2. **Action** — movements, interactions, expressions
3. **Scene / Context** — location, time, weather, period
4. **Camera Angles** — shot type and perspective
5. **Camera Movements** — dynamic motion
6. **Lens / Optical Effects** — how the camera "sees"
7. **Lighting** — source, direction, quality
8. **Tone / Mood** — emotional register
9. **Artistic Style** — photorealistic, cinematic, animation, art movement
10. **Ambiance** — color palettes, atmospheric effects, textures
11. **Temporal Elements** — pacing, time flow, rhythm
12. **Audio** — sound effects, ambient, dialogue (VEO 3 generates dialogue)
13. **Cinematic Terms** — editing techniques (match cut, montage, split diopter)
14. **Negative Prompt** — what to exclude
## VEO-Specific Strengths
- **Dialogue generation**: VEO 3 natively generates character speech. Write dialogue naturally.
- **Audio integration**: Ambient sound, music, and voice are generated together with video.
- **Negative prompts**: Explicitly supported — "no text overlays, no watermarks, no lens flare"
- **Editing vocabulary**: Understands "match cut", "jump cut", "montage", "split diopter" as prompt terms.
## VEO Lens Effects (Unique)
VEO specifically responds to optical effects most models ignore:
| Effect | Prompt Language |
|--------|----------------|
| **Rack focus** | "rack focus from foreground flower to background figure" |
| **Dolly zoom (vertigo)** | "vertigo effect as character realizes the truth" |
| **Fisheye** | "fisheye lens distortion, skatepark POV" |
| **Lens flare** | "anamorphic lens flare from setting sun" |
## VEO Art Movement References
VEO responds well to specific art movements as style anchors:
- "Van Gogh-inspired swirling sky"
- "Surrealist Dalí-esque melting landscape"
- "Art Deco geometric patterns in the architecture"
- "Bauhaus clean lines and primary colors"
- "Gritty graphic novel illustration style"
- "Chinese ink wash painting animation"
## Subtitle Prevention
VEO may add subtitles by default for dialogue. To prevent:
- Add to negative prompt: "no subtitles, no captions, no text overlays"
## Example
```
Subject: A lone astronaut in a weathered white spacesuit
Action: Slowly turns to face the camera, visor reflecting a dying star
Scene: Surface of a barren moon, cracked grey terrain, massive ringed
planet filling the horizon
Camera: Low-angle medium shot, slow arc around subject
Lens: Wide-angle, deep focus keeping both astronaut and planet sharp
Lighting: Harsh rim light from the star behind, cool blue fill from
planet reflection, no atmosphere diffusion
Mood: Awe, isolation, quiet grandeur
Style: Photorealistic sci-fi cinematography, IMAX-scale
Audio: Breathing inside helmet, faint radio static, low rumble
Negative: No text, no HUD overlay, no lens flare
```
skills/creative/scene-detect-usage.md
+108
View File
@@ -0,0 +1,108 @@
# Scene Detection Usage for OpenMontage
> Sources: PySceneDetect documentation, FFmpeg scenedetect filter docs, PySceneDetect
> GitHub issues #187 (threshold tuning) and #226 (adaptive discussion)
## Quick Reference Card
```
DEFAULT METHOD: content (ContentDetector) — works for most content
DEFAULT THRESH: 27.0 (range 0-255)
MIN SCENE LEN: 1.0s default, 2.0-3.0s for educational video
TUNING: Generate stats CSV first, inspect content_val column
HARD CUTS: Use content detector
FADE TO BLACK: Use threshold detector
MIXED CONTENT: Use adaptive detector
```
## Algorithm Selection
| Method | Default Threshold | Best For | How It Works |
|--------|------------------|----------|-------------|
| `content` | 27.0 | Hard cuts between shots | HSV color difference between adjacent frames (0-255) |
| `threshold` | 12.0 | Fades to/from black | Average pixel intensity; detects transitions through black |
| `adaptive` | 3.0 | Mixed content with camera motion | Rolling average of frame differences; adapts to local pace |
## Threshold Tuning Guide
### ContentDetector (Default, Start Here)
| Symptom | Action | New Threshold |
|---------|--------|---------------|
| Too many false cuts | Raise threshold | 35-45 |
| Missing real cuts | Lower threshold | 20-22 |
| Fast-paced content (music videos, action) | Raise | 35-40 |
| Slow/static content (talking heads, presentations) | Lower | 20-25 |
| Animated content (Manim, motion graphics) | Raise | 30-35 |
### AdaptiveDetector
- Multiplier on rolling average (default 3.0)
- Better than ContentDetector when there's fast camera motion causing false positives
- Good default for OpenMontage explainers where Manim segments are static but live-action may have motion
### ThresholdDetector
- Only for videos with deliberate fade-to-black transitions
- Most AI-generated video does NOT use fades — prefer `content` or `adaptive`
## Tuning Workflow
1. **Generate stats file first:**
```bash
scenedetect -i video.mp4 --stats stats.csv detect-content
```
2. **Inspect `stats.csv`** — look at the `content_val` column. Peaks = scene changes.
3. **Set threshold** just below the smallest real peak.
4. **Set `min_scene_length`** to suppress micro-scenes:
- Educational video: 2.0-3.0s minimum
- Fast-paced content: 0.5-1.0s
- Default: 1.0s
### Component Weights (Advanced)
ContentDetector score = weighted sum of HSV + edge differences:
```
weights = (delta_hue, delta_sat, delta_lum, delta_edges)
Default: (1.0, 1.0, 1.0, 0.0)
```
For animated content with color transitions but few actual cuts:
```
weights=(1.0, 0.5, 1.0, 0.2), threshold=32
```
## Post-Processing Detected Scenes
After detection, clean up the scene list:
1. **Merge too-short segments** — any scene under `min_scene_length` should be merged with the adjacent scene
2. **Validate boundaries** — check that scene boundaries align with narration pauses (for explainers)
3. **Label scenes** — map detected scenes to script sections for the edit stage
## Content-Type Presets
| Content Type | Method | Threshold | Min Scene Length |
|-------------|--------|-----------|-----------------|
| Talking head (single camera) | content | 22 | 3.0s |
| Talking head (multi-camera) | content | 27 | 1.0s |
| Screen recording | content | 30 | 2.0s |
| Animated explainer | adaptive | 3.0 | 2.0s |
| Fast-paced montage | content | 40 | 0.5s |
| Documentary with fades | threshold | 12 | 2.0s |
## Applying to OpenMontage
When using the `scene_detect` tool:
1. **Start with `content` method, threshold 27** — it works for most content
2. **For talking-head pipeline**, lower threshold to 22 and set min_scene_length to 3.0s
3. **For animated-explainer pipeline**, use `adaptive` with default threshold 3.0
4. **Always generate stats CSV first** when tuning — don't guess thresholds
5. **Set min_scene_length to 2.0s** for educational content to avoid micro-scenes
6. **Use detected scenes to inform the edit stage** — map scenes to script sections
7. **For AI-generated video clips**, use `content` not `threshold` — AI video rarely uses fade-to-black
skills/creative/screen-recording.md
+123
View File
@@ -0,0 +1,123 @@
# Screen Recording Pipeline
> Sources: OBS Studio documentation, Loom production guidelines, Fireship production
> methodology, Kevin Powell CSS tutorial techniques, Theo Browne dev content guides
## Quick Reference Card
```
RESOLUTION: 1920x1080 at 2x display (record at 3840x2160, deliver at 1080p)
FRAME RATE: 60fps for UI/scrolling, 30fps for static code
CURSOR: Enlarged (1.5-2x), highlighted with ring or glow
ZOOM: 1.5-2x for code focus, 0.8s ease-in-out transition
SPEED RAMP: 1.5x for navigation, 2x for repetitive actions, 1.0x for key moments
DEAD AIR: Remove pauses > 1.5 seconds
FONT SIZE (IDE): 18-22px minimum for readability at 1080p delivery
```
## Recording Settings
### Resolution Strategy
| Approach | Record At | Deliver At | Why |
|----------|----------|-----------|-----|
| **Recommended** | 3840x2160 (4K) | 1920x1080 | Enables 2x zoom into code without quality loss |
| Budget | 1920x1080 | 1920x1080 | Direct capture, limited zoom headroom |
| Vertical | 2160x3840 | 1080x1920 | Short-form screen recording |
### Frame Rate
| Content Type | FPS | Why |
|-------------|-----|-----|
| Code editing (mostly static) | 30 | Smaller file size, no visible difference |
| UI interaction, scrolling | 60 | Smooth scrolling and cursor movement |
| Animation/demo with motion | 60 | Motion clarity |
| Terminal output | 30 | Text updates don't need 60fps |
### IDE/Editor Setup
- **Font size:** 18-22px minimum (must be readable at 1080p delivery)
- **Theme:** Dark theme preferred (easier on eyes, looks better in video)
- **Line numbers:** ON (helps viewers follow along)
- **Minimap:** OFF (wastes screen space, distracting)
- **Sidebar:** Collapsed unless showing file structure is the point
- **Status bar:** Consider hiding (clutters bottom of frame)
- **Zoom level:** 150-175% for readability
## Cursor Management
### Visibility
| Setting | Value |
|---------|-------|
| Cursor size | 1.5-2x default system size |
| Highlight | Yellow or white ring/glow (50px radius) |
| Click indicator | Brief flash or ripple on click |
| Smoothing | Light smoothing to reduce jitter |
### Cursor Behavior
- **Move deliberately** — no random wandering
- **Pause on target** for 0.5s before clicking
- **Avoid circling** — don't circle the cursor around what you're talking about
- **Hide cursor** when it's not needed (during code explanation)
## Zoom and Pan
### Zoom Levels
| Context | Zoom | Duration of Transition |
|---------|------|----------------------|
| Full screen overview | 1.0x (100%) | — |
| Code focus | 1.5-2.0x | 0.8s ease-in-out |
| Terminal focus | 1.5x | 0.6s ease-in-out |
| UI element highlight | 2.0-2.5x | 0.8s ease-in-out |
| Return to overview | 1.0x | 0.6s ease-in-out |
### Pan Rules
- Pan to follow the active area — don't make viewers search
- Smooth pan (ease-in-out), not instant jump
- Hold position for at least **3 seconds** before next pan
- Announce what you're zooming into: "Let's look at this function..."
## Post-Processing
### Speed Ramping
| Action | Speed | Notes |
|--------|-------|-------|
| Typing boilerplate | 2-3x | Viewers don't need to watch you type imports |
| File navigation | 1.5-2x | Opening files, switching tabs |
| Package install / build | 2-4x or cut | Show start + end, skip the wait |
| Key code writing | 1.0x | Important moments at real speed |
| Debugging / thinking | 1.0x with cuts | Remove dead pauses, keep the reasoning |
### Dead Air Removal
- Remove **all pauses > 1.5 seconds** unless deliberate
- Remove "um", "uh", typing mistakes and backspaces (when possible)
- Jump cuts are acceptable and expected in screen recording content
- Add a subtle **zoom shift** (1.0x → 1.02x) at each jump cut to mask the edit
### Audio Enhancement
- Apply `clean_speech` preset from `audio_enhance`
- HPF at 80Hz to remove keyboard/desk rumble
- Compress at 3:1 to even out speaking volume
- Target -16 LUFS for screen recording content (slightly quieter than -14, more comfortable for long viewing)
## Applying to OpenMontage
When processing screen recordings in the talking-head pipeline:
1. **Record at 4K** if possible — enables quality zoom in post
2. **Set IDE font to 20px+** before recording
3. **Use `scene_detect`** with threshold 30, min_scene_length 2.0s to find natural segments
4. **Apply zoom/pan** in compose stage — 1.5-2x on code, 0.8s transitions
5. **Speed ramp navigation** to 1.5-2x, keep key moments at 1.0x
6. **Remove dead air** > 1.5s with `video_trimmer`
7. **Add cursor highlight** in post if not captured in recording
8. **Target -16 LUFS** (slightly below YouTube standard for comfortable viewing)
9. **Subtitles recommended** — use `subtitle_gen` for accessibility
10. **Dark theme** looks best in video — recommend to users before recording
skills/creative/short-form.md
+205
View File
@@ -0,0 +1,205 @@
# Short-Form Video Pipeline (TikTok / Reels / Shorts)
> Sources: TikTok Creator Portal, Instagram for Business blog, YouTube Shorts documentation,
> Hootsuite Social Trends Report 2025, OpusClip retention data 2025, Shortimize 35B Shorts
> analysis, PostPlanify safe zones 2026, Kreatli platform guides, TTS Vibes hook statistics
## Quick Reference Card
```
ASPECT RATIO: 9:16 vertical (1080x1920)
SAFE ZONE: 900x1400px centered (universal cross-platform)
DURATION: 15s (highest completion) | 30s (best engagement) | 60s (most flexible)
HOOK: First 1-2 seconds — visual or text pattern interrupt
CAPTIONS: Mandatory (85% watch muted on mobile)
TEXT SIZE: 42px+ minimum, bold sans-serif
PACING: Visual change every 1-3 seconds
TARGET LUFS: -14 LUFS, true peak -1 dBTP
MUSIC: 120-140 BPM for energetic, 90-110 for explainers
```
## Platform Safe Zones (1080x1920)
| Platform | Safe Zone | Top Dead | Bottom Dead | Right Dead |
|----------|-----------|----------|-------------|------------|
| TikTok | 900x1492 | 108px | 320px | 120px |
| Instagram Reels | 996x1400 | 210px | 310px | 84px |
| YouTube Shorts | 984x1500 | 120px | 300px | 96px |
| Facebook Reels | 1080x1520 | 100px | 300px | 60px |
**Universal safe zone: 900x1400px centered** — works across all platforms.
**Bottom dead zones are critical** — platform UI (comments, share buttons, captions) covers the bottom 300-320px. Never put important content there.
## Upload Specs
```
CODEC: H.264 High Profile, Level 4.2
BITRATE: 8-15 Mbps VBR (below 5 Mbps triggers quality downgrade)
FORMAT: .mp4 preferred
MAX SIZE: 500 MB (desktop), 287.6 MB (iOS), 72 MB (Android)
```
## Duration Strategy
| Duration | Avg Completion Rate | Best For |
|----------|-------------------|----------|
| 0-15s | 92% | Single fact, quick tip, visual gag |
| 16-30s | 84% | One concept explained, before/after |
| 31-60s | 68% | Mini tutorial, step-by-step, story arc |
| 60s+ | 48% | Deep explainer, only if retention structure is strong |
**Platform sweet spots:**
- TikTok: 21-34 seconds for completion; 60-180s for maximum total watch time
- Reels: 15-30 seconds for viral reach; 60-90s for highest engagement
- Shorts: Bimodal — ~13 seconds OR full 60 seconds (Shortimize 35B views analysis)
**Key formula:** A 45s video with 70% completion (31.5s watch time) outperforms a 15s video with 40% completion (6s). Total watch time is what the algorithm rewards.
## The 1-Second Hook
**70%+ of TikTok users decide to scroll or stay within 3 seconds** (average decision point: 1.7 seconds). The hook must be immediate.
### 3-Second Retention and Algorithmic Impact
| 3-Second Retention | Algorithmic Effect | View Multiplier |
|-------------------|-------------------|-----------------|
| Below 60% | Minimal promotion | 1.0x (baseline) |
| 60-70% | Average distribution | 1.6x |
| 70-85% | Optimal reach | 2.2x |
| 85%+ | Viral potential | 2.8x |
### Retention Checkpoints
| Timestamp | Target Retention |
|-----------|-----------------|
| 3 seconds | 70%+ |
| 15 seconds | 60%+ |
| 30 seconds | 50%+ |
### Hook Techniques
| Technique | Example | When to Use |
|-----------|---------|-------------|
| **Bold text on screen** | "STOP doing this..." (text appears frame 1) | Always — text hooks work even muted |
| **Pattern interrupt** | Unexpected visual, jump cut, color flash | Attention-grabbing |
| **Question** | "Why does X happen?" (text + voiceover) | Educational |
| **Result first** | Show the finished result, then explain how | Tutorial/how-to |
| **Controversy** | "Everyone gets this wrong" | Engagement bait |
### Hook Rules
1. **Frame 1 must have visual interest** — no blank intros, no logos, no "hey guys"
2. **Text appears in the first 0.5 seconds** — viewers scan text before listening
3. **Voice starts immediately** — no silent buildup
4. **Movement in frame 1** — static opening frames get scrolled past
## Pacing
| Rule | Value | Why |
|------|-------|-----|
| Visual change frequency | Every 1-3 seconds | Mobile attention span |
| Cuts per minute | 20-40 | 2-3x faster than long-form |
| Text on screen | 2-4 seconds per text block | Fast reading pace |
| No static holds | Max 3 seconds | Anything longer feels frozen |
| Speed ramp | 1.2-1.5x for setup, 1.0x for payoff | Compress boring parts |
**Impact of pacing on retention:**
- Pattern interrupts every 2-4s: **58% average retention**
- Static talking head (no interrupts): **41% average retention**
- That's a **41% relative improvement** from pacing alone
### Script Word Counts
| Duration | Word Count |
|----------|-----------|
| 15 seconds | 35-40 words |
| 30 seconds | 70-80 words |
| 60 seconds | 125-150 words |
## Text & Captions
### Mandatory Captions
**80% of short-form viewers watch without sound** (mid-2025 data). Videos with accurate captions average **12% higher retention**. Captions are not optional.
| Parameter | Value |
|-----------|-------|
| Font size | 42px+ at 1080p |
| Font weight | Bold |
| Font family | Sans-serif (Inter, Montserrat, Poppins) |
| Background | Semi-transparent black (75% opacity) or text stroke (3px) |
| Position | Center or lower-center, within safe zone |
| Max chars/line | 30 |
| Max lines | 2 |
| Word-by-word highlight | Recommended for engagement |
### On-Screen Text (Non-Caption)
- Position in the **top 40%** of the safe zone (above center)
- Bold, high contrast (white on dark or colored background box)
- 3-5 words maximum per text block
- Animate entrance (scale pop or fade, 0.2-0.3s)
## Audio
| Element | Level | Notes |
|---------|-------|-------|
| Voiceover | -12 to -14 dB peak | Primary |
| Music | -22 to -26 dB | Lower than long-form — less room |
| SFX | -18 to -14 dB | Brief pops/whooshes only |
| Target LUFS | -14 LUFS | Same as long-form YouTube |
| True peak | -1 dBTP | TikTok/Instagram spec |
### Music Selection
- **Energetic content:** 120-140 BPM
- **Explainer content:** 90-110 BPM
- **Match trending audio patterns** — short-form audiences expect music-forward content
- **Music should start immediately** — no silent intro
### Voiceover Pacing
- **180-200 WPM** for short-form (faster than long-form's 150-160)
- Speak with energy and urgency
- No long pauses — dead air = scroll
## Structure Templates
### 15-Second Quick Tip
```
[0-1s] HOOK: Bold text + voice starts immediately
[1-3s] CONTEXT: One sentence setup
[3-12s] CONTENT: The tip/fact/technique (show, don't tell)
[12-15s] PAYOFF: Result or CTA text overlay
```
### 30-Second Explainer
```
[0-1s] HOOK: Pattern interrupt or question
[1-5s] PROBLEM: Why this matters
[5-22s] SOLUTION: Step-by-step with visual changes every 2-3s
[22-28s] RESULT: Show the outcome
[28-30s] CTA: Follow/share/comment prompt
```
### 60-Second Mini Tutorial
```
[0-2s] HOOK: Show finished result first
[2-8s] SETUP: "Here's how to do X in Y steps"
[8-45s] STEPS: 3-5 steps, ~8s each, visual change per step
[45-55s] RESULT: Before/after or final demo
[55-60s] CTA + LOOP: End connects back to start for replay
```
## Applying to OpenMontage
When building short-form content:
1. **Set output resolution to 1080x1920** (9:16) in the compose stage
2. **Keep all text within 900x1400px safe zone** — centered in frame
3. **Captions are mandatory** — use `subtitle_gen` with word-by-word timing
4. **Hook in frame 1** — text overlay + voice starts immediately, no intro
5. **Visual change every 1-3 seconds** — use quick cuts, zooms, text pops
6. **Voiceover at 180-200 WPM** — faster than long-form
7. **Music starts immediately** — set `music_gen` to energetic BPM (110-140)
8. **Target 15-30 seconds** for maximum completion rate
9. **Test on phone** — view at actual mobile size before publishing
skills/creative/sound-design.md
+141
View File
@@ -0,0 +1,141 @@
# Sound Design for Video Production
> Sources: W3C accessibility standards, BBC audio guidelines, YouTube/TikTok platform specs,
> Sweetwater mastering guides, ElevenLabs documentation, Boris FX, HookSounds, Artlist
## Quick Reference Card
```
DIALOGUE: -12 dB peak | -16 to -14 LUFS integrated
MUSIC BED: -30 to -20 dB (18-20 dB below dialogue)
SFX: -18 to -12 dB (6 dB below dialogue minimum)
WHOOSH TIMING: Start 10-20ms before visual, duration 400-500ms
MUSIC BPM: Calm 60-80 | Standard 90-110 | Upbeat 120-140
TRUE PEAK: Never exceed -1.5 dBTP
VOICE EQ: HPF 80Hz, cut 500Hz, boost 2-5kHz, cut 6-8kHz
VOICE COMP: 3:1 ratio, 1-5ms attack, 10-20ms release
TARGET LUFS: -14 LUFS (YouTube/TikTok/IG) | -16 LUFS (podcasts)
```
## Audio Ducking Levels
| Element | Peak Level | Notes |
|---------|-----------|-------|
| Dialogue / Narration | -6 dB to -12 dB | Primary element |
| Background music (during speech) | -18 dB to -20 dB | 18-20 dB below dialogue |
| Sound effects | -12 dB to -18 dB | Between dialogue and music |
| Final mix | -10 dB to -20 dB | Never exceed 0 dB |
**Ducking rules:**
- W3C accessibility: music must be **20 dB lower** than foreground speech
- BBC guideline: lower music by an additional **4 dB** from where you think it sounds right
- Duck music **6-12 dB** when narration is active; for complex educational topics, duck up to **22 dB**
- EQ trick: cut **2-4 kHz** on background music to make room for speech clarity
- When testing, adjust in **1 dB increments** from a -20 dB baseline upward
## Music Selection by Content Type
| Content Type | BPM Range | Mood |
|-------------|-----------|------|
| Calm explainer / tutorial | 60-80 | Contemplative, focused, trust-building |
| Corporate / testimonial | 60-100 | Professional, calm, credible |
| Standard explainer / educational | 90-110 | Steady, engaging, not distracting |
| Upbeat explainer / promo | 110-130 | Enthusiastic, approachable |
| High-energy / product demo | 120-140 | Exciting, urgent, dynamic |
| Action / fast-paced | 140-200 | Adrenaline, intensity |
**Genre recommendations for explainers:**
- Lo-fi (steady, non-distracting, modern feel)
- Ambient (atmospheric, stays in background)
- Light acoustic guitar instrumentals (warm, approachable)
- Contemporary pop instrumentals (upbeat, familiar)
- Inspiring soundtrack / cinematic light (builds emotion without overwhelming)
**Key rules:**
- Always use **instrumental** tracks when voiceover is present — lyrics compete with narration
- Choose dynamically **even** tracks — avoid dramatic crescendos or beat drops
- Match energy to the learning context: upbeat for "exciting new concept," gentle for serious topics
## Sound Effects (SFX) Placement
### SFX Categories for Explainer Videos
| SFX Type | Use Case | Duration | Level |
|----------|----------|----------|-------|
| Whoosh / Swish | Scene transitions, slide changes | 400-500ms | -18 to -12 dB |
| Pop / Pluck | Text appearing, bullet points | <200ms | -15 to -12 dB |
| Click / Tap | UI interactions, button presses | <100ms | -20 to -15 dB |
| Riser / Swell | Building to a reveal or key point | 1-3s | -18 to -12 dB |
| Impact / Hit | Key reveal, important stat | <300ms | -12 to -6 dB |
| Subtle whoosh | Element sliding in/out | 200-400ms | -20 to -15 dB |
### Timing rules
- Start whoosh **10-20ms before** the visual transition (brain processes audio faster)
- Peak of whoosh energy should coincide with the **moment of greatest visual change**
- Fine-tune in **1-frame increments** for sync
- When stacking whooshes, keep them in different frequency bands
## Platform Loudness Targets (2025)
| Platform | Integrated LUFS | True Peak | Notes |
|----------|----------------|-----------|-------|
| YouTube | -14 LUFS | -1.5 dBTP | Normalizes down, not up |
| YouTube Shorts | -14 LUFS | -1.5 dBTP | Same as long-form |
| TikTok | -14 LUFS | -1 dBTP | Prioritize 2-4 kHz for phone speakers |
| Instagram Reels | -14 LUFS | -1 dBTP | Same mobile optimization |
| Spotify | -14 LUFS | -2 dBTP | Stricter true peak |
| Apple Podcasts | -16 LUFS | -1 dBTP | More headroom for speech |
### Content-type LUFS
| Content Type | Integrated LUFS | Dynamic Range |
|-------------|----------------|---------------|
| Dialogue-heavy / educational | -16 to -14 LUFS | 6-12 dB |
| Music videos | -14 to -12 LUFS | 6-10 dB |
| Gaming content | -14 to -12 LUFS | 8-12 dB |
### Technical specs
- Sample rate: **48 kHz** preferred
- Bit depth: **24-bit** preferred
- Bitrate: **192 kbps** minimum
- Noise floor: below **-60 dB**
- Headroom: at least **-6 dB** in the final mix
## AI TTS (ElevenLabs) Mixing
### Processing Chain
1. **High-pass filter:** 80-100 Hz (24 dB/oct slope) — removes rumble and low-frequency TTS artifacts
2. **EQ:**
- Cut ~500 Hz: removes muddiness/boxy quality
- Boost 2-5 kHz (+2-3 dB): adds presence and clarity
- Cut 6-8 kHz (gentle): reduces sibilance/harshness common in AI voices
- Optional: boost 120-250 Hz for thinner AI voices
3. **Compression:**
- Ratio: **3:1** (range 2:1 to 4:1)
- Attack: **1-5 ms**
- Release: **10-20 ms** (increase to 30ms if pumping)
- Threshold: **-26 dB** (target -4 to -6 dB gain reduction)
- Output gain: **+6 dB**
4. **De-esser:** target **6-8 kHz** if sibilance remains
5. **Limiter:** ceiling at **-1.5 dBTP**
### AI-specific tips
- AI TTS has inconsistent dynamics — compression is more important than for human speech
- ElevenLabs may have subtle artifacts in 4-6 kHz; use narrow notch cut if detected
- Sidechain background music to voiceover track for automatic ducking
- Cut 2-4 kHz on the music bed to clear the "intelligibility band" for voice
- Always test on phone speakers — if voice disappears, boost 2-4 kHz more aggressively
## Applying to OpenMontage
When the **audio_mixer** tool is used in the compose stage:
1. Set narration as primary track, music as secondary
2. Apply ducking: music -18 to -20 dB below narration during speech
3. Select music BPM from the table above based on the playbook mood
4. Place SFX at transition points with 10-20ms audio lead
5. Target -14 LUFS integrated for YouTube output
6. Keep true peak below -1.5 dBTP
7. For AI TTS narration, apply the processing chain above before mixing
8. Test the final mix on phone speakers — most viewers watch on mobile
skills/creative/stock-sourcing-usage.md
+127
View File
@@ -0,0 +1,127 @@
# Stock Sourcing Usage for OpenMontage
> How to use the stock image and video tools effectively — query construction,
> provider selection, license awareness, and integration with the asset pipeline.
## Available Stock Tools
| Tool | Provider | Content | Cost | Rate Limit | Best For |
|------|----------|---------|------|-----------|----------|
| `pexels_image` | Pexels | Photos | Free | 200/hr | High-quality photography, diverse library |
| `pixabay_image` | Pixabay | Photos, illustrations, vectors | Free | 100/min | Category filtering, large library (5M+) |
| `pexels_video` | Pexels | Video clips | Free | 200/hr | HD/4K real-world footage |
| `pixabay_video` | Pixabay | Video clips | Free | 100/min | Category-filtered video, animation clips |
## Provider Selection Guide
### When to Use Pexels
- Need **high-quality photography** (curated, professional)
- Need **video** (larger video library than Pixabay)
- Want **orientation filtering** (landscape/portrait/square)
- Want **color filtering** (match playbook palette)
- Need results in **multiple languages** (28 locales)
### When to Use Pixabay
- Need **category-based filtering** (nature, business, science, etc.)
- Want **illustrations or vectors** in addition to photos
- Want **editor's choice** curated results
- Need **higher rate limits** (100/min vs 200/hr)
- Need **video type filtering** (film vs animation)
### Decision Flow
```
Need stock image?
├── Need specific category (science, business, etc.)? → pixabay_image
├── Need illustration/vector? → pixabay_image
├── Need color matching? → pexels_image
└── General photo? → pexels_image (higher quality curation)
Need stock video?
├── Need 4K? → pexels_video (supports 4K via size="large")
├── Need animation clips? → pixabay_video (video_type="animation")
├── Need category filter? → pixabay_video
└── General footage? → pexels_video (better HD quality)
```
## Input Parameters Guide
### pexels_image / pexels_video
```python
{
"query": "city skyline sunset", # Required: search term
"orientation": "landscape", # Optional: landscape/portrait/square
"size": "large", # Optional: large/medium/small
"color": "FF6B35", # Optional: hex without # or color name
"per_page": 5, # Results per page (1-80)
"download_size": "large2x", # Image: original/large2x/large/medium
"preferred_quality": "hd", # Video: hd/sd
"output_path": "assets/images/s3.jpg" # Where to save
}
```
### pixabay_image / pixabay_video
```python
{
"query": "server room", # Required: search term (max 100 chars)
"image_type": "photo", # Image: all/photo/illustration/vector
"video_type": "film", # Video: all/film/animation
"orientation": "horizontal", # all/horizontal/vertical
"category": "computer", # One of 20 categories
"colors": "blue,gray", # Comma-separated color names
"editors_choice": true, # Curated high-quality only
"safesearch": true, # Always true for production
"output_path": "assets/video/s5.mp4" # Where to save
}
```
## Gotchas and Best Practices
### 1. Pixabay URLs Expire
Pixabay download URLs contain embedded tokens that expire. **Always download immediately** after searching. The tools handle this automatically, but never cache Pixabay URLs for later use.
### 2. Pixabay Resolution Limit
Standard Pixabay API users get max 1280px wide images (`largeImageURL`). Full resolution requires approved API access. For most video production overlays, 1280px is sufficient.
### 3. Pexels Auth Header
Pexels uses a bare API key in the `Authorization` header (NOT `Bearer`). The tool handles this, but be aware if debugging.
### 4. Search Results Vary by Locale
Pexels supports 28 locales. If searching for culturally specific content, set the locale parameter.
### 5. Stock Images Are Deterministic
Unlike AI generation, searching "ocean waves" twice returns the same results. If the first result isn't good enough, try different keywords — don't retry the same query.
### 6. Duration Filtering for Video
Both stock video tools support `min_duration` and `max_duration` parameters. Use these to avoid downloading 30-second clips when you only need 4 seconds — it saves bandwidth and time.
## Integration with Asset Pipeline
Stock tools integrate exactly like generation tools. In the asset manifest:
```json
{
"id": "broll-s3",
"type": "image",
"subtype": "broll",
"path": "assets/images/broll-s3.jpg",
"source_tool": "pexels_image",
"scene_id": "scene-3",
"cost_usd": 0.00,
"metadata": {
"photographer": "Joey Farina",
"source_url": "https://www.pexels.com/photo/2014422/",
"license": "Pexels License (free, no attribution required)"
}
}
```
The Edit Director and Compose Director treat stock assets identically to generated ones — they just reference the file path from the manifest.
## Licensing Summary
| Provider | Commercial Use | Attribution | Restrictions |
|----------|---------------|-------------|-------------|
| Pexels | Yes, free | Not required (appreciated) | Cannot sell unaltered; cannot imply endorsement |
| Pixabay | Yes, free | Not required | Cannot sell unaltered; cannot create competing stock service |
Both are safe for all OpenMontage use cases. No licensing fees, no per-use royalties, no attribution obligations.
skills/creative/storytelling.md
+149
View File
@@ -0,0 +1,149 @@
# Storytelling & Narrative Structure for Explainer Videos
> Sources: YouTube Creator Academy, Derek Muller PhD thesis (U. Sydney 2008), Kurzgesagt production
> methodology (Philipp Dettmer), 3Blue1Brown (Grant Sanderson), Richard Mayer "Multimedia Learning"
> (Cambridge UP, 2001/2020)
## The Explainer Arc Template
For a **3-minute explainer video** (scale proportionally for other lengths):
```
[0:00 - 0:08] HOOK
Pattern interrupt or counterintuitive claim. 1-2 sentences max.
Visual: striking image or animation that creates curiosity.
[0:08 - 0:30] TENSION / INFORMATION GAP
"Here's what most people think... but that's not quite right."
Establish stakes: why should I care?
Visual: show the misconception or the puzzle.
[0:30 - 0:50] CONCEPT 1 (Foundation)
Simplest building block needed. ONE idea, ONE visual.
End with a "but" or "therefore" transition.
[0:50 - 1:15] CONCEPT 2 (Complication)
Build on Concept 1. Introduce the wrinkle.
Visual: transform/evolve the previous visual.
[1:15 - 1:20] PALETTE CLEANSER
Brief pause, visual gag, or "let that sink in" moment.
Gives working memory a beat to consolidate.
[1:20 - 1:50] CONCEPT 3 (Key Insight)
The "aha" moment. Core of the video.
1-3 seconds of deliberate silence after the reveal.
Visual: the most polished animation in the video.
[1:50 - 2:20] PROOF / EXAMPLE
Concrete demonstration: "Watch what happens when..."
Visual: show the insight working in a specific case.
[2:20 - 2:45] IMPLICATIONS / "SO WHAT?"
Connect back to the real world. "This means that..."
Scale from specific back to general.
[2:45 - 3:00] REFRAME + CLOSE
Callback to the hook. Restate the core insight in one sentence.
Optional: open a new curiosity gap.
```
## Scaling by Duration
| Length | Concepts | Hook | Tension | Core | Proof | Close |
|--------|----------|------|---------|------|-------|-------|
| 1 min | 1-2 | 5s | 10s | 30s | 10s | 5s |
| 2 min | 2-3 | 8s | 15s | 60s | 25s | 12s |
| 3 min | 3-5 | 8s | 22s | 100s | 30s | 15s |
| 5 min | 5-8 | 10s | 30s | 180s | 50s | 20s |
## Hook Types
| Type | Pattern | Best For |
|------|---------|----------|
| **Contrarian** | "Everything you've been told about X is wrong." | Veritasium-style science/myth-busting |
| **Outcome** | "By the end of this video, you'll understand X." | 3Blue1Brown-style math/concept |
| **Mystery** | "In 1987, something impossible happened..." | Kurzgesagt-style story-driven |
| **Stakes** | "This one mistake costs people X every year." | Practical/how-to content |
## The 30-Second Rule
YouTube data shows **50% of viewer drop-off happens in the first 30 seconds**. The hook + tension
setup MUST be complete by second 30. Retention curves that survive the 30-second cliff typically
retain 40-60% through the full video.
## The "But-Therefore" Method
Never connect sections with "and then." Always use **"but"** or **"therefore."**
**Bad:** "Atoms have electrons, AND THEN those electrons have energy levels, AND THEN..."
**Good:** "Atoms have electrons, BUT they don't behave like tiny planets, THEREFORE we need a
completely new model..."
Applied structure:
```
SETUP: Here's what you think you know about X.
BUT: Here's why that's wrong / incomplete / surprising.
THEREFORE: We need to understand Y (the real mechanism).
BUT: Y creates a new puzzle...
THEREFORE: The actual answer is Z.
THEREFORE: This changes how you should think about X.
```
## Misconception-First Approach (Research-Backed)
Derek Muller's PhD research (University of Sydney, 2008) showed that **videos presenting common
misconceptions FIRST, then refuting them, produce significantly higher learning gains** than videos
that simply present correct information. Viewers who watched "misconception-first" videos scored
higher on post-tests and reported higher engagement.
Apply this: always consider opening with what the audience *thinks* is true before revealing what
*actually* is.
## Guided Discovery (3Blue1Brown Method)
Don't explain the answer. **Reconstruct the reasoning path** so the viewer feels they discovered it.
1. **The Question** — Pose a specific, concrete question
2. **The Naive Attempt** — Show the obvious approach; let it partially work, then break
3. **The Key Insight** — Introduce ONE new idea. Pause visually for 2-3 seconds of silence.
4. **The Build** — Apply the insight step by step. Each step feels inevitable.
5. **The Generalization** — "Notice this pattern works beyond our specific example..."
**Progressive Revelation:** Never show the full picture at once. Build visuals layer by layer.
Each layer arrives exactly when the narration references it.
## Pacing Rules
| Rule | Value | Source |
|------|-------|--------|
| Narration speed | 150-160 wpm | Kurzgesagt standard (conversational is 170-190) |
| New visual element | Every 3-5 seconds | Kurzgesagt production rules |
| Concept density | Max 1 new concept per 30-45 seconds | Mayer's Segmenting Principle |
| Pattern interrupt | Every 45-90 seconds | YouTube retention data |
| Deliberate silence | 1-3 seconds after key insights | 3Blue1Brown technique |
| Palette cleanser | Every 45-60 seconds | Kurzgesagt production rules |
## Mayer's Multimedia Learning Principles (Applied)
These are the most relevant research-backed rules from cognitive science:
1. **Segmenting** — Max 1 new concept per 30-45 seconds. A 3-min video = 4-6 concept segments.
2. **Signaling** — Use verbal signposts every 30-45 seconds ("Here's where it gets interesting").
3. **Temporal Contiguity** — Narration and visuals must be simultaneous. Learning drops ~30% when offset even by a few seconds.
4. **Coherence** — Remove interesting-but-irrelevant content. "Seductive details" reduce learning by 20-30% on transfer tests.
5. **Modality** — Use narration (audio) + visuals (animation), NOT on-screen text + visuals. Spoken words + pictures outperform written words + pictures.
## Applying to OpenMontage
When writing a **script artifact** for the animated-explainer pipeline:
1. Choose a hook type from the table above based on the topic
2. Structure sections using the Explainer Arc template
3. Apply "but-therefore" connectors between sections
4. Consider the misconception-first approach for science/technical topics
5. Set `narration_wpm: 155` in the script to calculate accurate timing
6. Plan visual changes every 3-5 seconds in the scene_plan
7. Mark "silence" beats in the script for key insights
8. Validate: total concepts should not exceed the scaling table above
skills/creative/talking-head-gen-usage.md
+137
View File
@@ -0,0 +1,137 @@
# Talking Head Generation Usage for OpenMontage
> Sources: SadTalker paper (Zhang et al. 2023), MuseTalk documentation, existing Layer 2 skills
> at `skills/creative/face-restore-usage.md` and `skills/creative/enhancement-strategy.md`
## Quick Reference Card
```
DEFAULT MODEL: sadtalker
INPUT: One face photo + one audio file → animated talking video
EXPRESSION: expression_scale=1.0 (0.5 = subtle, 1.5 = expressive)
STILL MODE: false (true = mouth-only animation, head stays fixed)
PREPROCESS: crop (default — crops face, animates, pastes back)
KEY RULE: Generate audio FIRST, then pass to talking_head
```
## When to Use the talking_head Tool
| Scenario | Use talking_head? |
|----------|-------------------|
| Avatar spokesperson video from a single photo | Yes |
| Personalized message — animate a headshot with custom narration | Yes |
| No video footage exists but a photo is available | Yes |
| Multi-language avatar — same face, different audio tracks | Yes |
| Existing video footage needs processing | No — use the talking-head pipeline |
| Lip-syncing existing video to new audio | No — use the `lip_sync` tool |
## Input Requirements
### Photo
- Clear, front-facing face with good lighting
- Minimum resolution: 256x256px
- Best results: 512x512 or larger
- Neutral expression, direct eye contact
- Avoid: extreme angles, accessories covering the face (large sunglasses, masks), multiple faces in the image
### Audio
- Clean speech audio — WAV or MP3
- Sample rate: 16kHz or higher
- Audio duration determines output video duration
- Remove background noise before feeding into talking_head — clean audio produces cleaner lip sync
## Model Selection
| Model | Strengths | Weaknesses |
|-------|----------|------------|
| sadtalker | Natural head motion, good expression range, well-tested | Can struggle with extreme expressions |
| musetalk | Higher quality lip sync, sharper mouth region | More constrained head motion |
**Default to `sadtalker`** unless lip sync precision is the top priority.
## Settings Reference
### Preprocess Modes
| Mode | What It Does | When to Use |
|------|-------------|-------------|
| `crop` | Crops face region, animates, pastes back into original frame | Default — best for headshots and portraits |
| `resize` | Resizes full input to model dimensions | When you want full-frame output at model resolution |
| `full` | No preprocessing — input passed directly | Advanced — input must already be correctly sized for the model |
### expression_scale Tuning
| Value | Effect | Use Case |
|-------|--------|----------|
| 0.5 | Subtle, minimal head movement | Corporate, formal, conservative |
| 0.7 | Calm, professional | Business presentations, news-style |
| 1.0 | Natural conversational (default) | General-purpose, explainers |
| 1.5 | Expressive, energetic | Social media, engaging content |
| >1.5 | Risk of artifacts | Avoid unless intentionally stylized |
### still_mode
| Value | Effect | Use Case |
|-------|--------|----------|
| `false` (default) | Head moves naturally while speaking | More realistic, conversational feel |
| `true` | Only mouth animates, head stays fixed | Formal/corporate look, or when head motion causes artifacts |
## Common Workflows
### 1. Avatar Spokesperson
```
photo + elevenlabs_tts → talking_head → face_enhance → compose
```
Standard avatar video: generate speech from script, animate the photo, polish the face, compose into final video.
### 2. Multi-Language Avatar
```
photo + tts per language → talking_head per language → compose variants
```
Same face photo, different audio tracks per language. Each produces a separate talking-head video for localized content.
### 3. Quick Social Content
```
headshot + script → piper_tts → talking_head → subtitle_gen → compose
```
Fast turnaround social video: generate speech locally, animate, add subtitles, compose.
### 4. Photo-to-Explainer
```
talking_head output → compose with diagram overlays
```
Use the talking-head video as a presenter layer, then overlay diagrams, charts, or screen recordings during composition.
## Quality Checklist
Before accepting talking_head output, verify:
- [ ] Lip movements match the audio naturally
- [ ] Head motion looks organic, not robotic
- [ ] No visual artifacts around face edges or jaw
- [ ] Eyes blink naturally (not frozen or blinking too fast)
- [ ] Output resolution is acceptable for the target platform
- [ ] Expression intensity matches the tone of the narration
## Applying to OpenMontage
When using the `talking_head` tool:
1. **Generate audio FIRST** (via `tts_selector`, `elevenlabs_tts`, `openai_tts`, or `piper_tts`), then pass to talking_head
2. **Use `expression_scale=1.0` as baseline** — only increase for high-energy content
3. **Always apply `face_enhance` AFTER talking_head** to polish the output
4. **For corporate/professional content**, use `still_mode=true` and `expression_scale=0.7`
5. **Source photo quality directly impacts output quality** — use the best available photo
6. **Crop mode is the safest default** — only use `resize` or `full` if crop produces bad framing
7. **Preview a 5-second clip before generating the full video** — catch artifacts early
8. **Fallback strategy:** if SadTalker is unavailable but Wav2Lip is, record a simple static video from the photo and lip-sync it with the `lip_sync` tool instead
skills/creative/typography.md
+226
View File
@@ -0,0 +1,226 @@
# Typography for Video Production
> Sources: School of Motion typography guides, legibility.info video text rules, Wave.video font
> pairing research, EBU/SMPTE broadcast standards, Netflix subtitle spec, BBC subtitle guidelines,
> WCAG 2.1 contrast requirements, Easings.net, postplanify.com safe zone data (2026)
## Quick Reference Card
```
TITLE SIZE: 60-90px at 1080p | 120-180px at 4K
BODY SIZE: 40-60px at 1080p | 80-120px at 4K
SUBTITLE SIZE: 42px+ at 1080p | 3-5% of video height
MAX CHARS/LINE: 32-42 (subtitles) | 30 (overlays)
MAX LINES: 2 (subtitles) | 3 (overlays)
READING SPEED: 21 chars/sec | 160-200 WPM
TITLE SAFE: 80% of frame (192px margin at 1080p)
ACTION SAFE: 90% of frame (96px margin at 1080p)
FONT FAMILIES: 1-2 per video maximum
CONTRAST: 4.5:1 minimum, 7:1 optimal
FADE DURATION: 0.3s opacity | 0.5-1.0s slide/scale
```
## Font Selection
### Recommended Video Fonts
| Category | Fonts | Use For |
|----------|-------|---------|
| **Body / Captions** | Inter, Open Sans, Roboto, Source Sans Pro, Lato, DM Sans | All body text, subtitles, captions |
| **Headlines** | Montserrat Bold, Bebas Neue, Oswald Bold, Poppins Bold | Titles, section headers, key stats |
| **Editorial** | Playfair Display, Roboto Slab | Luxury, cinematic, documentary |
| **System Safe** | Helvetica Neue, Arial, Avenir Next | When custom fonts unavailable |
### Font Pairing Rules
- Limit to **1-2 font families** per video — more creates visual noise
- Pair a **display/bold heading** font with a **neutral body** font
- Size difference between title and body: at least **50% larger**
- **Sans-serif** for motion graphics and captions (holds up in motion)
- **Serif** only for cinematic title cards and editorial content
- **Script/decorative** fonts: hero titles only, never body, never in motion
### Proven Pairings
| Heading | Body | Style |
|---------|------|-------|
| Bebas Neue | Open Sans | High-impact, social ads |
| Montserrat Bold | Lato | Clean modern |
| Oswald Bold | Raleway | Strong contrast |
| Playfair Display | Inter | Editorial |
| Poppins Bold | Poppins Light | Single-family hierarchy |
## Text Sizing
### Minimum Readable Sizes
| Element | 1080p (px) | 4K (px) | Notes |
|---------|-----------|---------|-------|
| Title / Hero text | 60-90 | 120-180 | Must be readable as thumbnail |
| Body text | 40-60 | 80-120 | Absolute minimum for readability |
| Subtitles | 42+ | 84+ | Accessibility requirement |
| Lower third name | 48-60 | 96-120 | Bold weight |
| Lower third role | 36-44 | 72-88 | Light/regular weight |
| Thumbnail text | — | — | Must read at 120-160px wide display |
## Safe Zones
### Broadcast Standard
| Zone | Coverage | Margin at 1080p | Purpose |
|------|----------|----------------|---------|
| **Title Safe** | 80% of frame | 192px H, 108px V | All text must stay within |
| **Action Safe** | 90% of frame | 96px H, 54px V | All important content |
At 1920x1080: Title Safe = inner **1536x864px**
At 3840x2160: Title Safe = inner **3072x1728px**
### Platform-Specific Safe Zones (Vertical 1080x1920)
| Platform | Safe Zone | Top Dead | Bottom Dead | Right Dead |
|----------|-----------|----------|-------------|------------|
| **TikTok** | 900x1492 | 108px | 320px | 120px |
| **Instagram Reels** | 996x1400 | 210px | 310px | 84px |
| **YouTube Shorts** | 984x1500 | 120px | 300px | 96px |
| **Facebook Reels** | 1080x1520 | 100px | 300px | 60px |
| **Instagram Stories** | 1080x1620 | 100px | 200px | — |
**Universal cross-platform safe zone: 900x1400px centered** — works on all platforms.
## Text Animation Timing
### Duration on Screen
- Reading speed: **13 characters per second** minimum dwell time
- 30-character line: minimum **2.3 seconds**
- General rule: **3 seconds per 63 characters**
- Title cards: **3-6 seconds**
- After animation completes, hold motionless for **1 second per 13 characters**
### Animation Durations
| Animation Type | Duration | Use Case |
|---------------|----------|----------|
| Fade in/out | 0.3-0.5s | Subtle, universal |
| Slide / scale entrance | 0.5-1.0s | Standard motion graphics |
| Kinetic text entrance | 1.0-2.0s | Bold, energetic |
| Lower third entrance | 1.0-2.0s | Speaker identification |
| Lower third exit | 0.5-1.0s | Quick departure |
### Easing Curves
| Easing | Cubic Bezier | Use For |
|--------|-------------|---------|
| **easeOutCubic** | `(0.33, 1, 0.68, 1)` | Text entrances (decelerates into place) — **default choice** |
| **easeOutQuart** | `(0.25, 1, 0.5, 1)` | Snappier entrance, kinetic type |
| **easeInOutQuad** | `(0.45, 0, 0.55, 1)` | Smooth position transitions |
| **easeInOutCubic** | `(0.65, 0, 0.35, 1)` | Scale and opacity changes |
| **easeInCubic** | `(0.32, 0, 0.67, 0)` | Exits (accelerates out) |
**Never use linear easing** for text animations — it feels robotic.
### Reveal Techniques
| Technique | Feel | Best For |
|-----------|------|----------|
| Mask reveal | Cinematic | Title cards, premium content |
| Scale pop | Energetic | Social media, short-form |
| Character stagger | Natural flow | Kinetic typography |
| Word-by-word sync | Engaging | Talking-head captions, TikTok |
| Fade | Subtle | Professional, corporate |
## Subtitle & Caption Typography
### Specifications
| Parameter | Value | Source |
|-----------|-------|--------|
| Font size | 42px+ at 1080p | Accessibility standard |
| Max characters per line | 32-42 | Platform dependent (see below) |
| Max lines | 2 per block | Universal standard |
| Line spacing | 1.3x | Readability standard |
| Background | Semi-transparent black, 70-80% opacity | Contrast requirement |
| Alternative style | White text + 2-4px dark stroke | No-box style |
| Minimum contrast | 4.5:1 (white on black = 21:1) | WCAG AA |
| Bottom margin | 60px from edge minimum | Mobile gesture clearance |
| Within frame width | 90% maximum | Title safe compliance |
### Character Limits by Platform
| Platform | Max Chars/Line |
|----------|---------------|
| YouTube | 42 |
| Netflix | 42 |
| BBC | 37 |
| TV broadcast | 37-42 |
| Cinema | 40-45 |
### Caption Timing
| Parameter | Value |
|-----------|-------|
| Minimum duration | 1 second |
| Maximum duration | 6-7 seconds |
| Reading speed | 21 characters/second |
| Fade-in transition | 0.3 seconds |
| Gap between captions | 2 frames |
| Sync tolerance | 3 frames of audio |
### Reading Speed by Platform
| Platform | WPM |
|----------|-----|
| TikTok / Instagram Reels | 180-200 |
| YouTube | 160-180 |
| LinkedIn | 140-160 |
| Educational content | 120-140 |
## Lower Thirds
### Standard Specs (1080p)
- Overlay region: **1920x360px** (bottom third)
- Sans-serif fonts (Helvetica, Open Sans, Roboto)
- White text with drop shadow or semi-transparent background bar
- Name: bold, larger weight
- Role/subtitle: lighter weight, smaller
### Timing
| Phase | Duration |
|-------|----------|
| Entrance animation | 1-2 seconds |
| Display | 3-6 seconds |
| Exit animation | 0.5-1 second |
## Contrast & Readability
### WCAG Requirements
| Element | Min Ratio | Standard |
|---------|----------|----------|
| Body text | 4.5:1 | WCAG AA |
| Large text (>18pt) | 3:1 | WCAG AA |
| Enhanced body | 7:1 | WCAG AAA |
| UI components | 3:1 | WCAG 2.1 |
### Text-Over-Video Techniques
1. **Semi-transparent box** — 70-80% black opacity behind text (most reliable)
2. **Text stroke** — 2-4px dark outline around light text
3. **Drop shadow** — subtle shadow for depth (less reliable on busy backgrounds)
4. **Darkened region** — gradient overlay behind text area
5. **Full-screen overlay** — 30-50% dark overlay for text-heavy screens
## Applying to OpenMontage
When generating text for video in the compose/asset stages:
1. **Font selection** — use the recommended video fonts above; prefer Inter or Open Sans for body, Montserrat Bold for titles
2. **Size check** — never go below 40px at 1080p for any text element
3. **Safe zones** — all text within 80% title-safe area; for vertical/short-form, use the 900x1400px universal safe zone
4. **Subtitle styling** — 42px+, max 2 lines, max 42 chars/line, semi-transparent background at 75% opacity
5. **Animation** — use easeOutCubic for entrances, hold text for at least 1 second per 13 characters after animation
6. **Contrast** — verify 4.5:1 minimum on a representative graded frame; prefer white-on-dark-background (21:1)
7. **Platform targeting** — check the platform safe zone table above and adjust text placement accordingly
8. **Remotion rendering** — all font families must be loaded via `@import` or `fontFamily` in the component; test that fonts render in the Docker/Lambda environment
skills/creative/upscale-usage.md
+130
View File
@@ -0,0 +1,130 @@
# Upscaling Usage for OpenMontage
> Sources: Real-ESRGAN documentation, GFPGAN face enhancement docs, Real-ESRGAN paper
> (Wang et al., 2021), practical upscaling benchmarks
## Quick Reference Card
```
DEFAULT MODEL: RealESRGAN_x4plus — real-world photos and video frames
DEFAULT SCALE: 4x (480p→1080p, 720p→4K)
ANIME MODEL: RealESRGAN_x4plus_anime_6B — flat color areas, illustrations
FACE ENHANCE: Enable face_enhance for footage with people (uses GFPGAN)
DENOISE: 0.5 default, raise to 0.8 for very noisy inputs
```
## When to Upscale
| Situation | Upscale? | Notes |
|-----------|----------|-------|
| User-provided footage is 480p or 720p, target is 1080p/4K | Yes | Most common use case |
| Generated images need higher resolution for video frames | Yes | AI image output is often 512-1024px |
| Thumbnail or still frames need crisp detail | Yes | Single-frame upscale is fast |
| Old/archival footage restoration | Yes | Combine with higher denoise_strength |
| Source is already 1080p+ and target is 1080p | **No** | Wastes compute, can introduce artifacts |
| Source is already 4K | **No** | Over-sharpening degrades quality |
## Model Selection
| Model | Best For | Notes |
|-------|----------|-------|
| `RealESRGAN_x4plus` | Real-world photos, video frames | Default choice |
| `RealESRGAN_x4plus_anime_6B` | Anime, illustrations, motion graphics | Preserves flat color areas |
| `RealESRNet_x4plus` | Fastest option, slightly lower quality | When speed matters |
## Scale Factor Guidance
| Scale | Use Case | Example |
|-------|----------|---------|
| 4x | Standard upscale for low-res sources | 480p→1080p, 720p→4K |
| 2x | Moderate upscale when 4x is overkill | 720p→1080p |
- **4x** is the most common choice. Use it for 480p sources targeting 1080p, or 720p targeting 4K.
- **2x** is appropriate when the source is already 720p and the target is 1080p — avoids unnecessary processing and potential artifacts.
- **Never upscale beyond 4x in a single pass.** Quality degrades sharply, and hallucinated details become obvious.
## Face Enhancement
- Enable `face_enhance` when the video contains human faces
- Uses GFPGAN internally to enhance face regions while Real-ESRGAN handles the rest
- Particularly valuable for webcam footage and old video
- Do NOT enable for content without faces — adds processing time with no benefit
## Denoising Strength
| Source Quality | denoise_strength | Rationale |
|---------------|-----------------|-----------|
| Clean digital source | 0.5 (default) | Minimal denoising needed |
| Slight compression artifacts | 0.6 | Light cleanup without over-smoothing |
| Old/noisy footage | 0.7-0.8 | Aggressive denoising for archival content |
| Very noisy / low-light footage | 0.8 | Maximum practical denoising |
Do not exceed 0.8 — higher values destroy legitimate detail.
## Video Upscaling Notes
- Video upscaling extracts frames, upscales each, reassembles
- This is **SLOW** — budget 5-10x real-time on GPU
- For long videos, consider upscaling only key scenes/clips rather than the full video
- Audio is preserved from the original
- Output file size will be significantly larger (~16x for 4x upscale)
## Common Workflows
### Workflow 1 — User-Provided Low-Res Footage
```
1. Assess source resolution (e.g., 480p webcam recording)
2. Choose scale factor: 4x for 480p→1080p, 2x for 720p→1080p
3. Enable face_enhance if footage contains people
4. Set denoise_strength based on source quality
5. Upscale → inspect output → proceed to compose stage
```
### Workflow 2 — AI-Generated Image Frames
```
1. Generate images at native model resolution (512-1024px)
2. Upscale with RealESRGAN_x4plus to target video resolution
3. Keep denoise_strength at 0.5 — AI output is clean
4. Do NOT enable face_enhance unless faces are prominent
```
### Workflow 3 — Manim / Motion Graphics Frames
```
1. Render Manim at default resolution
2. Upscale with RealESRGAN_x4plus_anime_6B (preserves flat colors)
3. Keep denoise_strength at 0.5
4. Verify text and line art remain sharp
```
### Workflow 4 — Archival Footage Restoration
```
1. Assess noise level and resolution
2. Set denoise_strength to 0.7-0.8
3. Enable face_enhance for footage with people
4. Use RealESRGAN_x4plus at 4x
5. Carefully inspect output for hallucinated details
```
## Quality Checklist
- [ ] Upscaled output is sharp without visible artifacts
- [ ] Faces look natural (no over-smoothing or distortion)
- [ ] Text/UI elements in screen recordings remain readable
- [ ] No hallucinated details in flat color areas
- [ ] File size is reasonable (4x upscale = ~16x file size)
## Applying to OpenMontage
When using the `upscale` tool in the asset stage:
1. **Upscale BEFORE the compose stage** — it is an asset-prep step, not a post-processing step
2. **Use `face_enhance=true` for any talking-head footage** — GFPGAN dramatically improves face quality
3. **Use `RealESRGAN_x4plus_anime_6B` model for Manim outputs** or flat illustration frames — preserves clean edges and flat color areas
4. **For budget-conscious pipelines**, upscale only hero shots and thumbnails rather than every frame
5. **Set `denoise_strength` to 0.7-0.8 for old/noisy footage**, keep at 0.5 for clean digital sources
6. **Check upscaled output for artifacts** — over-sharpening, hallucinated texture, face distortion
7. **Prefer 2x over 4x when the source is already 720p and target is 1080p** — less compute, fewer artifacts
skills/creative/video-editing.md
+61
View File
@@ -0,0 +1,61 @@
# Video Editing Skill
## When to Use
Apply this skill when making editorial decisions for talking-head content:
where to cut, what to remove, how to pace, and how to structure the final edit.
## Tools
| Tool | Role |
|------|------|
| `transcriber` | Analyze speech for filler words, dead air, false starts |
| `video_trimmer` | Execute cuts and speed adjustments |
| `frame_sampler` | Sample frames to evaluate visual quality at potential cut points |
| `video_compose` | Assemble the final edit |
## Editing Principles for Talking Heads
### What to Cut
1. **Filler words:** "um", "uh", "like", "you know" — cut at word boundaries using word timestamps.
2. **False starts:** When the speaker restarts a sentence, keep only the final take.
3. **Dead air:** Silence longer than 1.5 seconds should be trimmed to ~0.5 seconds.
4. **Off-topic tangents:** If the speaker wanders, cut to the next relevant segment.
5. **Repeated points:** Keep the best delivery, remove redundant takes.
### What NOT to Cut
- **Breath pauses:** Natural 0.3-0.8 second pauses between sentences. These sound natural.
- **Emphasis pauses:** Intentional pauses for dramatic effect.
- **Reactions and transitions:** Verbal bridges like "So..." or "Now..." that provide flow.
### Cut Technique
- **J-cut:** Audio from the next segment starts ~0.5s before the visual cut. Makes transitions feel smooth.
- **L-cut:** Audio from the current segment continues ~0.5s after the visual cut. Maintains continuity.
- **Hard cut:** Instant transition. Use at major topic changes.
### Pacing
- **Short-form (< 60s):** Aggressive cuts. Minimal dead air. High energy.
- **Medium-form (1-10 min):** Balanced. Keep natural pauses for breathing room.
- **Long-form (> 10 min):** Let scenes breathe. Only cut obvious problems.
## Edit Decision Structure
The `edit_decisions` artifact should include:
- **cuts:** Ordered list of segments to keep (source, in/out points, speed)
- **overlays:** Timed overlay placements (images, diagrams, lower thirds)
- **subtitles:** Subtitle configuration (enabled, style, source file)
- **music:** Background music settings (asset, volume, ducking, fades)
- **transitions:** Transition type and timing between cuts
## Quality Checklist
- [ ] No visible jump cuts (smooth transitions between segments)
- [ ] Audio doesn't pop or click at cut points
- [ ] Pacing matches the content energy and target platform
- [ ] Speaker's face is never covered by overlays
- [ ] All cuts are at word boundaries (not mid-word)
skills/creative/video-gen-prompting.md
+195
View File
@@ -0,0 +1,195 @@
# Video Generation Prompting — Universal Guide
## When to Use
When writing prompts for the video generation family (`video_selector`, `heygen_video`,
`wan_video`, `hunyuan_video`, `ltx_video_local`, `ltx_video_modal`, `cogvideo_video`).
This skill covers the universal prompt vocabulary that works across all video generation models.
For model-specific tips, see the linked guides below.
## Model-Specific Guides
| Model | Guide | Key Insight |
|-------|-------|-------------|
| **Sora 2 / Sora 2 Pro** | [OpenAI Sora 2 Cookbook](https://developers.openai.com/cookbook/examples/sora/sora2_prompting_guide) | Richest structured template. Advanced fields: lenses, filtration, grade, diegetic sound, wardrobe, finishing. |
| **VEO 3.1 / VEO 3** | [Vertex AI Prompt Guide](https://cloud.google.com/vertex-ai/generative-ai/docs/video/video-gen-prompt-guide) | Best vocabulary reference tables. 14-component prompt structure. |
| **LTX-2** | [LTX Prompting Guide](https://docs.ltx.video/api-documentation/prompting-guide) | 6-element structure. Audio/voice prompting. Strong "what to avoid" section. |
| **HunyuanVideo 1.5** | [Tencent Prompt Handbook](https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/main/assets/HunyuanVideo_1_5_Prompt_Handbook_EN.md) | Formula: Subject + Motion + Scene + [Shot] + [Camera] + [Lighting] + [Style] + [Atmosphere]. |
| **Runway Gen-4** | [Runway Prompting Guide](https://help.runwayml.com/hc/en-us/articles/39789879462419-Gen-4-Video-Prompting-Guide) | "Focus on motion, not appearance." One scene per clip. Simplicity wins. |
| **Kling 2.6** | [Kling Prompt Guide](https://fal.ai/learn/devs/kling-2-6-pro-prompt-guide) | 4-part structure. Supports `++emphasis++` syntax for key elements. |
| **Wan 2.1 / CogVideoX** | Use this generic guide | No official prompt guide. Standard cinematographic vocabulary works well. |
## Universal Prompt Formula
All video generation models respond to this structure. Include what's relevant, omit what's not.
```
[Shot type/framing] + [Camera movement] + [Subject description] +
[Action/motion in beats] + [Setting/environment] + [Lighting] +
[Style/aesthetic] + [Audio/atmosphere]
```
**Shorter prompts = more creative freedom. Longer prompts = more control.**
---
## Camera Shot Types
| Shot | When to Use |
|------|-------------|
| **Wide / establishing shot** | Open a scene, show location context |
| **Full / long shot** | Subject head-to-toe with environment |
| **Medium shot** | Waist up, balances detail with context |
| **Medium close-up** | Chest up, conversational intimacy |
| **Close-up** | Face or key object, emphasize emotion |
| **Extreme close-up** | Isolated detail (eye, drop, texture) |
| **Over-the-shoulder** | Conversation framing, connection |
| **Point-of-view (POV)** | Viewer becomes the character |
| **Bird's-eye / top-down** | Map-like overview, omniscient feel |
| **Worm's-eye view** | Looking straight up, emphasize height |
| **Dutch / canted angle** | Tilted horizon, unease or tension |
| **Low-angle** | Subject appears powerful, dominant |
| **High-angle** | Subject appears small, vulnerable |
## Camera Movements
| Movement | What It Does | Best For |
|----------|-------------|----------|
| **Static / fixed** | No movement | Dialogue, contemplation, stability |
| **Pan** (left/right) | Rotates horizontally | Revealing a scene, following action |
| **Tilt** (up/down) | Rotates vertically | Revealing height, slow reveal |
| **Dolly in / out** | Physically moves toward/away | Building tension, emphasis |
| **Truck** (left/right) | Moves sideways | Parallels subject movement |
| **Pedestal** (up/down) | Moves vertically | Smooth elevation changes |
| **Crane shot** | Sweeping vertical arcs | Epic reveals, transitions |
| **Tracking / follow** | Follows subject | Action sequences, walk-and-talk |
| **Arc shot** | Circles around subject | Dramatic emphasis, 360° reveal |
| **Zoom** (in/out) | Lens focal length change | Quick emphasis (cheaper than dolly) |
| **Whip pan** | Extremely fast pan (blurs) | Transitions, energy, surprise |
| **Handheld / shaky cam** | Unstable, human feel | Documentary, urgency, realism |
| **Aerial / drone** | High altitude, smooth | Landscapes, establishing shots |
| **Slow push-in** | Gradual forward movement | Building intimacy or tension |
| **Dolly zoom (vertigo)** | Dolly one way, zoom opposite | Disorientation, revelation |
## Lighting Vocabulary
| Term | Effect |
|------|--------|
| **Natural light** | Soft, realistic (morning sun, overcast, moonlight) |
| **Golden hour** | Warm sunlight, long shadows, romantic |
| **High-key** | Bright, even, cheerful — comedy, lifestyle |
| **Low-key** | Dark, high contrast — thriller, drama |
| **Rembrandt** | Triangle of light on cheek, classic portrait |
| **Film noir** | Deep shadows, stark highlights |
| **Volumetric** | Visible light rays through atmosphere (fog, dust) |
| **Backlighting** | Light behind subject, silhouette effect |
| **Side lighting** | Strong directional, dramatic shadows |
| **Practical lights** | In-frame sources (lamps, candles, neon signs) |
| **Rim / edge light** | Highlights subject outline, separates from background |
**Lighting direction modifiers**: key light, fill light, bounce, rim, spill, negative fill.
**Color temperature**: warm (tungsten, amber), cool (daylight, blue), mixed.
## Lens & Optical Effects
| Effect | Result |
|--------|--------|
| **Shallow depth of field** | Subject sharp, background bokeh |
| **Deep focus** | Everything sharp, foreground to background |
| **Wide-angle lens** (24-35mm) | Broader view, exaggerated perspective |
| **Telephoto** (85mm+) | Compressed perspective, subject isolation |
| **Anamorphic** | Stretched aspect, signature lens flares |
| **Lens flare** | Streaks from bright light hitting lens |
| **Rack focus** | Shift focus between subjects in-shot |
| **Fisheye** | Ultra-wide, barrel distortion |
## Style & Aesthetic References
### Cinematic Styles
- Film noir, period drama, thriller, modern romance
- Documentary, arthouse, experimental film
- Epic space opera, fantasy, horror
- 1970s romantic drama, 90s documentary-style
### Animation Styles
- Studio Ghibli / Japanese anime
- Classic Disney, Pixar-like 3D
- Stop-motion, claymation
- Hand-painted 2D/3D hybrid
- Cel-shaded, low-poly 3D
### Art Movements
- Impressionistic, surrealist, Art Deco, Bauhaus
- Watercolor, charcoal sketch, ink wash
- Graphic novel, blueprint schematic
### Film Stock / Grade
- Kodak warm grade, Fuji cool tones
- 16mm black-and-white, 35mm photochemical contrast
- Vintage grain overlay, halation on speculars
- Teal-and-orange color grade
## Temporal Effects
| Effect | Use |
|--------|-----|
| **Slow motion** | Emphasis, beauty, impact |
| **Time-lapse** | Passage of time, processes |
| **Freeze-frame** | Dramatic pause |
| **Rapid cuts** | Energy, urgency |
| **Continuous / long take** | Immersion, tension |
| **Fade in / fade out** | Scene transitions |
| **Match cut** | Visual continuity between scenes |
## Audio Descriptions
Models that support audio generation (LTX-2, Sora 2, VEO 3) respond to:
**Ambient**: wind, rain, traffic, crowd murmur, forest birds, mechanical hum
**Diegetic sound**: footsteps, door creaking, glass clinking, keyboard typing
**Voice style**: whisper, calm narration, energetic announcer, gravitas
**Music mood**: "soft piano in background", "upbeat electronic"
Put dialogue in quotation marks: `Character says: "Hello world."`
## What to Avoid
| Don't | Why | Do Instead |
|-------|-----|-----------|
| "Beautiful scene" | Too vague, no visual info | "Wet cobblestone street, warm streetlamp glow reflecting in puddles" |
| "Person moves quickly" | No visible action | "Woman sprints three steps and vaults over the railing" |
| "Cinematic look" | Every model already tries this | Specify: "anamorphic lens, shallow DOF, golden hour lighting" |
| "Sad character" | Internal states aren't visible | "Tears on cheek, shoulders slumped, staring at empty chair" |
| Readable text / logos | Models can't render text reliably | Avoid signs with text, or accept imperfect rendering |
| Complex physics | Chaotic motion causes artifacts | Keep physics simple; dancing/walking OK, explosions risky |
| Multiple characters talking | Multi-person dialogue breaks sync | One speaker per clip, or use reaction shots |
| Overloaded prompts | Too many elements = incoherent | Start simple, layer complexity one element at a time |
| Conflicting lighting | "Bright noon" + "dark shadows" | Pick one lighting setup and commit |
## Prompt Iteration Strategy
1. **Start simple** — subject + action + setting. See what the model gives you.
2. **Add one element at a time** — camera, then lighting, then style.
3. **If a shot misfires** — strip back. Freeze camera, simplify action, try again.
4. **For consistency across clips** — repeat the same style/lighting/grade description.
5. **Use seed values** — when you find a good result, save the seed for variations.
## Example: Generic Prompt Template
```
[Shot]: Medium close-up, slight low angle
[Camera]: Slow dolly-in
[Subject]: A weathered fisherman in his 60s, salt-and-pepper beard,
dark wool sweater, calloused hands gripping a rope
[Action]: He pulls the rope hand-over-hand, muscles straining,
then pauses and looks out to sea
[Setting]: Wooden dock at dawn, calm grey ocean, distant fog bank,
seagulls wheeling overhead
[Lighting]: Soft overcast with warm break in clouds on the horizon,
gentle rim light from the rising sun
[Style]: Documentary cinematography, 35mm film grain,
muted earth tones with a cold blue-grey palette
[Audio]: Rope creaking, water lapping, distant gull cries, wind
```
skills/creative/video-stitching.md
+307
View File
@@ -0,0 +1,307 @@
# Video Stitching Strategy Skill
## When to Use
Apply this skill when assembling multiple video clips into a unified output:
sequential narrative assembly, multi-take compilation, AI-generated clip chaining
(e.g., LTX-2 produces max ~8s per clip), or spatial compositions like side-by-side
comparisons and picture-in-picture commentary.
## Tools
| Tool | Role |
|------|------|
| `video_trimmer` | Cut segments to precise in/out points, concatenate clips (`concat` operation) |
| `video_compose` | Full composition with overlays, subtitles, audio mixing, spatial layouts |
| `frame_sampler` | Inspect frames at stitch boundaries for visual continuity |
| `audio_mixer` | Mix, duck, and crossfade audio tracks across stitch points |
| `scene_detect` | Find natural scene boundaries in source footage |
## When to Stitch — Decision Tree
```
Do you have multiple clips that need to become one video?
├── YES: Are they sequential (play one after another)?
│ ├── YES: Are they from the same shoot / same scene?
│ │ ├── YES → Multi-take assembly (pick best takes, stitch)
│ │ └── NO → Sequential narrative (match cuts, handle transitions)
│ └── NO: Do clips need to appear simultaneously on screen?
│ ├── YES → Spatial composition (side-by-side, PIP, stack)
│ └── MIXED → Hybrid (sequential with spatial inserts)
├── AI-generated clips (LTX-2, CogVideo)?
│ └── YES → AI clip chaining (handle 8s boundaries, maintain continuity)
└── NO → No stitching needed. Use video_trimmer for single-clip edits.
```
## Stitch Strategies
### 1. Sequential Stitching
Clips play one after another in timeline order. This is the most common strategy.
**When:** Narrative videos, multi-section explainers, compiled takes.
**Process:**
1. Order clips by narrative sequence (not filename)
2. Trim each clip to precise in/out points via `video_trimmer` (operation: `cut`)
3. Select transition type for each junction (see Transition Selection below)
4. Concatenate via `video_trimmer` (operation: `concat`) for hard cuts, or `video_compose` for transitions requiring filters
5. Verify audio continuity across all stitch points
**Audio continuity rules:**
- Match audio levels across clips before stitching (normalize to -16 LUFS)
- If background music spans multiple clips, mix it as a single track via `audio_mixer` and mux post-concat
- Never let music cut abruptly at a stitch point — crossfade or duck instead
### 2. Spatial Stitching
Multiple clips visible simultaneously on screen.
**When:** Reactions, comparisons, commentary, multi-angle coverage.
| Layout | FFmpeg Filter | Use Case |
|--------|---------------|----------|
| Side-by-side (duet) | `hstack` or `xstack` | Reaction videos, before/after |
| Vertical stack | `vstack` or `xstack` | Comparison (top vs bottom) |
| Picture-in-picture (PIP) | `overlay=x:y` via `video_compose` | Commentary, webcam + screen |
| Grid (2x2, 3x3) | `xstack` with layout string | Multi-angle, compilation |
**Spatial layout decision tree:**
```
What relationship do the clips have?
├── Reaction / response → Side-by-side (duet), main clip 70% width
├── Before / after → Side-by-side, equal 50/50 split
├── Comparison (A vs B) → Vertical stack or side-by-side depending on aspect ratio
├── Commentary over content → PIP, speaker in corner (20-25% frame size)
├── Multi-angle same event → Grid layout, synced to same timecode
└── Screen recording + face → PIP, face cam in bottom-right corner
```
**PIP placement rules:**
- Default position: bottom-right with 20px padding
- Size: 20-25% of frame width for commentary, 30-35% for equal importance
- Always ensure PIP does not cover critical content (subtitles, key visuals)
- Add a 2px border or subtle shadow to separate PIP from background
### 3. AI Clip Chaining (LTX-2 / CogVideo)
AI video generators produce short clips (LTX-2: ~8 seconds max). Stitching them
into longer sequences requires special care to maintain visual continuity.
**Process:**
1. Generate clips with overlapping prompts — last frame description of clip N should match first frame description of clip N+1
2. Use `frame_sampler` to extract the last frame of clip N and first frame of clip N+1
3. Visually inspect the pair for continuity breaks (color shift, subject position, background change)
4. If discontinuity is minor → use a 0.5-1.0s crossfade to smooth the junction
5. If discontinuity is major → insert a fade-through-black (0.5s out + 0.5s in) to signal scene transition
6. After stitching, apply a global color grade to unify the visual tone across clips
**AI clip chaining pitfalls:**
- AI clips may have inconsistent FPS — normalize all clips to the same FPS before stitching
- Color temperature often shifts between generations — apply consistent color grade post-stitch
- Motion direction may not match — review last/first frames for jarring movement reversals
- Audio (if any) will not be continuous — strip AI audio and use a single music/narration track
### 4. Hybrid Stitching
Sequential flow with spatial inserts at specific moments.
**When:** Explainer that switches to side-by-side for comparisons, tutorial that
shows PIP during demonstrations, documentary with occasional split-screen.
**Process:**
1. Plan the timeline: mark which segments are sequential and which are spatial
2. Render each spatial segment as a standalone composed clip via `video_compose` (overlay operation)
3. Treat the rendered spatial clips as regular clips in the sequential stitch
4. Concatenate everything in order using the sequential stitching process
## Transition Selection
### Decision Tree
```
What is the relationship between clip N and clip N+1?
│
├── Same scene, continuous action?
│ └── HARD CUT (0ms)
│
├── Same topic, different angle or take?
│ └── HARD CUT (0ms) — use J-cut or L-cut for audio smoothing
│
├── Topic change or new section?
│ └── CROSSFADE (0.5-1.0s)
│
├── Time passage or mood shift?
│ └── CROSSFADE (1.0-1.5s)
│
├── Major section break (intro→body, body→outro)?
│ └── FADE THROUGH BLACK (0.5-1.0s)
│
├── Dialogue transition between speakers?
│ └── L-CUT or J-CUT (audio leads or trails by 0.3-0.5s)
│
└── AI clip boundary (LTX-2 chain)?
├── Continuity is good → HARD CUT or short CROSSFADE (0.3-0.5s)
└── Continuity is broken → FADE THROUGH BLACK (0.5s)
```
### Transition Reference
| Transition | Duration | Implementation | Best For |
|-----------|----------|----------------|----------|
| Hard cut | 0ms | `video_trimmer` concat (codec: copy) | Same scene, fast pace, continuation |
| Crossfade | 0.5-1.5s | `video_compose` with `xfade` filter | Topic change, time passage, mood shift |
| Fade through black | 0.5-1.0s each | `video_compose`: fade out → black → fade in | Major section break, intro/outro |
| L-cut | 0.3-0.5s | Audio from clip N continues into clip N+1's video | Smooth dialogue exit, lingering emotion |
| J-cut | 0.3-0.5s | Audio from clip N+1 starts under clip N's video | Dialogue anticipation, building tension |
### Transition Duration by Content Pace
| Pacing | Crossfade | Fade Through Black |
|--------|-----------|-------------------|
| Fast (short-form, < 60s) | 0.3-0.5s | 0.3-0.5s |
| Medium (1-10 min) | 0.5-1.0s | 0.5-0.8s |
| Slow (documentary, > 10 min) | 1.0-1.5s | 0.8-1.0s |
## Audio Coordination
### Audio at Stitch Points
```
What audio exists at the stitch boundary?
│
├── Both clips have narration/dialogue?
│ ├── Hard cut → Ensure no audio pop (cut at zero-crossing or apply 5ms fade)
│ ├── Crossfade → Duck outgoing audio -6dB during overlap, bring in incoming
│ └── L-cut/J-cut → Blend: outgoing audio fades -∞dB over 0.3-0.5s
│
├── Music spans the stitch?
│ ├── Same track continues → Do not re-encode audio; use stream copy
│ ├── Track changes → Crossfade music 1.0-2.0s centered on the cut point
│ └── Music + narration → Duck music -12dB under narration at all times
│
├── One clip has audio, the other is silent?
│ └── Add a 0.3s fade-in/fade-out to avoid abrupt silence transitions
│
└── No audio on either clip?
└── No audio coordination needed. Add music/narration as a single track post-stitch.
```
### Audio Level Targets
| Content Type | Target LUFS | Headroom |
|-------------|-------------|----------|
| Narration / dialogue | -16 LUFS | -1 dB true peak |
| Background music (under narration) | -28 to -24 LUFS | -1 dB true peak |
| Music only (no narration) | -14 LUFS | -1 dB true peak |
| Sound effects | -20 LUFS | -1 dB true peak |
## Quality Checklist
Before declaring a stitch complete, verify every item:
- [ ] **Resolution match:** All input clips have the same resolution (or are scaled to match before stitching)
- [ ] **FPS match:** All input clips share the same frame rate (or are conformed with `fps` filter)
- [ ] **Aspect ratio consistency:** No mixed 16:9 / 9:16 / 4:3 unless intentional spatial layout
- [ ] **Color consistency:** No visible color temperature or exposure jumps at stitch boundaries
- [ ] **Audio level consistency:** All clips normalized to target LUFS before stitching
- [ ] **No audio pops or clicks:** Stitch points have micro-fades or are at zero-crossings
- [ ] **Transition appropriateness:** Transition type matches the content relationship (see decision tree)
- [ ] **Total duration check:** Final output duration matches expected sum (accounting for transition overlaps)
- [ ] **Codec consistency:** All clips use the same codec to allow stream copy; re-encode only if necessary
- [ ] **Playback test:** Scrub through every stitch point in the output and confirm smooth playback
## Common Pitfalls
### Codec Mismatch Causing Full Re-encode
**Problem:** Mixing clips encoded with different codecs (e.g., H.264 + H.265) or different
encoding parameters forces FFmpeg to re-encode everything during concat.
**Solution:** Before stitching, probe all clips with `ffprobe`. If codecs differ, re-encode
the minority clips to match the majority codec. This is faster than re-encoding everything.
```
Check: ffprobe -v error -select_streams v:0 -show_entries stream=codec_name,width,height,r_frame_rate -of csv=p=0 input.mp4
```
### Audio Drift in Long Stitches
**Problem:** When concatenating many short clips (10+), tiny timing mismatches accumulate,
causing audio to drift out of sync by the end.
**Solution:**
1. Re-encode each clip with constant frame rate before concatenation (`-vsync cfr`)
2. If using a separate audio track, align it to the final video's duration post-stitch
3. For AI clip chains, use `-async 1` to resync audio on concatenation
### Aspect Ratio Mixing
**Problem:** Stitching a 16:9 clip with a 9:16 clip creates letterboxing or stretching.
**Solution:** Decide on a target aspect ratio up front. Pad non-conforming clips with black
bars (`pad` filter) or crop them (`crop` filter) — never stretch.
### Variable Frame Rate (VFR) Sources
**Problem:** Screen recordings and phone footage often use VFR, which causes
desync and stuttering when stitched with CFR content.
**Solution:** Convert VFR sources to CFR before stitching:
`ffmpeg -i vfr_input.mp4 -vsync cfr -r 30 cfr_output.mp4`
### Concatenation with Stream Copy Fails
**Problem:** `video_trimmer` concat with `codec: copy` fails or produces glitchy output
when clips have different GOP structures or encoding parameters.
**Solution:** If stream copy fails, fall back to re-encoding with consistent parameters:
`-c:v libx264 -crf 18 -preset medium -c:a aac -b:a 192k`
Use CRF 18 (near-lossless) to avoid quality loss from the re-encode.
## Stitch Planning Template
When planning a stitch, produce this structure as part of `edit_decisions`:
```yaml
stitch_plan:
strategy: sequential | spatial | hybrid | ai_chain
target_resolution: "1920x1080"
target_fps: 30
target_codec: libx264
clips:
- id: clip_01
source: "assets/intro.mp4"
in_seconds: 0.0
out_seconds: 5.0
transition_out: crossfade
transition_duration: 0.8
- id: clip_02
source: "assets/section_1.mp4"
in_seconds: 0.0
out_seconds: 8.0
transition_out: hard_cut
- id: clip_03
source: "assets/section_2.mp4"
in_seconds: 0.0
out_seconds: 8.0
transition_out: fade_black
transition_duration: 0.5
audio:
narration: "assets/narration_full.wav"
music: "assets/bg_music.mp3"
music_volume: -24 # LUFS
ducking: true
spatial_inserts: # Only for hybrid strategy
- at_clip: clip_02
at_seconds: 3.0
layout: pip
overlay_source: "assets/webcam.mp4"
position: bottom_right
size_percent: 25
```
skills/creative/video-understand-usage.md
+124
View File
@@ -0,0 +1,124 @@
# Video Understanding Usage for OpenMontage
> Sources: OpenMontage video_understand tool implementation, CLIP/BLIP2/LLaVA model
> documentation, OpenCV image quality metrics
## Quick Reference Card
```
DEFAULT MODE: describe — generates captions for frames
FOR REVIEW: quality — assesses blur, brightness, contrast
FOR Q&A: qa mode with a query — "Is the speaker visible?" "Is the text readable?"
DEFAULT MODEL: clip (fastest, good for classification)
FOR DETAIL: blip2 or llava (slower, richer descriptions)
MAX FRAMES: 5 default for video — sample strategically, not exhaustively
```
## When to Use video_understand
- **Visual QA during review** — check rendered output quality before delivering
- **Footage analysis** — understand what's in user-provided footage before planning
- **Highlight extraction** — identify the most visually interesting frames
- **Quality gating** — programmatic check for blur, exposure, scene coherence
- **Scene classification** — categorize footage by content type
- **Asset validation** — verify generated images match the intended scene description
## Mode Selection
| Mode | What It Does | When to Use |
|------|-------------|-------------|
| `describe` | Generates a text description of the frame | Understanding footage content, logging |
| `qa` | Answers a specific question about the frame | Targeted checks ("Is text readable?", "Is face visible?") |
| `quality` | Measures blur, brightness, contrast numerically | Automated quality gating, comparing takes |
| `classify` | Categorizes the scene type | Sorting footage, pipeline routing |
### Quality Mode Metrics
| Metric | What It Measures | Bad | Good |
|--------|-----------------|-----|------|
| `blur_score` | Laplacian variance | Below 100 = blurry | Above 500 = sharp |
| `brightness` | Mean pixel value (0-255) | Below 50 = too dark, above 200 = overexposed | 50-200 |
| `contrast` | Pixel standard deviation | Below 30 = flat/washed out | Above 80 = good contrast |
## Model Selection
| Model | Speed | Capabilities | Best For |
|-------|-------|-------------|----------|
| `clip` | Fast | Classification, similarity matching | Quick scene categorization, batch processing |
| `blip2` | Medium | Detailed captions, visual QA | Understanding complex scenes, answering questions |
| `llava` | Slow | Most detailed understanding, reasoning | Deep analysis, subjective quality assessment |
### Model Selection Rules
- Use `clip` for batch operations and classification tasks
- Use `blip2` for describe and qa modes when detail matters
- Use `llava` only when you need the most thorough understanding
## Frame Selection for Video
- Default samples `max_frames` (5) evenly across the video
- Use `frame_indices` to target specific frames (e.g., check quality at specific timestamps)
- For quality review, sample the first frame, middle frame, and last frame minimum
## Common Workflows
### 1. Pre-Edit Footage Review
```
video_understand (describe, 10 frames) → inform scene_plan
```
Analyze user-provided footage before planning cuts or edits. Use `blip2` for detailed descriptions that inform the scene plan.
### 2. Post-Render Quality Gate
```
video_understand (quality) → pass/fail → re-render if needed
```
Run after composing the final video. Fail if any frame has blur_score < 100, brightness outside 50-200, or contrast < 30.
### 3. Highlight Selection
```
video_understand (describe, 20 frames) → rank by visual interest → select clips
```
Sample many frames, describe each, then select the most visually compelling segments for a montage or trailer.
### 4. Asset Validation
```
video_understand (qa, "Does this match: [scene description]?") → confirm or regenerate
```
After generating an image or video clip, verify it matches the intended scene description before proceeding.
### 5. Talking-Head Analysis
```
video_understand (qa, "Is the speaker's face clearly visible?") → face_enhance if needed
```
Check face visibility and framing before applying lip-sync or face restoration tools.
## Quality Checklist
- Descriptions accurately match what's in the frame
- Quality scores correlate with visual inspection (manually spot-check)
- QA answers are consistent across similar frames
- Classification categories are stable across adjacent frames
- No false positives in quality gating (good frames passing, bad frames failing)
## Applying to OpenMontage
When using the `video_understand` tool:
1. **Use `quality` mode as a post-render gate in the compose stage** — reject outputs below quality thresholds
2. **Use `describe` mode to analyze user-provided footage** at the start of the talking-head pipeline
3. **For batch quality checks, use `clip` model** (fastest) — switch to `blip2` only for detailed review
4. **Sample at least 3 frames for quality assessment** — beginning, middle, end
5. **Quality thresholds for passing:** blur_score > 100, brightness 50-200, contrast > 30
6. **Use `qa` mode to validate generated assets:** "Does this image show [expected content]?"
7. **In the review stage**, combine video_understand quality data with the reviewer skill's rubric
8. **Do NOT run video_understand on every frame of a long video** — sample strategically
skills/meta/checkpoint-protocol.md
+128
View File
@@ -0,0 +1,128 @@
# Checkpoint Protocol — Meta Skill
## When to Use
After completing a stage's work AND passing review. This skill teaches you when and how to checkpoint, and when to ask the human for approval. It replaces the Python `checkpoint_policy.py` with an instruction-driven protocol.
Checkpoints are the save points of a pipeline. They enable resume-from-failure, human oversight, and audit trails.
## Protocol
### Step 1: Check Manifest Policy
Read the current stage's configuration from the pipeline manifest:
```yaml
- name: idea
checkpoint_required: true # Must we checkpoint?
human_approval_default: true # Must we ask the human?
```
| `checkpoint_required` | `human_approval_default` | Action |
|----------------------|------------------------|--------|
| true | true | Checkpoint + present to human for approval |
| true | false | Checkpoint + proceed automatically |
| false | * | Skip checkpoint entirely (rare) |
### Step 2: Prepare Checkpoint Data
Gather everything needed for the checkpoint:
1. **Stage name** — which stage just completed
2. **Status** — `"completed"` (or `"awaiting_human"` if approval needed)
3. **Artifacts** — the canonical artifact(s) produced by this stage
4. **Metadata** — review findings, cost snapshot, timing info
### Step 3: Write Checkpoint
Call the checkpoint utility:
```python
write_checkpoint(
pipeline_dir, # Project working directory
project_name, # Project identifier
stage_name, # e.g., "idea"
status, # "completed" or "awaiting_human"
artifacts, # {"brief": {...}} — the stage's output
)
```
The checkpoint utility will:
- Validate the artifact against its schema
- Write the checkpoint JSON to disk
- Include timestamp and stage metadata
### Step 4: Human Approval (If Required)
When `human_approval_default: true`:
1. **Present a summary** to the human:
```
## Stage Complete: [stage_name]
### Artifact Summary
[Key details from the artifact — title, duration, key decisions]
### Review Findings
[Summary from reviewer: N critical (all fixed), N suggestions]
### Cost So Far
[Budget spent / total, breakdown by tool]
### Action Required
Please review and approve to continue, or provide feedback for revision.
```
2. **Wait for human response:**
- **Approved** → update checkpoint status to `"completed"`, proceed to next stage
- **Revision requested** → go back to the stage director skill with the human's feedback, produce revised artifacts, re-review, re-checkpoint
- **Abort** → stop the pipeline
3. **Approval stages** (which stages typically need human approval):
- `idea` — Always. The creative direction defines everything downstream.
- `script` — Always. The words are the foundation.
- `scene_plan` — Usually. Visual choices are subjective.
- `assets` — Rarely. Automated quality checks are sufficient.
- `edit` — Rarely. Technical assembly, not creative.
- `compose` — Rarely. But human may want to preview.
- `publish` — Always. Human must approve before anything goes public.
### Step 5: Determine Next Stage
After checkpoint is written and approved (if needed):
```python
next_stage = get_next_stage(pipeline_dir, project_name)
```
This reads all existing checkpoints and returns the next stage that needs to run, or `None` if the pipeline is complete.
### Step 6: Resume Protocol
At the START of any pipeline run (not just after a stage), always check for existing progress:
```python
next_stage = get_next_stage(pipeline_dir, project_name)
```
If `next_stage` is not the first stage:
1. Inform the human: "Found existing progress. Resuming from stage: [next_stage]"
2. Load prior artifacts from checkpoints for context
3. Continue from that stage
If a checkpoint exists with status `"awaiting_human"`:
1. Inform the human: "Stage [name] is awaiting your approval"
2. Present the checkpoint data for review
3. Wait for approval before proceeding
## Key Principles
1. **Always checkpoint completed work.** Even if `checkpoint_required: false`, consider checkpointing anyway if the stage took significant time or cost. Losing work is worse than an extra file on disk.
2. **Never skip human approval on creative stages.** `idea` and `script` shape everything. Rushing past them to save time produces videos nobody wants.
3. **Include cost snapshots.** The human should know how much has been spent and how much remains before approving expensive downstream stages (assets, compose).
4. **Checkpoints enable resume.** If the pipeline crashes at `compose`, the human can restart and it picks up from `compose` — not from `idea`. This is the whole point.
5. **Be transparent in approval requests.** Don't just show the artifact — show the review findings, the cost, and any concerns. Help the human make an informed decision.
skills/meta/reviewer.md
+116
View File
@@ -0,0 +1,116 @@
# Reviewer — Meta Skill
## When to Use
After completing any pipeline stage's work — before checkpointing. You are the quality gate between "work done" and "work accepted." This skill replaces the Python reviewer class with an instruction-driven self-review protocol.
Every stage gets reviewed. No exceptions. The review quality determines whether the final video is worth watching.
## Protocol
### Step 1: Load Review Context
Before reviewing, gather:
1. **Review focus items** from the pipeline manifest for this stage (`review_focus` field)
2. **Success criteria** from the manifest for this stage (`success_criteria` field)
3. **Active playbook** quality rules
4. **The artifact** produced by the stage
### Step 2: Schema Validation
First, the non-negotiable check:
- Validate the artifact against its JSON schema (`schemas/artifacts/<name>.schema.json`)
- If schema validation fails, this is a **critical** finding — fix immediately, do not proceed
### Step 3: Review Against Focus Items
For each `review_focus` item from the manifest:
1. Evaluate the artifact against this specific criterion
2. Assign a severity:
- **critical** — Must fix before proceeding. The artifact is broken, incomplete, or dangerously wrong.
- **suggestion** — Should fix. Improves quality significantly but doesn't block progress.
- **nitpick** — Could fix. Minor polish that's nice-to-have.
3. Write a specific, actionable finding (not vague)
**Good finding:** "Section 3 narration is 180 words for a 10-second window — that's 1080 wpm, impossible to speak. Cut to 25 words."
**Bad finding:** "Script might be too long."
### Step 4: Cross-Check Against Playbook
If a style playbook is active, verify:
- [ ] Color references match playbook palette
- [ ] Transition types are in the playbook's allowed set
- [ ] Pacing rules are respected (min/max durations)
- [ ] Asset descriptions include playbook style cues
- [ ] Quality rules are not violated
Each violation is a **suggestion** severity finding.
### Step 5: Evaluate Success Criteria
For each `success_criteria` item from the manifest:
- Is the criterion met? (yes/no/partial)
- If not met, create a **critical** finding
### Step 6: Make a Decision
Count findings by severity:
| Scenario | Action |
|----------|--------|
| 0 critical, any suggestions/nitpicks | **Pass** — proceed to checkpoint. Note suggestions for the record. |
| 1+ critical findings | **Revise** — fix all critical findings, then re-review (max 2 rounds). |
| After 2 revision rounds, still critical | **Pass with warnings** — proceed anyway, note unresolved issues. Never block indefinitely. |
### Step 7: Record Review
Structure your review as:
```
## Review: [stage_name] — Round [N]
**Decision:** PASS / REVISE / PASS_WITH_WARNINGS
### Findings
1. [CRITICAL] Title of finding
- Description: What's wrong
- Action: What to fix
- Status: pending / fixed / accepted / deferred
2. [SUGGESTION] Title of finding
- Description: What could be better
- Action: How to improve
- Status: pending / accepted / deferred
### Summary
- Critical: N (N fixed)
- Suggestions: N
- Nitpicks: N
- Playbook violations: N
- Success criteria met: N/M
```
## Key Principles
1. **Be specific, not vague.** "The hook is weak" is useless. "The hook asks a question but doesn't create urgency — try leading with the surprising stat from key_point #2" is actionable.
2. **Critical means critical.** Don't inflate severity. A missing schema field is critical. A slightly wordy paragraph is a suggestion. A comma splice is a nitpick.
3. **Two rounds max.** The goal is shipping, not perfection. After two revision rounds, pass with warnings and move on. Perfectionism kills pipelines.
4. **Review the artifact, not the process.** You're checking the output, not how it was produced. If the brief is compelling, it doesn't matter if the agent used an unusual approach.
5. **Playbook is law.** If the playbook says "no more than 3 colors on screen," that's not a suggestion — it's a constraint. Violations are always flagged.
## Stage-Specific Review Guidance
| Stage | What matters most |
|-------|-----------------|
| idea | Hook uniqueness, research depth, angle diversity |
| script | Timing accuracy, narrative arc, enhancement cue density |
| scene_plan | Full coverage, visual variety, asset feasibility |
| assets | File existence, style consistency, budget adherence |
| edit | Timeline coverage, audio sync, subtitle presence |
| compose | Playability, duration accuracy, audio quality |
| publish | SEO quality, metadata completeness, export packaging |
skills/meta/skill-creator.md
+112
View File
@@ -0,0 +1,112 @@
# Skill Creator — Meta Skill
## When to Use
When you encounter a situation where no existing skill covers what you need to do, and the gap is reusable — not a one-off task. This skill teaches you to create new skills dynamically during a pipeline run.
Examples of when to create a new skill:
- A scene requires a visualization technique not covered by existing skills (e.g., "interactive map animation")
- A new tool is available but has no usage skill (e.g., a new TTS provider)
- A recurring pattern emerges across multiple stages that could be codified
Do NOT create a skill for:
- One-off tasks that won't recur
- Things already covered by an existing skill
- Pure tool configuration (that belongs in tool code)
## Protocol
### Step 1: Identify the Gap
Clearly articulate:
1. **What you need to do** that no existing skill covers
2. **Why it's reusable** — will future pipelines benefit?
3. **Where it fits** — which layer and directory?
### Step 2: Research Best Practices
Before writing the skill, research:
1. **Web search** for established approaches to this task
2. **Existing skills** in the repo for structural patterns
3. **Tool documentation** for any tools the skill will reference
### Step 3: Choose the Skill Type
| Type | Directory | Purpose |
|------|-----------|---------|
| Stage director | `skills/pipelines/<pipeline>/` | How to execute a specific pipeline stage |
| Meta skill | `skills/meta/` | Cross-cutting protocol (review, checkpoint, etc.) |
| Tool skill | `.agents/skills/` | How to use a specific API or tool effectively |
| Style skill | `styles/` | Visual/audio style definition (YAML playbook) |
### Step 4: Write the Skill
Follow this structure (adapt to skill type):
```markdown
# [Skill Name] — [Context]
## When to Use
[1-2 sentences: what situation triggers this skill]
## Prerequisites
[Table of required resources: schemas, prior artifacts, tools, other skills]
## Process
### Step 1: [First action]
[Clear instructions with examples]
### Step 2: [Second action]
[Clear instructions with examples]
...
### Step N: Self-Evaluate
[Quality rubric: scoring table with 1-5 scale]
### Step N+1: Submit
[How to persist the output]
## Common Pitfalls
[Bullet list of mistakes to avoid]
```
### Key Principles for Good Skills
1. **Teach thinking, not just doing.** A skill that says "generate an image" is useless. A skill that says "consider the emotional beat of this scene, research how top creators visualize this concept, then craft a prompt that includes the playbook's style anchors" is valuable.
2. **Include examples.** Show what good output looks like. Show what bad output looks like. The contrast teaches more than instructions alone.
3. **Reference concrete resources.** Don't say "check the schema." Say "validate against `schemas/artifacts/brief.schema.json`." Don't say "use a tool." Say "use `image_selector` with the playbook's `image_prompt_prefix`."
4. **Include a self-evaluation rubric.** Every skill should end with a scoring table. This forces the agent to check its own work before proceeding.
5. **Document pitfalls.** What goes wrong when this skill is executed poorly? Warn explicitly.
6. **Be opinionated.** A skill that says "you could do A or B" is less useful than one that says "do A because [reason], but fall back to B when [condition]."
### Step 5: Register the Skill
After writing the skill file:
1. Add an entry to `skills/INDEX.md`
2. If it's a pipeline stage skill, ensure the pipeline manifest references it in the stage's `skill` field
3. If it's a tool skill, place it in `.agents/skills/<tool-name>/`
### Step 6: Validate
Quick checks:
- [ ] File is well-formed markdown with clear headers
- [ ] All referenced schemas, tools, and resources exist
- [ ] Process steps are numbered and actionable
- [ ] Self-evaluation rubric is present
- [ ] Common pitfalls section is present
- [ ] No orphan references (everything mentioned exists in the repo)
## Common Pitfalls
- **Creating skills for one-off tasks**: If you'll never do this again, just do it inline. Skills are for patterns.
- **Vague instructions**: "Make it good" is not a skill. Specificity is what makes a skill useful.
- **No self-evaluation**: Without a rubric, the agent has no way to know if it followed the skill correctly.
- **Missing prerequisites**: A skill that references `image_selector` but doesn't list it in prerequisites will confuse future agents.
- **Over-engineering**: A 500-line skill for a simple task is worse than a 50-line one. Match complexity to the task.
skills/pipelines/animation/asset-director.md
+70
View File
@@ -0,0 +1,70 @@
# Asset Director - Animation Pipeline
## When To Use
This stage prepares the actual animated ingredients: narration, diagrams, math renders, motion backgrounds, code visuals, and reusable type or layout systems.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Tool path and beat map |
| Tools | `tts_selector`, `image_selector`, `video_selector`, `math_animate`, `diagram_gen`, `code_snippet`, `music_gen` — selectors auto-discover all available providers from the registry | Asset production options |
| Playbook | Active style playbook | Visual consistency |
## Process
### 1. Start With Deterministic Assets
Prefer the lowest-variance useful path:
- `diagram_gen` before generic image generation for structured diagrams,
- `code_snippet` for code scenes,
- `math_animate` for real math motion,
- provided artwork before new generation.
### 1b. Sample Preview (Prevents Wasted Spend)
Before batch-generating assets, produce one sample of each expensive type and show the user:
1. **TTS sample** (if narration-led): Generate one section. Confirm voice and tone before batching.
2. **Visual sample**: Generate one representative scene visual (diagram, illustration, or motion background). Confirm style and quality before batching the rest.
If rejected, adjust parameters and retry (max 3 iterations). Do not batch until approved.
### 2. Build Reusable Systems
Create once:
- typography treatments,
- lower-third or label styles,
- repeated motif assets,
- background containers.
### 3. Narration Is Optional, But The Plan Must Be Explicit
If the project is narration-led, produce or source narration. If it is text-led or music-led, say so clearly in metadata.
### 4. Use Metadata For Feasibility Truth
Recommended metadata keys:
- `tool_path_map`
- `reusable_assets`
- `narration_assets`
- `scene_asset_index`
- `blocked_assets`
### 5. Quality Gate
- the asset path is explicit per scene,
- reusable assets are actually reused,
- missing capabilities are surfaced honestly,
- every referenced file exists.
## Common Pitfalls
- Using high-variance generation when a deterministic asset would work better.
- Rebuilding the same title or label system repeatedly.
- Hiding failed asset paths instead of reporting them.
skills/pipelines/animation/compose-director.md
+47
View File
@@ -0,0 +1,47 @@
# Compose Director - Animation Pipeline
## When To Use
Render the animation with an emphasis on text sharpness, timing integrity, and consistent output cadence.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | Timing plan and asset files |
| Tools | `video_compose`, `audio_mixer`, `video_stitch` | Final assembly |
| Playbook | Active style playbook | Render consistency |
## Process
### 1. Preserve Motion Timing
Do not let export settings or careless composition change the perceived timing of holds, stagger, or scene transitions.
### 2. Protect Text And Diagram Sharpness
Animation often fails on export through soft text, muddy thin lines, or cramped mobile framing.
### 3. Verify The First And Last Frames
Ensure:
- the opening frame reads immediately,
- the final frame lands cleanly,
- nothing important is clipped by safe zones.
### 4. Use Render Metadata
Recommended metadata keys:
- `render_fps`
- `sharpness_checks`
- `safe_zone_checks`
- `variant_outputs`
## Common Pitfalls
- Soft or aliased text after rendering.
- Compression choices that damage diagrams.
- Scene cadence changing between preview and final.
skills/pipelines/animation/edit-director.md
+54
View File
@@ -0,0 +1,54 @@
# Edit Director - Animation Pipeline
## When To Use
This stage turns the scene plan into an animatic-grade edit plan. Timing is the product.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/edit_decisions.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["assets"]["asset_manifest"]`, `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]` | Assets, timing plan, and beats |
| Playbook | Active style playbook | Motion and typography rules |
## Process
### 1. Protect Hold Time
After key reveals, plan enough time for the viewer to process the frame. Do not stack every scene edge to edge with motion.
### 2. Stagger Secondary Elements
Primary element first, supporting elements second. The edit decisions should reinforce hierarchy.
### 3. Keep Motion Meaningful
Motion should signal:
- emphasis,
- transition,
- transformation,
- contrast.
### 4. Use Metadata For Timing Detail
Recommended metadata keys:
- `hold_windows`
- `stagger_rules`
- `transition_map`
- `scene_timing_notes`
### 5. Quality Gate
- key information has enough dwell time,
- movement clarifies hierarchy,
- transitions stay consistent,
- the edit remains readable on the target platform.
## Common Pitfalls
- Overcrowding the timeline with continuous motion.
- Revealing all elements at once.
- Letting stylistic motion reduce readability.
skills/pipelines/animation/executive-producer.md
+420
View File
@@ -0,0 +1,420 @@
# Executive Producer — Animation Pipeline
## When to Use
You are the **Executive Producer (EP)** for a generated animation video. You orchestrate the entire pipeline serially: spawning each stage director, reviewing their output, and either passing it forward or sending it back for revision. You are the stateful brain; the directors are stateless workers.
**You replace the default parallel/sequential execution model.** Instead of running all stages blindly, you exercise judgment at every gate.
## Why This Exists
Animation pipelines have unique failure modes that parallel execution cannot catch:
- Motion consistency breaks when scenes are generated independently
- Mathematical accuracy errors compound if not caught after script
- Animation timing requires hold times and reveals that get squeezed out without cross-stage awareness
- Reuse strategy degrades when each stage plans independently
- Budget allocation between AI-generated assets and free programmatic animation needs active management
- Text readability and diagram sharpness must be verified at compose time, not assumed
The EP solves all of these by maintaining cumulative state and applying animation-specific judgment at each gate.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/animation.yaml` | Stage definitions, review focus, success criteria |
| Skills | All 9 director skills + `meta/reviewer` | Stage execution knowledge |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
| Tools | Full tool registry | Available capabilities |
## Cumulative State
The EP maintains a running state object that flows through the entire pipeline:
```
EP_STATE:
pipeline: animation
playbook: <selected playbook name>
target_duration_seconds: <from proposal_packet.selected_concept>
budget_total_usd: <from proposal_packet.approval.approved_budget_usd or configured limit>
budget_spent_usd: 0.0
budget_remaining_usd: <budget_total>
# Animation-specific state
animation_mode: <manim | remotion | ai_video | diagram_stills | mixed>
reuse_strategy:
recurring_motifs: []
layout_system: null
transition_family: null
typography_hierarchy: null
unique_scene_count: 0
reused_template_count: 0
math_accuracy_notes: [] # constraints from research on what NOT to oversimplify
# Accumulated from each stage (8 stages)
artifacts:
research: null # → research_brief
proposal: null # → proposal_packet (includes approval gate)
script: null # → script
scene_plan: null # → scene_plan
assets: null # → asset_manifest
edit: null # → edit_decisions
compose: null # → render_report
publish: null # → publish_log
# Pre-production context (carried forward from research + proposal)
research_brief: null # full research_brief artifact
selected_concept: null # the approved concept from proposal_packet
production_plan: null # the approved tool/provider plan
approved_budget_usd: null # explicit user-approved spend cap
# Cross-stage tracking
narration_durations: {} # section_id → actual_seconds
total_narration_seconds: 0
total_visual_seconds: 0
style_anchors: {} # consistency tokens carried forward
revision_counts: {} # stage_name → number of revisions
issues_log: [] # all issues found, with resolution status
```
## Execution Protocol
### Phase 0: Initialize
1. Load the pipeline manifest (`animation.yaml`)
2. Load the playbook (from user selection or default)
3. Set budget from configuration or user input (default: $2.00)
4. Initialize EP_STATE
### Phase 1: Execute Stages Serially
For each stage in order: `research → proposal → script → scene_plan → assets → edit → compose → publish`
**Pre-production stages (research, proposal)** run before any money is spent:
- **research** gathers topic data AND animation technique references via web search — zero cost
- **proposal** presents concepts with animation mode selection and costs to the user — zero cost, but contains the **approval gate**
- The pipeline MUST NOT proceed past proposal without `approval.status == "approved"` or `"approved_with_changes"`
After proposal approval, extract and store in EP_STATE:
- `selected_concept` from `proposal_packet.selected_concept`
- `animation_mode` from `selected_concept.animation_mode`
- `reuse_strategy` from `selected_concept.reuse_strategy`
- `production_plan` from `proposal_packet.production_plan`
- `approved_budget_usd` from `proposal_packet.approval.approved_budget_usd`
- `playbook` from `proposal_packet.selected_concept → suggested_playbook`
- `math_accuracy_notes` from research_brief (if applicable)
```
EXECUTE_STAGE(stage_name):
1. PREPARE
- Load the director skill for this stage
- Inject EP_STATE as context (prior artifacts, budget remaining, style anchors, animation mode, reuse strategy)
- Inject any EP feedback from previous revision attempts
2. SPAWN DIRECTOR
- The director executes its full process (as defined in its skill MD)
- Director produces an artifact
3. REVIEW (EP performs this, not a separate reviewer)
- Schema validation against artifact schema
- Check review_focus items from pipeline manifest
- Check success_criteria from pipeline manifest
- Cross-check against playbook constraints
- Run EP-SPECIFIC CROSS-STAGE CHECKS (see below)
4. GATE DECISION
If PASS:
- Store artifact in EP_STATE
- Update cumulative tracking (budget, durations, etc.)
- Log: "[stage] PASSED — moving to next stage"
- Continue to next stage
If REVISE:
- Increment revision_counts[stage_name]
- If revision_counts[stage_name] >= 3:
- PASS WITH WARNINGS (never block forever)
- Log unresolved issues
- Else:
- Compose specific feedback for the director
- Re-run SPAWN DIRECTOR with feedback injected
- Re-run REVIEW
If SEND_BACK(target_stage):
- Only used when a downstream discovery invalidates upstream work
- Re-execute from target_stage forward (artifacts after target are invalidated)
- Max 1 send-back per stage pair (prevent infinite loops)
```
### Phase 2: Final Quality Assurance
After all stages complete, the EP performs a holistic review:
```
FINAL_QA:
1. PROBE the output video:
- Duration: within ±5% of target?
- Resolution: matches media profile?
- Audio: narration audible throughout? Music balanced?
- File: valid container, reasonable size?
2. TEXT AND DIAGRAM SHARPNESS (ANIMATION-SPECIFIC):
- Are text elements readable at target resolution?
- Are diagram lines crisp, not blurry from scaling?
- Are mathematical symbols rendered correctly?
- Is typography hierarchy maintained across scenes?
3. MOTION CONSISTENCY:
- Do transitions follow the declared transition family?
- Are hold times preserved (not squeezed by timing)?
- Do staggered reveals play correctly?
- Is the pacing animation-friendly (not rushed)?
4. STYLE CONSISTENCY:
- Do all scenes follow the reuse strategy?
- Is the color palette consistent?
- Do recurring motifs appear correctly across scenes?
5. MATHEMATICAL ACCURACY (if applicable):
- Do animated formulas/diagrams match the research brief's accuracy notes?
- Are any simplifications flagged in the research still correct?
6. BUDGET RECONCILIATION:
- Total actual spend vs. budget
- Log per-stage cost breakdown
7. DECISION:
If all checks pass → APPROVE for publish stage
If issues found → Send back to the specific stage(s) that can fix them
- Text/diagram issues → compose director (re-render) or asset director (regenerate)
- Motion issues → edit director (re-time) or scene director (replan)
- Audio issues → compose director
- Duration issues → script director (rewrite)
- Math errors → script director (fix content) then cascade forward
```
## EP-Specific Cross-Stage Checks
These checks use information accumulated across stages — something no individual director can do.
### After RESEARCH stage:
```
CHECK: Research depth
- At least 3 data_points with source URLs?
- At least 3 angles_discovered with grounded_in references?
- At least 2 animation technique references?
- At least 5 sources cited?
- If any minimum not met: REVISE research
- Note: Do NOT checkpoint with user — research is informational, not a decision point
```
### After PROPOSAL stage:
```
CHECK: Approval gate (CRITICAL)
- Is approval.status == "approved" or "approved_with_changes"?
- If "pending" or "rejected": STOP. Present to user and wait.
- If "approved_with_changes": apply modifications before proceeding
- Extract: animation_mode, reuse_strategy, target_duration, playbook, budget, tool selections
CHECK: Animation mode feasibility
- Does the selected animation mode's required tools exist in the registry?
- If Manim mode selected: is math_animate available?
- If Remotion mode selected: is video_compose (Remotion) available?
- If AI video mode selected: are video generation providers available?
- If any required tool is unavailable: alert user, offer alternatives
CHECK: Reuse strategy validity
- Does the reuse strategy define recurring motifs?
- Is the unique-to-template ratio reasonable (aim for ≤ 3:1)?
```
### After SCRIPT stage:
```
CHECK: Word count vs. duration target
- Calculate: total_words / 150 = estimated_minutes
- If estimated_minutes > target_duration * 1.15:
REVISE script: "Script is {X} words → {Y}min. Target: {Z}min. Cut {N} words."
- If estimated_minutes < target_duration * 0.7:
REVISE script: "Script is too short. Add {N} words."
CHECK: Animation beat structure
- Does each section express ONE clear visual idea?
- Are hold times budgeted (not every second filled with new information)?
- Is on-screen text concise (phrases, not paragraphs)?
CHECK: Mathematical accuracy (if applicable)
- Does the script's explanation match the research brief's accuracy notes?
- Are any simplifications technically defensible?
- If inaccurate: REVISE script with specific correction from research
```
### After SCENE_PLAN stage:
```
CHECK: Total scene duration covers full script
- Sum all scene durations
- Compare to script's total duration
- If gaps > 1 second: REVISE scene_plan
- If overlaps: REVISE scene_plan
CHECK: Animation mode adherence
- Does every scene specify which animation mode/tool it uses?
- Are mode choices consistent with the proposal's selected mode?
- If mixed mode: are transitions between modes planned?
CHECK: Reuse strategy enforcement
- Does the scene plan reference the recurring motifs from the proposal?
- Are templates reused where specified?
- If every scene is unique: flag as potential over-complexity
CHECK: Visual variety within constraints
- Count consecutive same-type scenes
- If > 3 consecutive: REVISE scene_plan
```
### After ASSETS stage:
```
CHECK: Narration duration feedback loop (CRITICAL)
- For each TTS audio file, probe actual duration
- Store in EP_STATE.narration_durations
- For each section:
If actual_duration > planned_duration * 1.15:
Option A: SEND_BACK to script director
Option B (within 25% over): Adjust scene_plan durations
- Update EP_STATE.total_narration_seconds
CHECK: Budget gate
- If budget_spent > budget_total * 0.9 and stages remain:
Alert: "90% budget consumed with {N} stages remaining"
Adjust remaining stages to free/cheap alternatives
CHECK: Style consistency
- Compare visual styles across all generated assets
- Are recurring motifs visually consistent?
- Store style_anchors for downstream use
CHECK: Programmatic asset integrity (if Manim/Remotion)
- Did math_animate or video_compose succeed without errors?
- Are output files valid and correctly sized?
```
### After EDIT stage:
```
CHECK: Timeline completeness
- Verify edit decisions cover 0 to total_duration with no gaps
- Verify all asset references point to existing files
- Verify audio ducking is configured for all narration segments
CHECK: Hold time preservation (ANIMATION-SPECIFIC)
- Verify hold times from scene_plan are preserved in edit decisions
- Verify staggered reveals are not compressed
- Verify motion serves hierarchy, not decoration
CHECK: A/V sync pre-validation
- For each cut: narration_start aligns with visual_start (±0.5s)
- For each scene: narration_duration ≤ visual_duration
```
### After COMPOSE stage:
```
CHECK: Output validation
- ffprobe the output: duration, resolution, codec, audio channels
- If duration drift > 5%: investigate which stage caused it
- If audio missing: check audio_mixer configuration
- If resolution wrong: check media profile selection
CHECK: Text and diagram sharpness (ANIMATION-CRITICAL)
- Text must be readable at target resolution
- Diagram lines must be crisp (no scaling artifacts)
- Mathematical symbols must render correctly
- If any text/diagram is blurry: REVISE compose with resolution/scaling adjustments
```
## Feedback Message Templates
### To Script Director:
```
EP FEEDBACK — Script Revision Required
Reason: {reason}
Specific issue: {detail}
Constraint: {word_count_limit / duration_target / math_accuracy}
Animation mode: {current mode — affects how text and beats should be structured}
Keep: {what was good}
Change: {what specifically needs to change}
```
### To Scene Director:
```
EP FEEDBACK — Scene Plan Revision Required
Reason: {reason}
Affected scenes: {scene_ids}
Animation mode: {current mode}
Reuse strategy: {what motifs/templates should be reused}
Available tools: {current tool registry status}
```
### To Asset Director:
```
EP FEEDBACK — Asset Regeneration Required
Reason: {reason}
Affected assets: {asset_ids}
Style anchors: {consistency requirements}
Animation mode: {current mode — affects which tools to use}
Budget remaining: ${remaining}
```
### To Compose Director:
```
EP FEEDBACK — Re-render Required
Reason: {reason}
Specific issue: {text_sharpness / motion_timing / audio_sync / etc.}
Expected: {what the output should be}
Actual: {what was produced}
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | research | Data depth, technique references, angle diversity | Revise research |
| G2 | proposal | Concept quality, mode feasibility, user approval | Revise proposal OR wait for user |
| G3 | script | Word count, beat structure, math accuracy | Revise script |
| G4 | scene_plan | Coverage, mode adherence, reuse strategy, variety | Revise scene_plan |
| G5 | assets | Narration duration, budget, style, asset integrity | Revise assets OR send-back to script |
| G6 | edit | Timeline completeness, hold times, A/V pre-sync | Revise edit |
| G7 | compose | Output probe, text sharpness, motion timing | Revise compose OR send-back |
| G8 | publish | Metadata, packaging, animation-mode tags | Revise publish |
| FINAL | all | Holistic review: sharpness, motion, accuracy, style | Send-back to specific stage |
## Execution Limits (Anti-Loop Protection)
| Limit | Value | Rationale |
|-------|-------|-----------|
| Max revisions per stage | 3 | Prevent perfectionism loops |
| Max send-backs per stage pair | 1 | Prevent ping-pong |
| Max total send-backs | 3 | Cap total re-work |
| Max total budget | Configurable (default $2) | Hard stop on spending |
| Max total wall-time | 15 minutes | Timeout for entire pipeline |
After any limit is hit: **proceed with warnings**, never block indefinitely.
## Integration with Existing Skills
The EP doesn't replace any director skill — it wraps them. Each director skill continues to work exactly as documented. The EP adds:
1. **Context injection**: Directors receive EP_STATE with cross-stage information
2. **Feedback injection**: Directors receive specific revision instructions when sent back
3. **Budget awareness**: Directors receive remaining budget and adjust tool choices
4. **Animation mode context**: Directors know the selected mode and reuse strategy
5. **Style anchors**: Directors receive consistency tokens from prior stages
6. **Math accuracy notes**: Directors receive constraints on technical accuracy
## Common Pitfalls
- **Over-revising**: A "good enough" animation in the right mode is better than a "perfect" one after 5 rounds.
- **Ignoring text sharpness**: The #1 animation quality issue. Always verify text readability at final resolution.
- **Letting reuse strategy erode**: If the proposal specified 3 templates, the scene plan should use 3 templates, not 8 unique designs.
- **Not probing outputs**: Always ffprobe the final video. Never trust metadata alone.
- **Losing animation mode context**: If the proposal selected Manim, every downstream stage should know it's a Manim project. Don't let stages default to generic image_selector when programmatic animation was approved.
- **Skipping math accuracy checks**: For technical topics, this is non-negotiable. A wrong animation is worse than no animation.
skills/pipelines/animation/idea-director.md
+72
View File
@@ -0,0 +1,72 @@
# Idea Director - Animation Pipeline
## When To Use
Use this pipeline when the video should be built primarily through designed motion: motion graphics, kinetic typography, diagram-led explainers, math visuals, or illustrative animation.
Do not use this pipeline when the project is really footage-led with a few overlays. That belongs in `hybrid`.
## Reference Inputs
- `docs/animation-best-practices.md`
- `skills/creative/animation-pipeline.md`
- `skills/creative/storytelling.md`
## Process
### 1. Classify The Animation Mode
Choose the primary mode:
- `diagrammatic`
- `motion_graphics`
- `kinetic_type`
- `math_animation`
- `illustrative`
- `mixed_animation`
### 2. Decide The Visual Path Early
Figure out which tools are supposed to do the work:
- `diagram_gen`
- `math_animate`
- `code_snippet`
- `image_selector`
- `video_selector` or a concrete video provider tool
- source-provided art
If the requested mode depends on unavailable tools, say so in the brief metadata immediately.
### 3. Choose A Reuse Strategy
Animation gets expensive when every scene is unique. Define:
- recurring motifs,
- layout system,
- transition family,
- typography hierarchy.
### 4. Build The Brief
Recommended metadata keys:
- `animation_mode`
- `visual_path`
- `narration_strategy`
- `reuse_strategy`
- `timing_style`
- `blocked_capabilities`
### 5. Quality Gate
- the animation mode is explicit,
- the visual path is feasible,
- the project is designed for reuse,
- the brief is honest about missing tools.
## Common Pitfalls
- Treating all animation as one generic category.
- Planning bespoke visuals for every scene.
- Hiding missing tool paths until the asset stage.
skills/pipelines/animation/proposal-director.md
+263
View File
@@ -0,0 +1,263 @@
# Proposal Director — Animation Pipeline
## When to Use
You are the **Proposal Director** for a generated animation video. You sit between the Research Director and the Script Director. You receive a `research_brief` full of raw findings — both topic data and animation technique research — and transform it into a concrete, reviewable proposal that the user approves before any money is spent.
**This is the approval gate.** Nothing downstream runs until the user says "go."
Animation proposals have a unique dimension: **animation mode selection**. Unlike explainer videos where the visual approach is secondary to the narrative, animation videos ARE their visual approach. The mode choice (Manim vs Remotion vs AI video vs motion graphics) fundamentally shapes the entire production.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/proposal_packet.schema.json` | Artifact validation |
| Prior artifact | `research_brief` from Research Director | Raw findings + technique research |
| Pipeline manifest | `pipeline_defs/animation.yaml` | Stage and tool definitions |
| Tool registry | `support_envelope()` output | What's actually available right now |
| Cost tracker | `tools/cost_tracker.py` | Cost estimation data |
| Style playbooks | `styles/*.yaml` | Available visual styles |
| User input | Topic, any preferences expressed | Creative direction |
## Process
### Step 1: Absorb the Research
Read the `research_brief` thoroughly. Extract:
- **`research_summary`** — read first. Contains both the key insight and the most promising animation approach.
- **`angles_discovered`** — raw concept candidates, each with an `animation_fit` field.
- **`data_points`** — especially those with high `visual_potential` ratings.
- **Animation technique references** — from the animation-specific research step. These directly inform mode selection.
- **`audience_insights.misconceptions`** — animation excels at showing "wrong way → right way" transitions.
- **Mathematical/technical accuracy notes** — critical constraints on what we can and cannot simplify.
### Step 2: Run Preflight
Before designing concepts, know what tools are available:
```bash
python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.support_envelope(), indent=2))"
```
Also check the capability catalog:
```bash
python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.capability_catalog(), indent=2))"
```
**Animation-specific preflight checks:**
| Capability | What to Check | Impact if Missing |
|------------|---------------|-------------------|
| `math_animate` | Is ManimCE installed and working? | Cannot do programmatic math animation — fall back to diagram_gen + image_selector |
| `diagram_gen` | Is Mermaid rendering available? | Cannot do diagram-led animation — fall back to image_selector |
| `video_selector` | Which video gen providers are available? | Limits AI video clip options |
| `image_selector` | Which image gen providers are available? | Limits still frame options |
| `tts_selector` | Which TTS providers are available? | Affects narration quality |
| `video_compose` | Is FFmpeg/Remotion available? | Critical — cannot render without this |
Record all findings. **Do not propose an animation mode that requires tools you don't have.**
### Step 3: Animation Mode Decision Matrix
This is the key differentiator from the explainer proposal. For each viable animation mode, evaluate:
| Mode | Best For | Tool Required | Visual Quality | Cost | Iteration Speed |
|------|----------|---------------|----------------|------|-----------------|
| **Manim (ManimCE)** | Math, physics, geometry, algorithms | `math_animate` | Precise, programmatic | Free (local) | Fast (code-driven) |
| **Remotion** | Data viz, charts, React components, kinetic type | `video_compose` (Remotion mode) | Smooth, web-native | Free (local) | Fast (code-driven) |
| **AI Video Generation** | Abstract concepts, metaphors, transitions | `video_selector` providers | Variable, cinematic | $0.05-0.50/clip | Slow (generation time) |
| **Diagram + Image Stills** | Process flows, architecture, comparisons | `diagram_gen` + `image_selector` | Clean, reliable | $0-0.05/image | Fast |
| **Mixed Mode** | Complex topics needing multiple techniques | Multiple tools | Varied | Varies | Moderate |
**Mode selection rules:**
- If the topic involves math/formulas/geometry → prefer Manim
- If the topic involves data/statistics/charts → prefer Remotion or diagram_gen
- If the topic is abstract/conceptual → consider AI video for key moments
- If the topic is process/workflow → prefer diagram builds
- Always check tool availability before committing to a mode
- Mixed mode is valid when different sections need different approaches
### Step 4: Design Concept Options
Build **at least 3 genuinely different concepts.** Start from the `angles_discovered` in the research brief and the animation mode analysis.
For each concept, specify:
#### 4a: Title and Hook
**Hook construction patterns for animation:**
| Pattern | Template | When to Use |
|---------|----------|-------------|
| **Visual surprise** | "Watch [thing] transform into [unexpected thing]." | When the animation itself IS the hook |
| **Misconception flip** | "You've been visualizing [topic] wrong. Here's what it actually looks like." | When common mental models are wrong |
| **Progressive reveal** | "Start with [simple]. End with [complex]. Every step animated." | When the topic has layered complexity |
| **Impossible camera** | "What if you could see [invisible process] happening in real time?" | When animation reveals the unseeable |
| **Data surprise** | "[Counterintuitive number]. Watch it happen." | When animated data is more powerful than static |
**Rules:**
- Hook must be under 20 words
- Hook must promise a VISUAL experience, not just information
- Hook must be grounded in a specific research finding
#### 4b: Animation Mode and Approach
For each concept, specify:
- **Primary animation mode**: manim / remotion / ai_video / diagram_stills / mixed
- **Why this mode**: grounded in technique research from the brief
- **Reuse strategy**: What's the visual system? (recurring motifs, layout grid, color scheme, transition family)
- **Complexity estimate**: How many unique scene types vs. reusable templates?
#### 4c: Narrative Structure
Choose from: `myth_busting`, `problem_solution`, `data_narrative`, `comparison`, `timeline`, `journey`, `analogy`, `progressive_build`, `transformation`
**Animation-specific structure: `progressive_build`** — start simple, add complexity layer by layer. This is the classic 3Blue1Brown approach and works exceptionally well for math/technical topics.
#### 4d: Duration and Platform
| Platform | Duration Range | Word Budget (150 WPM) |
|----------|---------------|----------------------|
| TikTok | 30-60s | 65-150 words |
| YouTube Shorts | 30-60s | 65-150 words |
| YouTube | 60-300s | 150-750 words |
| LinkedIn | 60-120s | 150-300 words |
**Animation note:** Animation videos can be longer than live-action explainers because the visual density sustains attention. A 3-minute math animation holds attention better than a 3-minute talking head.
#### 4e: Concept Diversity Check
- [ ] No two concepts use the same animation mode
- [ ] No two concepts use the same narrative structure
- [ ] At least one concept is achievable with free/local tools only
- [ ] At least one concept leverages the most surprising data point
- [ ] Each concept's animation mode is grounded in technique research
### Step 5: Present Concepts and Get Selection
Present all concepts clearly to the user. For each concept, show:
1. **Title** and **hook** — the creative pitch
2. **Animation mode** — what the video will LOOK like (with a plain-language description)
3. **Why this works** — research backing, in one sentence
4. **Duration** — how long
5. **Reuse strategy** — "5 scenes built from 2 templates" vs "8 unique scenes"
Let the user select, combine, modify, or redirect.
Record the selection in `selected_concept` with rationale and any modifications.
### Step 6: Build the Production Plan
For the selected concept, design the stage-by-stage production plan.
**Animation-specific production plan fields:**
```
PRODUCTION PLAN (Animation Pipeline)
animation_mode: [selected mode]
reuse_strategy:
recurring_motifs: [list]
layout_system: [description]
transition_family: [type]
typography_hierarchy: [levels]
estimated_unique_scenes: [N]
estimated_reusable_templates: [N]
stages:
script:
tools: [none — creative work]
cost: $0
notes: "Script must be written in animation beats — one visual idea per section"
scene_plan:
tools: [none — planning work]
cost: $0
notes: "Scene plan must specify animation mode per scene and reuse template references"
assets:
tools: [specific providers from preflight]
cost: [itemized]
notes: "Reusable motifs generated once, referenced by multiple scenes"
edit:
tools: [none — planning work]
cost: $0
notes: "Edit must preserve hold times and staggered reveals"
compose:
tools: [video_compose, audio_mixer]
cost: $0 (local rendering)
notes: "Text and diagrams must remain sharp at final resolution"
publish:
tools: [none — metadata work]
cost: $0
```
### Step 7: Build the Cost Estimate
Itemize every paid operation:
```
COST ESTIMATE
├── TTS Narration: [provider] × 1 run $X.XX
├── Image Generation: [provider] × N scenes $X.XX
│ (N unique + M reused = total scenes)
├── AI Video Clips: [provider] × K clips (if any) $X.XX
├── Music: music_gen × 1 track $X.XX
├── Math Animation: math_animate (local/free) $0.00
├── Diagram Generation: diagram_gen (local/free) $0.00
└── TOTAL ESTIMATED $X.XX
Budget cap: $X.XX
Verdict: within_budget ✓ / over_budget ✗
Headroom: $X.XX for revisions
```
**Animation cost note:** Programmatic animation (Manim, Remotion, diagram_gen) is FREE. This means animation pipelines can often be much cheaper than explainer pipelines — the primary cost is TTS narration and any AI-generated images/video used as backgrounds or transitions.
### Step 8: Assemble the Approval Gate
```
────────────────────────────────────────
PROPOSAL READY FOR APPROVAL
Concept: [selected title]
Animation mode: [mode] — [plain description]
Duration: [X] seconds for [platform]
Reuse strategy: [N] unique scenes from [M] templates
Estimated cost: $[X.XX] of $[budget] budget
Production path: [premium/standard/budget/free]
Proceed? (approve / approve with changes / reject)
────────────────────────────────────────
```
**Critical rule:** The pipeline MUST NOT proceed past this stage without explicit approval.
### Step 9: Submit
Validate the `proposal_packet` artifact against `schemas/artifacts/proposal_packet.schema.json` and submit.
## How This Connects Downstream
| Downstream Stage | What It Takes From proposal_packet |
|------------------|------------------------------------|
| Script Director | `selected_concept` (title, hook, key_points, animation_mode, narrative_structure) + research data |
| Scene Director | `selected_concept.animation_mode` + `reuse_strategy` + `production_plan.playbook` |
| Asset Director | `production_plan.stages[assets].tools` — knows exactly which providers to use |
| Executive Producer | `cost_estimate` — initializes budget tracking |
| All stages | `approval.approved_budget_usd` — hard spending cap |
## Common Pitfalls
- **Ignoring animation mode feasibility**: If Manim isn't installed, don't propose a Manim-based concept. Design around constraints.
- **Three versions of the same concept with different titles**: Structural diversity means different animation modes, different narrative structures, different hooks.
- **Not leveraging free tools**: Animation has a huge cost advantage — Manim, Remotion, and diagram_gen are free. If proposing expensive AI video, justify why free alternatives won't work.
- **Over-promising visual complexity**: 20 unique hand-crafted scenes is not realistic. Design reuse strategies that look varied but share underlying templates.
- **Skipping the approval gate**: This is the whole point of pre-production. No shortcuts.
- **Ignoring mathematical accuracy**: If the research brief flagged technical accuracy constraints, the concept MUST respect them. A beautiful but wrong animation is a failure.
skills/pipelines/animation/publish-director.md
+45
View File
@@ -0,0 +1,45 @@
# Publish Director - Animation Pipeline
## When To Use
Package the animation so the metadata, thumbnail concept, and platform framing reflect the actual visual system of the project.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/publish_log.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["compose"]["render_report"]`, `state.artifacts["idea"]["brief"]`, `state.artifacts["script"]["script"]` | Final outputs and topic framing |
| Playbook | Active style playbook | Visual naming consistency |
## Process
### 1. Match Packaging To The Animation Mode
Examples:
- diagram-heavy videos should look structured and legible,
- kinetic-type pieces should package around strong copy,
- illustrative animation should package around hero imagery.
### 2. Preserve Visual-System Truth
Store in `publish_log.metadata`:
- `animation_mode`
- `hero_frame_notes`
- `thumbnail_concept`
- `platform_notes`
### 3. Quality Gate
- metadata fits the actual animation mode,
- thumbnail concept matches the final visual system,
- exports are labeled by purpose and platform,
- the package is usable without extra manual work.
## Common Pitfalls
- Writing generic metadata that ignores the animation style.
- Creating a thumbnail concept unrelated to the final frames.
- Mixing platform variants without clear labels.
skills/pipelines/animation/research-director.md
+256
View File
@@ -0,0 +1,256 @@
# Research Director — Animation Pipeline
## When to Use
You are the **Research Director** for a generated animation video. You are the first stage in the pipeline — before any creative decisions, before any script, before any money is spent. Your job is to **deeply research the topic AND the animation approach** using web search and produce a `research_brief` artifact that grounds the entire video in real data, real pedagogy, and proven visual techniques.
Animation videos differ from general explainers: the research must cover both **what to explain** (topic) and **how to animate it** (technique). A math-animation video about eigenvalues needs different visual research than a kinetic-typography brand video.
**You do NOT make creative decisions.** You gather raw material. The Proposal Director downstream will use your findings to craft concept options with animation-mode recommendations.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/research_brief.schema.json` | Artifact validation |
| User input | Topic, audience hint, animation hint | Research scope |
| Tools | Web search, web fetch | Research execution |
## Process
### Step 1: Scope the Research
Before searching anything, establish boundaries:
- **Topic**: What is the core subject? Extract from user input.
- **Audience hint**: Did the user mention who this is for? (developers, students, general public, professionals)
- **Animation hint**: Did the user mention an animation style? (math animation, motion graphics, kinetic typography, diagram-led, illustrative)
- **Platform hint**: Did the user mention where this will go? (YouTube, TikTok, LinkedIn, classroom)
- **Depth**: Is this a well-known topic or niche?
If the user's request is a single phrase like "make a math animation about eigenvalues," that's fine — you have enough to research. Do NOT ask clarifying questions at this stage.
### Step 2: Content Landscape Scan
**Goal:** Understand what already exists so we can find gaps.
```
SEARCH BATCH 1 — Landscape (run all in parallel)
Q1: "[topic] animation" site:youtube.com
→ Find: Existing animated explainers. Note animation styles used, view counts, quality.
Q2: "[topic]" (animation OR "motion graphics" OR "animated explainer") -site:youtube.com
→ Find: Articles, tutorials, and write-ups about animating this topic.
Q3: "[topic] [current month] [current year]"
→ Find: The freshest content. What's being published RIGHT NOW?
Q4: "[topic]" (manim OR "3blue1brown" OR "motion design" OR "animated diagram")
→ Find: Programmatic or technical animation approaches to this topic.
```
**Parse results for:**
- Which animation styles have been used for this topic (and which haven't)
- Quality benchmarks — what do the best animations of this topic look like?
- Gaps — which visual approaches haven't been tried?
- Whether programmatic animation (Manim) has been used for this topic before
Record at least 3 entries in `landscape.existing_content` with specific titles, sources, and gap analysis.
### Step 3: Trending Pulse
**Goal:** Find what's happening RIGHT NOW — news, debates, discoveries.
```
SEARCH BATCH 2 — Trending (run all in parallel)
Q5: "[topic]" (announcement OR discovery OR update OR breakthrough) after:[current year]-01-01
→ Find: Recent events that make this topic timely.
Q6: "[topic]" site:reddit.com after:[6 months ago]
→ Find: Active community discussions, pain points.
Q7: "[topic]" site:news.ycombinator.com
→ Find: Technical audience opinions and analysis.
Q8: "why is [topic]" (trending OR important OR everywhere) [current year]
→ Find: Meta-commentary on why people care right now.
```
If no trending signal exists, note `timeliness_window: "evergreen"` and move on.
### Step 4: Data and Evidence Gathering
**Goal:** Find specific, citable facts that will anchor the script AND drive visual moments.
```
SEARCH BATCH 3 — Data (run all in parallel)
Q9: "[topic]" statistics [current year]
→ Find: Hard numbers — adoption rates, performance benchmarks, measurements.
Q10: "[topic]" (study OR research OR survey) [current year - 1] OR [current year]
→ Find: Academic or industry research.
Q11: "[topic]" "surprisingly" OR "counterintuitively" OR "most people don't know"
→ Find: Surprising facts — these become visual hooks.
Q12: "[topic]" (comparison OR benchmark OR "vs") data
→ Find: Comparative data that becomes animated stat cards or side-by-side visuals.
```
**For each data point, record:**
- The specific claim (precise, not vague)
- Source URL and source name
- Credibility rating: `primary_source`, `secondary_source`, `anecdotal`
- Surprise factor: expected or counterintuitive?
- **Visual potential**: Can this be animated? (e.g., "73% → 23%" is a great shrinking bar chart moment; "it's important" is not animatable)
**Minimum: 3 data points. Target: 5-8.**
### Step 5: Audience Mining
**Goal:** Understand what real people ask, believe, and get wrong.
```
SEARCH BATCH 4 — Audience (run all in parallel)
Q13: "[topic]" site:reddit.com "help" OR "confused" OR "why does" OR "ELI5"
→ Find: Real questions from real people.
Q14: "[topic]" site:quora.com OR site:stackoverflow.com
→ Find: Structured Q&A — what do beginners ask?
Q15: "[topic]" "common mistakes" OR "myths" OR "misconceptions"
→ Find: What people get wrong — animation can powerfully show myth vs reality.
Q16: "[topic]" "wish I knew" OR "before you start" OR "nobody tells you"
→ Find: Insider knowledge.
```
**Parse results for:**
- Top 5+ real questions
- Common misconceptions (great for "wrong way → right way" animation transitions)
- Knowledge level of the target audience
### Step 6: Animation Technique Research (ANIMATION-SPECIFIC)
**Goal:** Research how to best ANIMATE this topic — what visual techniques work.
This step is what makes the animation research-director different from the explainer version.
```
SEARCH BATCH 5 — Animation Techniques (run all in parallel)
Q17: "[topic]" (visualization OR "visual explanation" OR infographic OR diagram)
→ Find: How others have visualized this concept.
Q18: "[topic category]" animation technique (motion graphics OR manim OR "after effects")
→ Find: Specific animation techniques used for this kind of content.
Q19: "[topic]" "step by step" OR "how it works" visual
→ Find: Sequential visual breakdowns — inform scene progression.
Q20: "animate [topic-related-process]" OR "[topic] animation tutorial"
→ Find: Technical approaches to animating this concept.
```
**For each technique found, record:**
- What the technique is (e.g., "progressive diagram build", "morph between states", "particle simulation")
- Where it was used (source URL)
- Which animation mode it maps to: `manim`, `remotion`, `motion_graphics`, `ai_video`, `illustrative`
- Complexity: `simple` (reusable components), `moderate` (custom but repeatable), `complex` (bespoke per scene)
- Whether it's been done before for this topic (novelty signal)
**Minimum: 2 technique references. Target: 4-6.**
### Step 7: Mathematical/Technical Accuracy Check (If Applicable)
**For math-animation, science, or technical topics:**
```
Q21: "[topic]" (formal definition OR mathematical OR "technically")
→ Find: The precise technical definition — animation must not oversimplify to the point of being wrong.
Q22: "[topic]" "common error" OR "often confused with" OR "technically incorrect"
→ Find: Technical pitfalls that the animation must avoid.
```
**Record:**
- The precise definition or formula
- Common simplification errors
- What level of simplification is acceptable for the target audience
- Any visual metaphors that are technically misleading (e.g., "electrons orbiting like planets" is wrong)
If the topic is not math/science, skip this step.
### Step 8: Angle Synthesis
Using everything from Steps 2-7, identify at least 3 genuinely different angle candidates.
For each angle, specify:
| Field | What | Quality Bar |
|-------|------|-------------|
| `name` | Short title (5-8 words) | Specific, not generic |
| `hook` | One-sentence grabber | Must create an information gap or surprise |
| `type` | `trending`, `evergreen`, `contrarian`, `narrative`, `data_driven` | Categorize honestly |
| `why_now` | Why this angle is compelling right now | Must cite specific research findings |
| `grounded_in` | Which data points or audience insights support it | Cross-reference your findings |
| `animation_fit` | Which animation mode(s) best serve this angle | Must reference technique research from Step 6 |
**Angle diversity checklist:**
- [ ] At least one angle leverages a surprising data point or visual
- [ ] At least one angle is evergreen
- [ ] At least one angle maps to a different animation mode than the others
- [ ] No two angles use the same hook structure
- [ ] Each angle's `animation_fit` references specific technique research
### Step 9: Source Bibliography
Compile all URLs used, organized by section. Minimum 5 sources.
**Source quality rules:**
- Primary sources > secondary > anecdotal
- At least 2 primary sources
- Every data_point must have a source_url
- Flag sources older than 2 years
### Step 10: Assemble and Submit
Build the `research_brief` artifact per the schema. Include:
1. `research_summary` — one paragraph: the most important insight AND the most promising animation approach.
2. All sections from Steps 2-9
Validate against `schemas/artifacts/research_brief.schema.json` before submitting.
## Quality Bar
| Criterion | Minimum | Target |
|-----------|---------|--------|
| Existing content surveyed | 3 pieces | 5-8 pieces |
| Data points with sources | 3 | 5-8 |
| Audience questions sourced | 3 | 5-10 |
| Animation techniques researched | 2 | 4-6 |
| Angle candidates | 3 | 4-5 |
| Total sources cited | 5 | 10-15 |
| Searches executed | 12 | 18-22 |
## Execution Constraints
| Constraint | Value | Why |
|------------|-------|-----|
| Max time on research | 3-5 minutes | Diminishing returns |
| Max searches | 25 | Prevent rabbit holes |
| Min searches | 12 | Ensure coverage |
| No paid tools | — | Research uses web search only — zero cost |
## Common Pitfalls
- **Skipping animation technique research**: The explainer research-director doesn't need this, but animation does. The `animation_fit` field in angles is mandatory.
- **Ignoring mathematical accuracy**: For math topics, the research MUST include the precise definition. An animation that looks cool but teaches wrong math is worse than no animation.
- **Only searching topic, not visualization**: If the topic is "Fourier transforms," you must search both "Fourier transforms" AND "Fourier transform visualization/animation." The technique research is half the value.
- **Treating all animation as one category**: Manim, Remotion, AI video, and motion graphics are fundamentally different tools with different strengths. Research should inform which mode fits the topic.
- **Recording vague visual references**: "A nice animation" is not useful. "Progressive circle-to-wave morph showing sine decomposition (3Blue1Brown style, Manim)" is useful.
skills/pipelines/animation/scene-director.md
+65
View File
@@ -0,0 +1,65 @@
# Scene Director - Animation Pipeline
## When To Use
You are converting the script into a feasible animation plan. This is the stage that decides whether the project feels designed or chaotic.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/scene_plan.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Beat map and tool path |
| Playbook | Active style playbook | Palette, typography, motion consistency |
## Process
### 1. Make An Animatic-Minded Plan
For each scene, define:
- what appears first,
- what changes,
- what is held,
- how the scene exits.
### 2. Limit Transition Families
Choose a small set of transition meanings:
- cut,
- fade,
- slide,
- transform.
### 3. Match Scene Type To Tool Path
Use:
- `diagram` scenes for structured explanation,
- `animation` scenes for motion-first sequences,
- `text_card` for clean high-impact copy moments,
- `generated` only where needed.
### 4. Use Metadata For Timing Rules
Recommended metadata keys:
- `animatic_rules`
- `transition_rules`
- `hold_rules`
- `tool_path_map`
- `reusable_motifs`
### 5. Quality Gate
- every scene has a clear timing intent,
- the transition system is limited and meaningful,
- the tool path is explicit,
- the sequence feels like one designed system.
## Common Pitfalls
- Adding a new transition idea in every scene.
- Planning scenes that have no realistic production path.
- Overanimating text-heavy scenes.
skills/pipelines/animation/script-director.md
+117
View File
@@ -0,0 +1,117 @@
# Script Director — Animation Pipeline
## When to Use
This stage turns the approved proposal into animation-ready beats. The script must leave room for motion, staging, and hold time — and must integrate the research findings and respect the selected animation mode.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/script.schema.json` | Artifact validation |
| Prior artifact | `proposal_packet` from Proposal Director | Selected concept, animation mode, target duration, reuse strategy |
| Optional artifact | `research_brief` from Research Director | Data points, audience insights, accuracy constraints |
| Tools | `transcriber` | Optional source transcript support |
## Process
### 1. Absorb the Proposal
Read the `proposal_packet.selected_concept` thoroughly. Extract:
- **Title and hook** — the opening must deliver on this promise
- **Animation mode** — `manim`, `remotion`, `ai_video`, `diagram_stills`, or `mixed`. This constrains how you write.
- **Narrative structure** — `progressive_build`, `myth_busting`, `journey`, etc. Follow it.
- **Target duration** — word budget = target_seconds × 2.5 words/sec (at 150 WPM)
- **Key points** — from `selected_concept.key_points`
- **Reuse strategy** — recurring motifs mean recurring script structures
If `research_brief` is available, also extract:
- **Data points** — weave specific, sourced facts into the narration (not vague claims)
- **Audience misconceptions** — address them directly in the script
- **Mathematical accuracy notes** — constraints on what can and cannot be simplified
### 2. Write in Animation Beats
Each section should express ONE clear visual idea:
- **Statement** — introduce a concept (entrance animation)
- **Demonstration** — show it working (the main animation)
- **Transformation** — morph from one state to another (transition)
- **Comparison** — show two things side by side (split screen or sequential)
- **Conclusion** — land the insight (hold + emphasis)
**Animation mode affects writing style:**
| Mode | Writing Style |
|------|---------------|
| Manim | Precise, mathematical. Each beat maps to a specific geometric transformation. Write narration that describes what's being shown. |
| Remotion | Data-driven, punchy. Each beat maps to a chart/component animation. Narration complements the visual data. |
| AI Video | Descriptive, evocative. Each beat describes a scene the AI should generate. Narration adds context the visual can't convey. |
| Diagram Stills | Explanatory, progressive. Each beat adds a layer to a building diagram. Narration walks through the build. |
| Mixed | Varies per section — tag each section's mode in metadata. |
### 3. Keep On-Screen Text Tight
Animation-heavy pieces fail when the viewer has to read too much while motion is happening:
- **Max 8 words** for on-screen titles
- **Max 15 words** for on-screen descriptions
- Prefer phrases over sentences
- Prefer numbers and labels over paragraphs
- Mathematical notation is fine — it IS the content in math-animation mode
### 4. Leave Room for Visual Holds
Do NOT fill every second with new information. The scene plan will need time for:
- **Entrances** (0.5-1s): objects appearing on screen
- **Reveals** (1-2s): progressive disclosure of complexity
- **Holds** (1-3s): letting the viewer absorb what they see
- **Exits** (0.5s): clearing the stage for the next beat
**Rule of thumb:** For every 10 seconds of narration, budget 3-4 seconds of visual breathing room. A 90-second video should have ~60-65 seconds of narration and ~25-30 seconds of visual holds.
### 5. Use Metadata for Motion Intent
Recommended metadata keys per section:
- `beat_type`: statement / demonstration / transformation / comparison / conclusion
- `animation_mode`: which mode this section uses (important for mixed mode)
- `text_constraints`: max words for on-screen text in this section
- `narration_plan`: how narration relates to visual (describes / complements / silent)
- `visual_priority`: what the viewer should focus on (the animation, the text, the data)
- `hold_time_seconds`: minimum visual hold time after this section's content
- `data_source`: if this section uses a research data point, reference it
### 6. Research Integration
If a `research_brief` is available:
- Use at least 2 data points from the research in the narration
- Ground the hook in the research's most surprising finding
- Address at least 1 audience misconception if the narrative structure supports it
- Cite sources naturally ("According to [source]..." or "A [year] study found...")
- Do NOT invent statistics — only use what the research found
### 7. Quality Gate
Before submitting the script, verify:
- [ ] Every section supports ONE strong visual idea
- [ ] On-screen text is concise (phrases, not paragraphs)
- [ ] Timing is animation-friendly (holds budgeted)
- [ ] Word count is within ±10% of target duration
- [ ] Animation mode is respected in writing style
- [ ] Research data points are integrated (if research_brief available)
- [ ] Mathematical accuracy is maintained (if applicable)
- [ ] Later stages can map scenes cleanly from this script
## Common Pitfalls
- **Writing too many ideas into one section.** One beat = one visual idea.
- **Treating captions and on-screen text as the same thing.** Subtitles are narration transcribed. On-screen text is designed content that's part of the animation.
- **Forgetting that motion needs pause and emphasis.** Budget hold times.
- **Ignoring the animation mode.** A Manim script reads differently than an AI video script.
- **Writing research-less scripts when a research_brief exists.** If the research found surprising data, use it. Generic scripts waste the research investment.
- **Oversimplifying math to the point of being wrong.** Check the research brief's accuracy notes.
skills/pipelines/avatar-spokesperson/asset-director.md
+101
View File
@@ -0,0 +1,101 @@
# Asset Director - Avatar Spokesperson Pipeline
## When To Use
This stage prepares the actual spokesperson ingredients: narration, avatar or lip-sync footage, subtitle assets, branded backgrounds, and the minimal support graphics needed to complete the cut.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Presenter plan and narration needs |
| Tools | `talking_head`, `lip_sync`, `tts_selector`, `subtitle_gen`, `image_selector`, `audio_enhance` — selectors auto-discover all available providers from the registry | Avatar, narration, and support asset options |
| Playbook | Active style playbook | Background, type, and subtitle rules |
## Process
### 1. Lock The Avatar Generation Path
Use one primary path and record it clearly:
- `talking_head` from still image plus audio,
- `lip_sync` from existing presenter plate plus new audio,
- externally supplied avatar render if created outside the current runtime.
Do not hide a blocked avatar path. Record it.
### 1b. Sample Preview (Prevents Wasted Spend)
Before batch-generating assets, produce one sample of each expensive type and show the user:
1. **TTS sample** (if generating narration): Generate one section. Confirm voice, pace, and persona before batching the rest.
2. **Avatar sample** (if using `talking_head`): Generate a short test clip. Confirm the avatar quality is acceptable before committing to full generation.
If rejected, adjust parameters and retry (max 3 iterations). Do not batch until approved.
### 2. Resolve Narration Before Support Graphics
Spokesperson videos depend on speech. Determine whether narration is:
- supplied,
- TTS-generated,
- already embedded in a presenter plate.
If narration is missing and no TTS tool is available, mark the project blocked instead of pretending the stage succeeded.
### 3. Build The Minimal Support Kit
Prepare only what the scene plan actually needs:
- subtitle files,
- one lower-third system,
- CTA card,
- background or plate assets,
- optional still or product support images.
### 4. Use Metadata For Capability Truth
Recommended metadata keys:
- `avatar_generation_path`
- `narration_assets`
- `subtitle_assets`
- `background_assets`
- `scene_asset_index`
- `blocked_assets`
### 5. Quality Gate
- the avatar path is explicit,
- narration and avatar assets align,
- support graphics stay minimal,
- every referenced file exists.
## No-Avatar Path
When the EP has triggered a narration-over-graphics pivot (neither `talking_head` nor `lip_sync` available), skip avatar generation entirely and produce a graphics-driven asset kit instead:
### What to produce:
1. **Narration audio** — via `tts_selector` (mandatory; block the project if no TTS is available either).
2. **Scene visuals** — via `image_selector` or `video_selector`. One primary visual per scene that reinforces the spoken point (diagram, illustration, product shot, or stock footage).
3. **Subtitle files** — same as standard path.
4. **Text cards** — key-point overlays, stat cards, CTA end card.
5. **Backgrounds** — consistent family matching the playbook.
### What to skip:
- No `talking_head` or `lip_sync` calls.
- No presenter framing metadata.
- `avatar_generation_path` should be set to `"none — narration-over-graphics pivot"`.
### Metadata for this path:
- `avatar_generation_path`: `"narration_over_graphics"`
- `pivot_reason`: why the no-avatar path was chosen
- All other metadata keys remain the same.
## Common Pitfalls
- Building decorative assets before the narration path is solved.
- Mixing multiple avatar-generation strategies in one simple spokesperson video.
- Marking the stage complete when the core presenter asset is still hypothetical.
- (No-avatar path) Generating filler visuals with no connection to the narration — every image must reinforce the spoken point.
skills/pipelines/avatar-spokesperson/compose-director.md
+60
View File
@@ -0,0 +1,60 @@
# Compose Director - Avatar Spokesperson Pipeline
## When To Use
Render the final spokesperson outputs. The bar is simple: the presenter must look stable, speech must be clear, and subtitles or support cards must not crowd the frame.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | What to render |
| Tools | `video_compose`, `audio_mixer`, `video_stitch`, `audio_enhance` | Render and audio finishing |
| Playbook | Active style playbook | Typography and layout rules |
## Process
### 1. Render The Hero Cut First
Prefer one strong master before derivatives. Compose:
- presenter video,
- subtitles,
- lower-thirds,
- CTA cards,
- mixed narration.
### 2. Keep The Frame Clean
Subtitle and CTA placement matter more here than flashy transitions. Leave the face and mouth region unobstructed.
### 3. Verify Mouth Timing And Audio
If the avatar path used lip sync or audio-driven talking head, check:
- mouth timing,
- face artifacts,
- drift on long sections,
- audio clarity.
### 4. Verify Every Output
Record important findings in:
- `render_report.verification_notes`
- `render_report.warnings`
- `render_report.metadata.variant_notes`
### 5. Quality Gate
- the output file is valid,
- speech is clear,
- subtitles stay readable,
- the presenter remains visually stable.
## Common Pitfalls
- Letting subtitles cover the chin or mouth area.
- Shipping a long lip-sync render without spot-checking drift.
- Making derivative crops that cut off the presenter or CTA.
skills/pipelines/avatar-spokesperson/edit-director.md
+47
View File
@@ -0,0 +1,47 @@
# Edit Director - Avatar Spokesperson Pipeline
## When To Use
Turn the planned presenter scenes and produced assets into a coherent spokesperson timeline. The quality bar is steady delivery, readable support layers, and a clear CTA landing.
## Process
### 1. Cut The Presenter Track First
Assemble the core spokesperson performance before adding support layers. If the presenter cut is weak, extra graphics will not rescue it.
### 2. Add Support Layers Sparingly
Use overlays only where they help:
- short proof points,
- product names,
- pricing or feature cards,
- CTA reinforcement,
- subtitles.
### 3. Respect Spoken Rhythm
Keep pauses where they help emphasis. Do not trim so tightly that the avatar feels rushed or robotic.
### 4. Plan Deliverables Clearly
Recommended metadata keys:
- `hero_cut_order`
- `cta_frame_range`
- `overlay_timing_map`
- `variant_decisions`
### 5. Quality Gate
- the presenter remains the anchor,
- overlays are timed cleanly,
- scene transitions are calm and intentional,
- the CTA lands once and clearly.
## Common Pitfalls
- Overcutting to simulate energy.
- Letting captions, side panels, and lower thirds compete for the same area.
- Ending without a clean CTA hold.
skills/pipelines/avatar-spokesperson/executive-producer.md
+171
View File
@@ -0,0 +1,171 @@
# Executive Producer — Avatar Spokesperson Pipeline
## When to Use
You are the **Executive Producer (EP)** for an avatar spokesperson video. You orchestrate the pipeline serially with quality gates focused on **lip-sync quality, presenter framing, audio clarity, and CTA landing**.
**No pre-production stages.** The project is script-driven with a digital presenter as the anchor. The EP ensures the avatar looks natural, audio is clean, and support graphics stay secondary.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/avatar-spokesperson.yaml` | Stage definitions |
| Skills | All 7 director skills + `meta/reviewer` | Stage execution |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
## Cumulative State
```
EP_STATE:
pipeline: avatar-spokesperson
playbook: <selected>
target_duration_seconds: <from brief>
budget_total_usd: <configured>
budget_spent_usd: 0.0
# Avatar-specific
avatar_path: null # heygen_api / sadtalker / musetalk / stock
narration_source: null # tts / provided_audio
cta_type: null # what the viewer should do after watching
presenter_framing: null # layout: center, left-third, etc.
artifacts:
idea: null
script: null
scene_plan: null
assets: null
edit: null
compose: null
publish: null
revision_counts: {}
issues_log: []
```
## Pivot Decision Matrix
`talking_head` is the preferred tool but commonly unavailable (requires GPU or HeyGen API key). When blocked, the EP must route the project explicitly — not improvise.
```
IF talking_head AVAILABLE:
→ Standard avatar path. Proceed as normal.
IF talking_head UNAVAILABLE and lip_sync AVAILABLE:
→ Lip-sync path. User must supply a presenter plate (existing footage).
Script and scene plan stay the same.
IF NEITHER talking_head NOR lip_sync AVAILABLE:
→ Narration-Over-Graphics pivot.
Tell the user: "No avatar tool is configured. I can produce a
narration-over-graphics video instead — your script and CTA stay the same,
but the presenter is replaced with styled visuals, text overlays, and
voice-over narration."
If the user approves:
- Switch scene-director to narration-over-graphics layout (see its fallback section).
- Switch asset-director to no-avatar path (see its fallback section).
- CTA and script are unchanged.
If the user declines:
- Mark the project blocked. Do not proceed with a half-working avatar.
```
**The pivot decision happens at G1 (after IDEA).** Do not wait until the ASSETS stage to discover the tool is missing.
## EP-Specific Cross-Stage Checks
### After IDEA stage:
```
CHECK: Avatar path feasibility
- Is the avatar generation path explicit (which tool)?
- Is the required tool available in the registry?
- If tool unavailable: run the Pivot Decision Matrix above
- Are CTA and audience appropriate for spokesperson format?
```
### After SCRIPT stage:
```
CHECK: Spoken copy quality
- Is the script concise and natural-sounding when read aloud?
- Are scene breaks realistic for avatar delivery (no mid-sentence cuts)?
- Is on-screen text restrained (presenter is the focus, not graphics)?
CHECK: Duration fit
- Word count aligns with natural speaking pace (~140-160 WPM for spokesperson)
```
### After SCENE_PLAN stage:
```
CHECK: Presenter layout
- Is the speaker layout consistent and coherent?
- Are support overlays secondary to the presenter?
- Are background changes minimal (max 2-3 distinct backgrounds)?
CHECK: Subtitle safety
- Is subtitle placement planned to avoid overlapping the presenter's face?
```
### After ASSETS stage:
```
CHECK: Avatar generation
- Did the avatar tool produce a usable video?
- Is lip-sync timing acceptable?
- Is narration audio clear and natural?
- Budget gate: 90% threshold warning
CHECK: Support asset restraint
- Are support graphics (backgrounds, overlays) minimal?
- Do they match the playbook style?
```
### After EDIT stage:
```
CHECK: Presenter primacy
- Is the presenter visually primary in every scene?
- Are graphics and captions reinforcing, not crowding?
- Does CTA land clearly (dedicated end section)?
CHECK: Timeline completeness
- All cuts reference valid assets
- Audio ducking if background music present
```
### After COMPOSE stage:
```
CHECK: Output validation
- ffprobe: duration, resolution, codec
- Lip-sync or mouth timing acceptable for the chosen path
- Subtitle placement clean and non-overlapping
- Audio clear and presenter-focused
- No uncanny-valley artifacts that break immersion
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | idea | Avatar path feasibility, CTA fit | Revise |
| G2 | script | Spoken copy quality, duration | Revise |
| G3 | scene_plan | Presenter layout, subtitle safety | Revise |
| G4 | assets | Avatar quality, lip-sync, budget | Revise |
| G5 | edit | Presenter primacy, CTA landing | Revise |
| G6 | compose | Lip-sync, subtitle placement, audio | Revise or send-back |
| G7 | publish | Metadata, presenter thumbnail | Revise |
| FINAL | all | Avatar naturalness, audio, CTA | Send-back |
## Execution Limits
| Limit | Value |
|-------|-------|
| Max revisions per stage | 3 |
| Max send-backs per stage pair | 1 |
| Max total send-backs | 3 |
| Max total budget | Configurable (default $2) |
| Max total wall-time | 12 minutes |
## Common Pitfalls
- **Uncanny valley**: If avatar quality is low, it undermines the entire video. Be honest about tool capabilities.
- **Graphics overload**: The presenter IS the content. Support graphics should be minimal.
- **Unnatural script**: Spokesperson scripts must sound conversational, not robotic or essay-like.
- **Ignoring CTA**: Every spokesperson video has a purpose. The CTA must land clearly.
skills/pipelines/avatar-spokesperson/idea-director.md
+73
View File
@@ -0,0 +1,73 @@
# Idea Director - Avatar Spokesperson Pipeline
## When To Use
Use this pipeline when the deliverable is a presenter-led avatar video: a spokesperson spot, product intro, onboarding message, internal comms update, or short scripted explainer where the speaker remains the visual anchor.
Your first job is to classify the avatar path honestly before anyone writes polished copy for an impossible production setup.
## Reference Inputs
- `docs/avatar-spokesperson-best-practices.md`
- `skills/creative/storytelling.md`
- `skills/creative/short-form.md`
## Process
### 1. Classify The Avatar Path
Record which production mode the project actually has:
- `platform_avatar`
- `photo_talking_head`
- `presenter_plate_lip_sync`
Also record whether the avatar already exists or still has to be created outside the current run.
### 2. Define The Message Shape
Capture:
- audience,
- core offer or CTA,
- runtime target,
- platform targets,
- whether the video is sales, onboarding, support, or announcement led.
Spokesperson videos work best when they have one clear job.
### 3. Capture Source Reality
The brief should explicitly state:
- whether clean narration is supplied,
- whether TTS is acceptable,
- whether brand backgrounds or overlays exist,
- whether subtitles are required,
- whether multilingual variants are expected.
### 4. Build The Brief
Recommended metadata keys:
- `avatar_path`
- `avatar_exists`
- `narration_source`
- `target_audience`
- `cta_type`
- `background_strategy`
- `deliverable_mix`
- `missing_capabilities`
### 5. Quality Gate
- the avatar path is explicit,
- the message is narrow enough for a spokesperson format,
- missing narration or avatar dependencies are visible early,
- deliverables fit the actual source setup.
## Common Pitfalls
- Treating a generic generated-video request as a deterministic avatar workflow.
- Writing the CTA before confirming the avatar and narration path.
- Planning multiple aspect ratios before the hero layout is proven.
skills/pipelines/avatar-spokesperson/publish-director.md
+44
View File
@@ -0,0 +1,44 @@
# Publish Director - Avatar Spokesperson Pipeline
## When To Use
Package the finished spokesperson outputs for delivery. This stage should make it obvious which file is the hero cut, which are derivatives, and what message or audience each version serves.
## Process
### 1. Label Deliverables Clearly
Distinguish:
- hero cut,
- vertical cutdown,
- square cutdown,
- language variants,
- watermark or review versions.
### 2. Keep Metadata Message-Led
Recommended metadata keys:
- `audience_segment`
- `cta_copy`
- `offer_name`
- `locale`
- `thumbnail_concept`
### 3. Package Review Notes
If the avatar path has limitations such as visible lip-sync risk, retain that note in the package instead of hiding it.
### 4. Quality Gate
- exports are clearly named,
- metadata matches the intended message,
- poster frame or thumbnail concept features the presenter cleanly,
- review notes stay attached to the package.
## Common Pitfalls
- Mixing hero and derivative exports without clear naming.
- Reusing generic metadata that ignores the spokesperson offer.
- Dropping risk notes that matter for downstream publishing teams.
skills/pipelines/avatar-spokesperson/scene-director.md
+82
View File
@@ -0,0 +1,82 @@
# Scene Director - Avatar Spokesperson Pipeline
## When To Use
Plan the visual system for the presenter-led cut. This stage decides how the speaker sits in frame, what support layers appear, and which aspect ratios are actually worth building.
## Reference Inputs
- `docs/avatar-spokesperson-best-practices.md`
- `skills/creative/typography.md`
## Process
### 1. Lock The Presenter Layout
Choose a primary layout:
- full presenter center frame,
- presenter plus side panel,
- presenter plus lower-third system,
- presenter over branded background.
Keep the hero layout simple and reusable.
### 2. Choose Background Discipline
Use one family:
- clean solid or gradient background,
- branded office or set,
- product UI or image plate,
- transparent-avatar composite plan.
Background changes should mark real section changes, not compensate for weak copy.
### 3. Map Support Layers
For each scene, decide whether the support layer is:
- none,
- subtitle only,
- lower third,
- product image,
- side-panel proof point,
- CTA card.
### 4. Plan Variants Late
Default to one hero cut first. Only plan vertical or square variants if the presenter layout and text system can survive the crop.
### 5. Quality Gate
- the presenter remains primary,
- support layers solve specific communication needs,
- the background system is consistent,
- variant ambition stays realistic.
## Narration-Over-Graphics Fallback
When the EP triggers a no-avatar pivot (no `talking_head` or `lip_sync` available), redesign the visual system to carry the story with graphics and voice-over instead of a presenter:
### Layout changes:
- **Primary frame**: full-screen visual (background, image, diagram, or motion graphic) with voice-over narration underneath.
- **Support layers**: key-point text cards, lower-third stats, illustrated diagrams, product screenshots.
- **CTA scene**: dedicated end card with clear CTA text, no presenter needed.
### What stays the same:
- The script and scene breaks are unchanged — the narration drives pacing.
- Subtitle planning is unchanged.
- Background discipline still applies (consistent family, not random).
### What changes:
- No presenter framing decisions (center, left-third, etc.) — replaced with full-frame visual compositions.
- Each scene needs a **primary visual** that reinforces the spoken content, since there's no face to anchor attention.
- Plan more visual variety per scene — without a presenter, static backgrounds feel empty.
## Common Pitfalls
- Switching backgrounds every few seconds.
- Filling empty space with decorative panels.
- Assuming a landscape presenter layout will survive a vertical crop untouched.
- (Fallback mode) Producing a wall of text on screen to compensate for no presenter — let the narration carry the content.
skills/pipelines/avatar-spokesperson/script-director.md
+65
View File
@@ -0,0 +1,65 @@
# Script Director - Avatar Spokesperson Pipeline
## When To Use
Turn the approved brief into scene-safe spoken copy for an avatar presenter. The quality bar is not literary flourish. It is spoken clarity, believable pacing, and one clean point per scene.
## Reference Inputs
- `docs/avatar-spokesperson-best-practices.md`
- `skills/creative/storytelling.md`
## Process
### 1. Write For Speech, Not For Slides
Prefer:
- short sentences,
- direct verbs,
- one idea per beat,
- explicit transitions,
- conversational emphasis.
If the copy sounds like a brochure when read aloud, rewrite it.
### 2. Break Into Scene-Safe Chunks
Avatar scenes are easier to manage when each section is compact. A useful starting point is:
- hook,
- value statement,
- proof or feature beat,
- CTA.
### 3. Keep On-Screen Text Light
The presenter is already carrying attention. Use on-screen text only for:
- product names,
- short proof points,
- CTA copy,
- legal or compliance text that must appear.
### 4. Use Metadata For Delivery Notes
Recommended metadata keys:
- `scene_copy_map`
- `cta_language`
- `pronunciation_notes`
- `supplied_script_source`
- `legal_text_requirements`
### 5. Quality Gate
- the copy sounds spoken,
- scene lengths are realistic,
- CTA placement is clear,
- text overlays are restrained.
## Common Pitfalls
- Overstuffing one scene because the script reads well on paper.
- Duplicating the same sentence in speech and large text overlays.
- Writing humor or improvisational beats the avatar path cannot sell.
skills/pipelines/cinematic/asset-director.md
+78
View File
@@ -0,0 +1,78 @@
# Asset Director - Cinematic Pipeline
## When To Use
This stage prepares the usable media for the final cinematic edit: source selects, title-card assets, optional support inserts, music, ambience, and subtitle assets when needed.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Scene intent and beat plan |
| Tools | `subtitle_gen`, `audio_enhance`, `image_selector`, `video_selector`, `music_gen` — selectors auto-discover all available providers from the registry | Optional support asset creation |
| Playbook | Active style playbook | Brand and typography consistency |
## Process
### 1. Prioritize Source Selects
Start with:
- source footage selects,
- stills,
- title-card backgrounds,
- any approved provided music or ambient beds.
These are the primary materials. Everything else is support.
### 1b. Sample Preview (Prevents Wasted Spend)
Before batch-generating support assets, produce one sample of each expensive generated type and show the user:
1. **Generated insert sample** (if using `image_selector` or `video_selector`): Generate one representative visual. Confirm it complements the source footage before batching.
2. **Music sample** (if using `music_gen`): Generate a short clip. Confirm mood and energy match the beat plan.
If rejected, adjust parameters and retry (max 3 iterations). Do not batch until approved.
### 2. Generate Support Assets Only Where Needed
Optional generated assets should fill clear gaps:
- missing transitional b-roll,
- concept-led inserts,
- texture or atmosphere cards,
- simple textural motion backgrounds.
### 3. Prepare A Real Audio Plan
Store:
- chosen music track or prompt,
- ambience layers,
- impact or transition sounds,
- subtitle assets if dialogue or narration is present.
### 4. Use Metadata For Rights And Intent
Recommended metadata keys:
- `source_selects`
- `music_plan`
- `ambience_plan`
- `title_assets`
- `generated_support_assets`
- `rights_notes`
### 5. Quality Gate
- source and support assets are clearly distinguished,
- generated inserts are limited and purposeful,
- audio plan matches the beat map,
- every referenced file exists.
## Common Pitfalls
- Generating extra shots before proving the source edit works.
- Treating music as a single loop instead of a beat-aware element.
- Forgetting rights or provenance notes for supplied assets.
skills/pipelines/cinematic/compose-director.md
+53
View File
@@ -0,0 +1,53 @@
# Compose Director - Cinematic Pipeline
## When To Use
Render the cinematic piece with careful attention to grade, audio dynamics, and frame treatment. This is not a generic export step.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | Edit plan and media assets |
| Tools | `video_compose`, `audio_mixer`, `video_stitch`, `video_trimmer`, `color_grade`, `audio_enhance` | Render and finishing |
| Playbook | Active style playbook | Finish consistency |
## Process
### 1. Use Frame Treatment Deliberately
Only use letterbox, 24fps intent, or heavy grading if they help the piece. Do not apply them because the pipeline name says cinematic.
### 2. Preserve Audio Dynamics
The mix should allow:
- quiet moments,
- impact moments,
- clear dialogue or narration,
- controlled music swells.
### 3. Verify The Final Mood
Check:
- opening frame,
- reveal beat,
- final landing,
- subtitle readability where relevant.
### 4. Use Render Metadata
Recommended metadata keys:
- `frame_treatment`
- `grade_profile`
- `mix_notes`
- `variant_outputs`
## Common Pitfalls
- Flattening the audio so the piece loses dynamics.
- Applying letterbox to footage that needs every pixel.
- Letting grading or sharpening damage faces or text.
skills/pipelines/cinematic/edit-director.md
+56
View File
@@ -0,0 +1,56 @@
# Edit Director - Cinematic Pipeline
## When To Use
This stage turns the beat map into a paced cinematic timeline. Rhythm and restraint matter more than effect count.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/edit_decisions.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["assets"]["asset_manifest"]`, `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]` | Assets, hero frames, beat map |
| Playbook | Active style playbook | Typography and transition consistency |
## Process
### 1. Cut By Emotion First
Cuts should follow:
- emotional emphasis,
- reveal timing,
- musical turns,
- visual contrast.
Do not optimize only for information density.
### 2. Protect Strong Moments
If a look, line, or gesture is doing the work, let it live. Do not over-cover it with extra inserts.
### 3. Use Sound To Push The Edit
Ambience, impacts, dropouts, and music changes should help create momentum between scenes.
### 4. Use Metadata For Timing Logic
Recommended metadata keys:
- `beat_timing`
- `audio_turns`
- `title_card_windows`
- `reframe_notes`
### 5. Quality Gate
- the emotional arc is intact,
- reveals land clearly,
- title cards are sparse and timed with intent,
- strong moments are not buried under coverage.
## Common Pitfalls
- Overcutting emotional material.
- Using speed ramps or flashy transitions by default.
- Letting title cards replace editorial clarity.
skills/pipelines/cinematic/executive-producer.md
+147
View File
@@ -0,0 +1,147 @@
# Executive Producer — Cinematic Pipeline
## When to Use
You are the **Executive Producer (EP)** for a cinematic video (trailers, brand films, montages, short dramatic edits). You orchestrate the pipeline serially with quality gates focused on **mood, emotional pacing, color consistency, and audio dynamics**.
**No pre-production stages.** Source footage or direction exists. The EP adds cross-stage gates that enforce emotional arc integrity and cinematic polish.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/cinematic.yaml` | Stage definitions |
| Skills | All 7 director skills + `meta/reviewer` | Stage execution |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
## Cumulative State
```
EP_STATE:
pipeline: cinematic
playbook: <selected>
target_duration_seconds: <from brief>
budget_total_usd: <configured>
budget_spent_usd: 0.0
# Cinematic-specific
emotional_arc: null # from brief: build → reveal → landing
color_grade_target: null # mood-driven color palette
hero_moments: [] # key reveal/climax frames
music_beat_map: null # audio-driven pacing reference
artifacts:
idea: null
script: null
scene_plan: null
assets: null
edit: null
compose: null
publish: null
revision_counts: {}
issues_log: []
```
## Execution Protocol
Same as standard EP: Initialize → Execute stages serially (idea → script → scene_plan → assets → edit → compose → publish) → Final QA.
Each stage: PREPARE → SPAWN DIRECTOR → REVIEW → GATE DECISION (pass / revise / send-back).
## EP-Specific Cross-Stage Checks
### After IDEA stage:
```
CHECK: Emotional arc definition
- Is the emotional arc explicit (build → reveal → landing)?
- Is source mode clear (supplied footage vs generated inserts)?
- Is the target mood defined and achievable?
```
### After SCRIPT stage:
```
CHECK: Beat escalation
- Does the beat map escalate cleanly toward the reveal?
- Are dialogue/title cards sparse and purposeful?
- Is the landing beat distinct from the build?
CHECK: Duration fit
- Word count aligns with cinematic pacing (slower than explainer — ~120 WPM)
```
### After SCENE_PLAN stage:
```
CHECK: Hero moment definition
- Are hero frames (climax, reveal) explicitly identified?
- Is source footage prioritized over generated inserts?
- Do transitions support mood (not distract)?
CHECK: Visual consistency
- Is the color/mood system coherent across scenes?
- Are aspect ratio choices consistent (letterbox if used)?
```
### After ASSETS stage:
```
CHECK: Music/ambience alignment
- Does the music beat map align with the script beat map?
- Are generated inserts limited and justified?
- Budget gate: 90% threshold warning
CHECK: Source selects quality
- Are source clips properly identified and accessible?
- Do support assets (generated or stock) match source quality level?
```
### After EDIT stage:
```
CHECK: Emotional pacing
- Strong moments are not overcut
- Audio cues reinforce story beats
- Title-card timing is restrained
CHECK: Timeline completeness
- Full runtime covered, no gaps
- All asset references valid
```
### After COMPOSE stage:
```
CHECK: Output validation
- ffprobe: duration, resolution, codec
- Color grade applied and consistent
- Audio dynamics controlled — dialogue intelligible, music balanced
- Letterbox or frame treatment improves (not harms) the output
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | idea | Emotional arc, source mode | Revise |
| G2 | script | Beat escalation, duration | Revise |
| G3 | scene_plan | Hero moments, visual consistency | Revise |
| G4 | assets | Music alignment, source quality, budget | Revise |
| G5 | edit | Emotional pacing, timeline | Revise |
| G6 | compose | Output probe, color grade, audio dynamics | Revise or send-back |
| G7 | publish | Metadata, poster frame | Revise |
| FINAL | all | Mood coherence, audio, visual polish | Send-back |
## Execution Limits
| Limit | Value |
|-------|-------|
| Max revisions per stage | 3 |
| Max send-backs per stage pair | 1 |
| Max total send-backs | 3 |
| Max total budget | Configurable (default $2) |
| Max total wall-time | 12 minutes |
## Common Pitfalls
- **Over-grading**: Color grade should enhance mood, not make footage look artificial.
- **Overuse of generated inserts**: Source footage should be primary. Generated content fills gaps, not replaces.
- **Ignoring audio dynamics**: Cinematic videos live and die by their audio. Music/dialogue balance is critical.
- **Rushing the reveal**: The climax moment needs breathing room. Don't let pacing compress it.
skills/pipelines/cinematic/idea-director.md
+79
View File
@@ -0,0 +1,79 @@
# Idea Director - Cinematic Pipeline
## When To Use
Use this pipeline for trailers, brand films, dramatic montages, or mood-led short videos where rhythm, atmosphere, and emotional escalation matter more than direct explanation.
Do not use this pipeline just because the user said "make it look cinematic." If the project is really a screen demo, explainer, or repurposing job, route it there instead.
## Reference Inputs
- `docs/cinematic-best-practices.md`
- `skills/creative/cinematic.md`
- `skills/creative/storytelling.md`
## Process
### 1. Classify The Source Reality
Capture the source mode:
- `footage_only`
- `footage_plus_stills`
- `still_led`
- `generated_support`
- `mixed_montage`
Do not assume stock, generated b-roll, or music exists unless the user has provided it or the environment can actually make it.
### 2. Define The Emotional Arc
Choose the arc in plain language:
- tension -> reveal
- wonder -> scale
- intimacy -> payoff
- urgency -> resolution
- mystery -> CTA
The brief should tell later stages what the video is trying to make the viewer feel, not just what it is about.
### 3. Pick The Delivery Shape
Common output shapes:
- `teaser`
- `trailer`
- `hero_brand_film`
- `mood_cut`
- `social_cutdown`
Store longer planning detail in `brief.metadata`.
Recommended metadata keys:
- `source_mode`
- `delivery_shape`
- `emotional_arc`
- `anchor_assets`
- `music_strategy`
- `generated_support_level`
- `aspect_ratio_plan`
- `rights_constraints`
### 4. Reality Check The Treatment
If the user has weak source media and no generation path, say so. A cinematic result still needs enough visual or audio material to carry mood.
### 5. Quality Gate
- the source truth is explicit,
- the emotional arc is specific,
- the output shape fits the available assets,
- the treatment is cinematic for a reason, not by label only.
## Common Pitfalls
- Calling something cinematic when it is really just a normal edit with black bars.
- Assuming generated inserts are available without checking tools.
- Planning a trailer shape with no reveal or payoff.
skills/pipelines/cinematic/publish-director.md
+56
View File
@@ -0,0 +1,56 @@
# Publish Director - Cinematic Pipeline
## When To Use
Package the cinematic piece and any cutdowns so the hero version stays clear and the distribution intent is obvious.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/publish_log.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["compose"]["render_report"]`, `state.artifacts["idea"]["brief"]`, `state.artifacts["script"]["script"]` | Final outputs and beat map |
| Playbook | Active style playbook | Tone and naming consistency |
## Process
### 1. Separate Hero And Derivatives
Typical deliverables:
- hero trailer or brand film,
- teaser cut,
- social cutdown,
- poster-frame or thumbnail concept.
### 2. Match Metadata To Tone
Packaging should reflect the actual mood:
- dramatic,
- premium,
- mysterious,
- reflective,
- urgent.
### 3. Preserve Editorial Truth
Store in `publish_log.metadata`:
- `hero_output`
- `derivative_outputs`
- `poster_frame_notes`
- `distribution_notes`
### 4. Quality Gate
- hero export is clearly identified,
- derivative exports are labeled by purpose,
- metadata fits the tone,
- the package is usable without manual cleanup.
## Common Pitfalls
- Mixing teaser and hero outputs without clear naming.
- Writing generic metadata that ignores the mood.
- Treating all cutdowns as interchangeable.
skills/pipelines/cinematic/scene-director.md
+63
View File
@@ -0,0 +1,63 @@
# Scene Director - Cinematic Pipeline
## When To Use
You are deciding how each cinematic beat will look and transition. This is where mood becomes a visual plan.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/scene_plan.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Beat map and source truth |
| Tools | `frame_sampler`, `scene_detect` | Source inspection and reframing checks |
| Playbook | Active style playbook | Color and typography consistency |
## Process
### 1. Make Hero Frames Explicit
Every cinematic piece needs a few memorable frames. Define them directly:
- opening image,
- reveal image,
- final image,
- any title-card hero moments.
### 2. Keep Source-Led Scenes Primary
If source footage exists, let it carry the piece. Generated inserts or text cards should support transitions, emphasis, or missing coverage, not dominate the timeline.
### 3. Limit Transition Vocabulary
Choose a small set:
- hard cut,
- fade to black,
- slow dissolve,
- restrained push or punch-in.
Too many transition types kill the mood.
### 4. Use Metadata For Visual Rules
Recommended metadata keys:
- `hero_frames`
- `transition_rules`
- `aspect_ratio_rules`
- `title_card_rules`
- `support_insert_rules`
### 5. Quality Gate
- every beat has a scene treatment,
- hero frames are identifiable,
- support inserts are justified,
- the visual language stays consistent across the piece.
## Common Pitfalls
- Using title cards as filler.
- Treating generated inserts like the primary story without saying so.
- Planning flashy transitions for every beat.
skills/pipelines/cinematic/script-director.md
+69
View File
@@ -0,0 +1,69 @@
# Script Director - Cinematic Pipeline
## When To Use
This stage builds the beat map, selected lines, title-card copy, and reveal structure for the cinematic piece. You are shaping rhythm, not writing a dense explainer.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/script.schema.json` | Artifact validation |
| Prior artifact | `state.artifacts["idea"]["brief"]` | Emotional arc and source truth |
| Tools | `transcriber`, `scene_detect` | Optional dialogue mining and source review |
## Process
### 1. Build A Beat Map First
Use a simple structure:
- hook,
- escalation,
- reveal,
- landing.
If the piece is longer, add one midpoint turn. Do not let it become essay-shaped.
### 2. Use Dialogue Sparingly
If source speech exists, use `transcriber` to find:
- strong standalone lines,
- emotional phrases,
- concise declarations,
- reveal phrases.
If there is no useful dialogue, keep the script title-led or narration-led and say so in metadata.
### 3. Keep Title Cards Short
Title-card copy should feel trailer-like:
- fewer words,
- more contrast,
- more whitespace,
- more timing precision.
### 4. Store Beat Truth In Metadata
Recommended metadata keys:
- `beat_map`
- `dialogue_selects`
- `title_card_copy`
- `music_turns`
- `silence_windows`
### 5. Quality Gate
- the beat map escalates cleanly,
- dialogue and title cards do not explain the same thing twice,
- the reveal lands distinctly,
- the landing gives the viewer a final feeling or action.
## Common Pitfalls
- Writing full explanatory paragraphs instead of beats.
- Using too many title cards.
- Revealing the best moment too early.
skills/pipelines/clip-factory/asset-director.md
+68
View File
@@ -0,0 +1,68 @@
# Asset Director - Clip Factory Pipeline
## When To Use
This stage builds the shared visual and audio kit for the entire clip batch. The key is reuse, not bespoke design per clip.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Clip plans and rankings |
| Tools | `subtitle_gen`, `audio_enhance` | Batch-ready subtitles and audio cleanup |
| Playbook | Active style playbook | Subtitle and overlay consistency |
## Process
### 1. Build Shared Assets First
Prefer reusable assets over per-clip reinvention:
- one subtitle style system,
- one hook text treatment,
- one lower-third treatment,
- one watermark / brand frame,
- one CTA / end-tag treatment if needed.
### 2. Generate Per-Clip Subtitles
Each approved clip needs its own subtitle asset, timed from clip start rather than source start. This timestamp rebasing is critical.
Store clip-relative timing details in `asset_manifest.metadata.subtitle_map`.
### 3. Normalize Audio Consistently
Use `audio_enhance` across the clip set so the batch feels like one series:
- similar loudness,
- similar noise floor,
- similar vocal clarity.
### 4. Keep Hook Assets Lightweight
Most hook overlays should be text-first and template-based. Do not spend time or budget generating bespoke art unless the batch truly benefits.
### 5. Use Metadata For Batch Structure
Recommended metadata keys:
- `shared_assets`
- `subtitle_map`
- `audio_profile`
- `clip_asset_index`
- `style_tokens`
### 6. Quality Gate
- every clip has subtitles,
- every clip has a clean audio asset or verified source audio path,
- shared assets are referenced consistently,
- the asset count stays practical for the batch size.
## Common Pitfalls
- Forgetting to rebase subtitle timing per clip.
- Overdesigning hook assets so the batch becomes inconsistent.
- Normalizing some clips and not others.
- Treating a 10-clip batch like 10 unrelated projects.
skills/pipelines/clip-factory/compose-director.md
+61
View File
@@ -0,0 +1,61 @@
# Compose Director - Clip Factory Pipeline
## When To Use
Render each clip and platform variant independently. The important behaviors here are consistency, batch resilience, and clear reporting of partial failures.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | Clip edits and assets |
| Tools | `video_trimmer`, `video_compose`, `audio_mixer`, `color_grade` | Render pipeline |
| Media profiles | `lib/media_profiles.py` | Platform targets |
## Process
### 1. Treat Each Output As Its Own Job
One clip across three platforms is three render jobs. Name and track them explicitly.
### 2. Reuse What Can Be Shared
- shared audio mix where possible,
- shared subtitle styling,
- shared overlay assets,
- shared grading if the source needs it.
### 3. Fail Softly
If one clip or one platform variant fails:
- log it clearly,
- continue the rest of the batch,
- do not block successful exports.
### 4. Verify Every Output
Per render:
- correct duration,
- correct resolution/aspect ratio,
- no black opening frame,
- hook appears on time,
- subtitles render correctly,
- audio is present and consistent.
### 5. Use Render Report Metadata
Recommended metadata keys:
- `job_index`
- `failed_jobs`
- `shared_intermediates`
- `platform_groupings`
## Common Pitfalls
- Rendering sequentially without reason when jobs are independent.
- Treating a failed clip as a reason to stop the batch.
- Letting one platform variant quietly use the wrong framing or subtitle zone.
skills/pipelines/clip-factory/edit-director.md
+66
View File
@@ -0,0 +1,66 @@
# Edit Director - Clip Factory Pipeline
## When To Use
This stage turns the approved clips into independent mini-edits. Each clip must work alone, but the collection should still feel like a coherent series.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/edit_decisions.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["assets"]["asset_manifest"]`, `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]` | Assets, layouts, transcripts |
| Playbook | Active style playbook | Transition and subtitle consistency |
## Process
### 1. Build A Shared Edit Template
Lock the batch defaults first:
- subtitle style,
- hook timing,
- lower-third timing,
- watermark behavior,
- audio fade lengths.
Then apply per-clip overrides only where necessary.
### 2. Optimize The First 2-3 Seconds
For every clip:
- start on motion, face, or result,
- show hook text immediately if needed,
- let subtitles begin with the first spoken word,
- avoid intros that delay the point.
### 3. Keep Boundaries Clean
- no cuts mid-word,
- no trailing silence after the point lands,
- no "setup for setup's sake" before the hook,
- no outro cards unless they earn the time.
### 4. Use Metadata For Multi-Variant Detail
Recommended metadata keys:
- `batch_template`
- `clip_variants`
- `hook_windows`
- `cta_windows`
### 5. Quality Gate
- each clip is self-contained,
- the first seconds hook fast,
- overlay stack is readable on mobile,
- the batch retains consistent styling and fades.
## Common Pitfalls
- Building one highlight reel instead of independent clips.
- Letting branding delay the hook.
- Overcrowding the screen with hook text, subtitles, watermark, and lower third simultaneously.
- Applying inconsistent transition timing across the batch.
skills/pipelines/clip-factory/executive-producer.md
+136
View File
@@ -0,0 +1,136 @@
# Executive Producer — Clip Factory Pipeline
## When to Use
You are the **Executive Producer (EP)** for a clip factory project. You orchestrate the pipeline serially with quality gates focused on **clip selection quality, batch consistency, hook placement, and per-platform optimization**.
**No pre-production stages.** Long-form source content exists. The EP manages the extraction of multiple independent short clips, ensuring each stands alone while maintaining series consistency.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/clip-factory.yaml` | Stage definitions |
| Skills | All 7 director skills + `meta/reviewer` | Stage execution |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
## Cumulative State
```
EP_STATE:
pipeline: clip-factory
playbook: <selected>
budget_total_usd: <configured>
budget_spent_usd: 0.0
# Clip-factory specific
source_type: null # webinar / stream / presentation / interview
clip_count_target: 0
platform_targets: [] # per-clip platform assignments
clips_completed: 0
artifacts:
idea: null
script: null
scene_plan: null
assets: null
edit: null
compose: null
publish: null
revision_counts: {}
issues_log: []
```
## EP-Specific Cross-Stage Checks
### After IDEA stage:
```
CHECK: Source and targets
- Source content type identified?
- Clip count target realistic for source duration? (rule of thumb: 1 clip per 5-10 min source)
- Platform mix defined?
- Clip selection criteria clear?
```
### After SCRIPT stage:
```
CHECK: Transcript and ranking
- Full transcription with accurate timestamps?
- At least N candidate clips where N >= clip_count_target?
- Each candidate is self-contained (standalone test)?
- Clips ranked by quality/engagement potential?
```
### After SCENE_PLAN stage:
```
CHECK: Clip boundaries
- Each clip has clean in/out points (no mid-sentence cuts)?
- Platform-specific framing planned (vertical vs square vs landscape)?
- No clip exceeds platform max duration?
CHECK: Batch diversity
- Clips cover different topics/moments from the source?
- Not all clips from one section of the source?
```
### After ASSETS stage:
```
CHECK: Batch consistency
- Per-clip subtitles with correct time offsets?
- Shared branding assets (title cards, hooks) prepared?
- Audio normalized consistently across all clips?
- Budget gate: 90% threshold warning
```
### After EDIT stage:
```
CHECK: Hook placement
- Each clip has its hook within first 2-3 seconds?
- Subtitle styling consistent across all clips?
- Each edit is independent (no cross-clip dependencies)?
CHECK: Completeness
- Edit decisions exist for every planned clip?
```
### After COMPOSE stage:
```
CHECK: Batch render validation
- All clips rendered successfully?
- Each clip meets target platform specs (resolution, aspect ratio)?
- Audio levels consistent across clips?
- No clip has rendering artifacts?
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | idea | Source assessment, clip targets | Revise |
| G2 | script | Transcript quality, clip ranking | Revise |
| G3 | scene_plan | Clip boundaries, batch diversity | Revise |
| G4 | assets | Batch consistency, audio normalization | Revise |
| G5 | edit | Hook placement, completeness | Revise |
| G6 | compose | Batch render probe, platform specs | Revise or send-back |
| G7 | publish | Per-clip metadata, posting order | Revise |
| FINAL | all | Clip quality, consistency, hooks | Send-back |
## Execution Limits
| Limit | Value |
|-------|-------|
| Max revisions per stage | 3 |
| Max send-backs per stage pair | 1 |
| Max total send-backs | 3 |
| Max total budget | Configurable (default $1) |
| Max total wall-time | 12 minutes |
## Common Pitfalls
- **Context-dependent clips**: Each clip must make sense alone. No "as I was saying" openings.
- **Slow hooks**: Social clips need to hook in 2-3 seconds. Front-load the interesting part.
- **Inconsistent audio levels**: Clips from different parts of the source have different audio levels. Normalize.
- **Missing platform optimization**: A YouTube clip and a TikTok clip need different aspect ratios.
- **All clips from one section**: Diverse clips from across the source perform better than 5 clips from the same 10 minutes.
skills/pipelines/clip-factory/idea-director.md
+98
View File
@@ -0,0 +1,98 @@
# Idea Director - Clip Factory Pipeline
## When To Use
Use this pipeline when the source is long-form footage and the goal is multiple short-form deliverables: webinar clips, interview cuts, livestream highlights, keynote excerpts, or presentation snippets.
You are not planning one video. You are planning a ranked portfolio of clips.
## Reference Inputs
- `docs/clip-factory-best-practices.md`
- `skills/creative/short-form.md`
- `skills/creative/video-editing.md`
## Process
### 1. Understand The Source And The Goal
Capture the source shape:
- webinar
- interview
- panel
- keynote
- stream
- customer story
Then capture the business goal:
- awareness
- thought leadership
- lead generation
- product education
- event recap
### 2. Choose A Clip Portfolio Strategy
A good batch mixes clip types instead of extracting the same energy repeatedly.
Common clip families:
- `hook`: surprising claim or strong cold open
- `insight`: useful takeaway or lesson
- `story`: narrative moment with emotional shape
- `proof`: stat, case study, demo result
- `opinion`: hot take, disagreement, contrarian point
Use the brief metadata to define the intended balance across those families.
### 3. Set Yield Targets Realistically
Guideline ranges:
- `15-30 min`: 3-6 strong clips
- `30-60 min`: 5-10 strong clips
- `60+ min`: 8-15 strong clips if the source quality supports it
Do not inflate clip count to satisfy a round number. A smaller strong batch beats a padded weak batch.
### 4. Map Platforms Before Extraction
Plan platform fit early:
- `9:16` for Shorts, Reels, TikTok
- `1:1` for LinkedIn and safer feed repurposing
- `16:9` when slides, demos, or wide context matter
If the source framing clearly will not survive vertical crops, say so in the brief metadata now.
### 5. Build The Brief
Keep the schema-level brief concise and put the richer batch plan in `brief.metadata`.
Recommended metadata keys:
- `source_type`
- `source_duration_seconds`
- `clip_target_range`
- `clip_families`
- `primary_platforms`
- `secondary_platforms`
- `selection_criteria`
- `known_visual_constraints`
- `distribution_goal`
### 6. Quality Gate
- the clip count target is realistic,
- the platform mix matches the content,
- the brief defines ranking criteria before extraction starts,
- the agent has acknowledged any obvious reframing limits.
## Common Pitfalls
- Planning a batch around quantity before quality.
- Assuming every source can produce vertical clips cleanly.
- Treating all clips as interchangeable instead of intentionally varied.
- Starting extraction without defining what "good" means for this batch.
skills/pipelines/clip-factory/publish-director.md
+60
View File
@@ -0,0 +1,60 @@
# Publish Director - Clip Factory Pipeline
## When To Use
This stage packages the clip batch into a distribution plan. The goal is not just exported files. The goal is a usable content engine.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/publish_log.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["compose"]["render_report"]`, `state.artifacts["idea"]["brief"]`, `state.artifacts["script"]["script"]` | Outputs, rankings, and goals |
| Playbook | Active style playbook | Brand voice |
## Process
### 1. Lead With The Strongest Clip
Do not schedule by chronology. Schedule by ranking.
The first published clip should usually be:
- the strongest hook,
- the cleanest standalone clip,
- the clip most aligned with the batch goal.
### 2. Tailor Copy By Platform
Each platform needs its own tone and packaging:
- TikTok / Reels: direct, fast, hook-led
- Shorts: searchable, keyword-aware
- LinkedIn: insight-led and more professional
- X: short, punchy, opinion-friendly
### 3. Package The Batch Cleanly
Group by platform and include ready-to-paste text assets, not just video files.
### 4. Preserve Batch Truth
Store in `publish_log.metadata`:
- `clip_catalog`
- `posting_order`
- `platform_copy_map`
- `schedule_notes`
### 5. Quality Gate
- strongest clips lead the rollout,
- captions are platform-specific,
- export folders are usable without extra cleanup,
- the batch catalog clearly links ranking, file paths, and publishing intent.
## Common Pitfalls
- Publishing the whole batch on the same day.
- Using one caption everywhere.
- Losing the rank/order logic after rendering is complete.
skills/pipelines/clip-factory/scene-director.md
+76
View File
@@ -0,0 +1,76 @@
# Scene Director - Clip Factory Pipeline
## When To Use
You are planning how each selected clip will be framed and packaged for its destination platform. This is where clip viability gets proven or disproven.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/scene_plan.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Selected clips and platform goals |
| Tools | `frame_sampler`, `scene_detect` | Visual checks and boundary inspection |
| Playbook | Active style playbook | Layout language and safe zones |
## Process
### 1. Choose The Right Frame For Each Clip
Do not default every clip to `9:16`.
Use:
- `9:16` when a face-first crop works,
- `1:1` when speaker plus context both matter,
- `16:9` when slides, demos, or multi-speaker width are essential.
OpenMontage does not yet have first-class auto-reframe. If a vertical crop is weak, plan a safer aspect ratio instead of pretending the crop will work.
### 2. Plan First-Second Composition
For each clip, define:
- what the viewer sees on frame 1,
- where hook text can safely appear,
- where subtitles can live,
- whether the speaker needs a punch-in or whether the original framing is already good.
### 3. Standardize The Batch
Use the scene plan to lock series consistency:
- same top hook zone,
- same subtitle zone,
- same watermark / brand area,
- same lower-third logic.
### 4. Store Reframe Detail In Metadata
The schema is generic, so store richer layout notes in `scene_plan.metadata`.
Recommended metadata keys:
- `clip_layouts`
- `safe_zones`
- `crop_variants`
- `speaker_positions`
- `platform_variants`
### 5. Use Scenes To Represent Deliverables
Each scene should map to one clip variant or one clip family deliverable. Keep `description` human-readable and use `required_assets` for hook overlays, lower thirds, or branded frames.
### 6. Quality Gate
- every clip has a platform-aware framing plan,
- hook and subtitle zones do not collide,
- weak vertical crops are downgraded honestly,
- the batch will feel visually consistent when rendered together.
## Common Pitfalls
- Center-cropping a wide shot and calling it vertical optimization.
- Ignoring slide or screen-share content while focusing only on faces.
- Letting each clip invent its own layout.
- Forgetting that the first frame determines whether a viewer keeps watching.
skills/pipelines/clip-factory/script-director.md
+90
View File
@@ -0,0 +1,90 @@
# Script Director - Clip Factory Pipeline
## When To Use
This stage converts the long-form source into a ranked candidate list and then into the final clip selections. You are mining for standout moments, not summarizing the entire source.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/script.schema.json` | Artifact validation |
| Prior artifact | `state.artifacts["idea"]["brief"]` | Batch goals and platform targets |
| Tools | `transcriber`, `scene_detect` | Transcript-first selection and visual checks |
## Process
### 1. Transcribe The Full Source
Use `transcriber` first. The transcript is the search surface for hooks, not an afterthought.
Use `scene_detect` only to sanity-check visual boundaries, speaker changes, or slide changes near promising moments.
### 2. Score Candidate Moments
Use the brief's ranking criteria and evaluate each moment on:
- `hook`
- `coherence`
- `value`
- `energy`
- `platform_fit`
This mirrors the way modern clipping products talk about virality and clip quality, while keeping the judgment transparent.
### 3. Apply The Standalone Test
Every approved clip must make sense to a cold viewer.
Reject or widen clips that contain:
- unresolved pronouns,
- references to earlier context,
- long lead-ins before the point lands,
- endings that stop before the payoff.
### 4. Select The Final Batch
Pick the smallest set that best satisfies the batch goal.
Maintain diversity across:
- source sections,
- speakers,
- clip families,
- energy levels.
### 5. Use Metadata For Ranking Truth
The script schema is small, so store the richer batch analysis in `script.metadata`.
Recommended metadata keys:
- `candidate_clips`
- `selected_clip_ids`
- `ranking_model`
- `rejected_candidates`
- `source_coverage_map`
- `platform_assignments`
Each candidate should record:
- source in/out,
- hook text,
- reason selected or rejected,
- scoring dimensions,
- likely crop viability.
### 6. Quality Gate
- the top-ranked clips are genuinely the strongest, not just the earliest found,
- every selected clip passes the standalone test,
- the set covers the source deliberately instead of clustering in one section,
- low-quality candidates are rejected honestly.
## Common Pitfalls
- Trusting first-pass candidate timestamps without transcript-level review.
- Selecting too many calm, same-energy clips.
- Preserving chronological order instead of ranking by quality.
- Treating transcript quality issues as minor when they affect selection accuracy.
skills/pipelines/explainer/asset-director.md
+192
View File
@@ -0,0 +1,192 @@
# Asset Director — Explainer Pipeline
## When to Use
You are the Asset Producer for a generated explainer video. You have a `scene_plan` with required assets and a `script` with narration text. Your job is to generate every asset needed: narration audio, images, diagrams, code snippets, and background music. Every file must exist on disk before you finish.
This is where plans become real files. A missing or low-quality asset will torpedo the final video.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | What to produce |
| Playbook | Active style playbook | Image prompts, diagram style, audio preferences |
| Tools | `tts_selector`, `image_selector`, `video_selector`, `diagram_gen`, `code_snippet`, `music_gen` — selectors auto-discover all available providers from the registry | Generation capabilities |
| Cost tracker | `tools/cost_tracker.py` | Budget governance |
## Process
### Step 1: Inventory Required Assets
Walk every scene in the scene plan. For each `required_assets` entry, create an asset task:
```
Asset Task:
scene_id: scene-3
type: diagram
description: "Mermaid flowchart: query -> encode -> search -> rank -> return"
source: generate
tool: diagram_gen
estimated_cost: $0.00
```
Also create tasks for:
- **Narration audio** — one per script section (use `tts_selector` or a concrete TTS provider)
- **Background music** — one track for the whole video (use `music_gen` or select from library)
- **Sound effects** — per playbook's `sfx_style` (optional, use `music_gen` or stock)
### Step 2: Check Budget
Before generating anything:
1. Sum all estimated costs from the asset tasks
2. Compare against the cost tracker's remaining budget
3. If over budget:
- Switch expensive tools to cheaper alternatives (use `tts_selector` with `preferred_provider` to route to cheaper TTS; use `image_selector` to route to cheaper image providers)
- Reduce image count (combine similar scenes)
- Skip optional assets (SFX, B-roll)
4. Get cost approval via cost tracker before proceeding
### Step 2b: Sample Preview (Prevents Wasted Spend)
Before batch-generating assets, produce one sample of each expensive asset type and present them to the user for approval:
1. **TTS sample**: Generate narration for the first script section only. Play it for the user. Confirm voice, pace, and tone are acceptable before generating the rest.
2. **Image sample**: Generate one image for the most representative scene. Show it to the user. Confirm the style, quality, and prompt approach before batch-generating all images.
3. **Music sample** (if using `music_gen`): Generate one short clip. Confirm mood and energy before committing.
If the user rejects a sample:
- Adjust the parameters (voice, prompt style, provider) and regenerate the sample.
- Do not batch-generate until the sample is approved.
- Max 3 sample iterations per asset type before escalating to the user for a decision.
This step typically costs $0.03–0.08 total and prevents $1–3 of wasted generation.
### Step 3: Generate Narration
For each script section:
1. Extract the narration text
2. Apply speaker directions from the script (pace, emphasis, emotion)
3. Apply the playbook's `audio.voice_style`
4. Generate using `tts_selector` — it auto-routes to the best available TTS provider based on user preference and availability. Check the registry's `best_for` fields to understand each provider's strengths.
5. Verify the audio file exists and duration matches expected timing (±15%)
**Pronunciation guide**: If the script contains technical terms, jargon, or names with non-obvious pronunciation, include a pronunciation map in the TTS request.
### Step 4: Generate Visual Assets
Process asset tasks grouped by tool for efficiency:
**Images (`image_selector`)**:
1. Build the prompt: `playbook.asset_generation.image_prompt_prefix` + scene description + style cues
2. Add negative prompt from playbook
3. Include consistency anchors (same palette, same style across all images)
4. Generate and verify the file exists
5. If the result doesn't match expectations, refine the prompt and regenerate (max 2 retries)
**Diagrams (`diagram_gen`)**:
1. Convert the scene description into valid Mermaid syntax
2. Apply playbook's `asset_generation.diagram_style`
3. Generate SVG/PNG
4. Verify all nodes and edges are present
**Code snippets (`code_snippet`)**:
1. Extract language and code from the scene description
2. Apply syntax highlighting theme from playbook's overlay styles
3. Generate highlighted image or Remotion-compatible data
### Step 5: Generate Music
1. Read playbook's `audio.music_mood` and `audio.music_volume`
2. Generate or select a background track:
- **Primary**: `music_gen` (ElevenLabs Music) — custom, costs per generation
- **Fallback**: Stock music library (if available)
3. Duration should match total video duration (or be loopable)
4. Verify the audio file exists
### Step 6: Build Asset Manifest
Assemble all generated assets into the manifest:
```json
{
"version": "1.0",
"assets": [
{
"id": "narration-s1",
"type": "audio",
"subtype": "narration",
"path": "assets/narration/s1.mp3",
"source_tool": "tts_selector",
"scene_id": "scene-1",
"duration_seconds": 8.2,
"cost_usd": 0.003
},
{
"id": "img-scene-3",
"type": "image",
"path": "assets/images/scene-3-diagram.png",
"source_tool": "diagram_gen",
"scene_id": "scene-3",
"cost_usd": 0.00
},
{
"id": "music-bg",
"type": "audio",
"subtype": "music",
"path": "assets/music/background.mp3",
"source_tool": "music_gen",
"duration_seconds": 62,
"cost_usd": 0.05
}
],
"total_cost_usd": 0.053,
"generation_summary": {
"narration_sections": 5,
"images_generated": 8,
"diagrams_generated": 2,
"music_tracks": 1
}
}
```
### Step 7: Verify All Assets
**Existence check:**
- [ ] Every asset `path` exists on disk
- [ ] Every narration section has a corresponding audio file
- [ ] Every scene with `required_assets` has all assets generated
- [ ] Background music file exists
**Quality check:**
- [ ] Narration durations within ±15% of expected timing
- [ ] Images match the playbook's style (review consistency anchors)
- [ ] Diagrams are legible and complete
- [ ] Total cost within budget
### Step 8: Self-Evaluate
Score (1-5):
| Criterion | Question |
|-----------|----------|
| **Completeness** | Does every scene have all required assets? |
| **Audio quality** | Does narration sound natural with correct pacing? |
| **Visual consistency** | Do all images look like they belong to the same video? |
| **Budget adherence** | Is total cost within the approved budget? |
| **Playbook fidelity** | Do assets match the playbook's style guide? |
If any dimension scores below 3, fix before proceeding.
### Step 9: Submit
Validate the asset_manifest against the schema and persist via checkpoint.
## Common Pitfalls
- **Generating before checking budget**: Always estimate total cost first. A 60-second video with 15 images can burn $3+ quickly.
- **Inconsistent image style**: Each image_selector call is independent. Without explicit consistency anchors in every prompt, images will drift. Always include the playbook prefix.
- **Ignoring narration timing**: If TTS produces 12s of audio for a 10s section, the edit phase will struggle. Check durations.
- **Missing pronunciation guide**: "PostgreSQL" or "Kubernetes" will be mispronounced without explicit guidance.
- **One retry then give up**: If an image doesn't match, refine the prompt specifically — don't just retry the same prompt.
skills/pipelines/explainer/compose-director.md
+170
View File
@@ -0,0 +1,170 @@
# Compose Director — Explainer Pipeline
## When to Use
You are the Compositor for a generated explainer video. You have `edit_decisions` with the complete edit timeline and an `asset_manifest` with all file paths. Your job is to render the final video: assemble visuals, layer audio, burn subtitles, and encode to the target format.
This is the last technical stage before the video exists as a playable file. Everything converges here.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | What to render |
| Playbook | Active style playbook | Quality targets |
| Tools | `video_compose`, `audio_mixer` | Rendering capabilities |
| Media profiles | `lib/media_profiles.py` | Output format specs (resolution, codec, bitrate) |
## Process
### Step 1: Choose Render Strategy
Based on the edit decisions, pick the rendering approach:
**FFmpeg pipeline** (simpler videos):
- Static images with Ken Burns
- Audio layering
- Subtitle burn-in
- Best for: diagram-heavy, image-based explainers
**Remotion render** (motion-heavy videos):
- Animated text cards, stat cards
- Complex transitions (morph, zoom)
- Programmatic motion graphics
- Best for: flat-motion-graphics playbook, animation-heavy plans
You can combine both: Remotion for animated segments, FFmpeg for final assembly.
### Step 2: Prepare Render Inputs
For each cut in the edit decisions:
1. Verify the source asset exists at its declared path
2. Check asset dimensions/duration match expectations
3. Prepare transform parameters (scale, position, crop)
For audio:
1. Verify all narration segments exist
2. Verify music track exists
3. Prepare ducking parameters from edit decisions
### Step 3: Determine Output Profile
Read the target platform from the brief artifact. Map to a media profile:
| Platform | Profile | Resolution | Notes |
|----------|---------|-----------|-------|
| YouTube | `youtube_landscape` | 1920x1080 | Default for most explainers |
| TikTok/Reels | `tiktok` | 1080x1920 | Vertical, needs reframing |
| Twitter/X | `twitter_landscape` | 1280x720 | Shorter format |
| LinkedIn | `linkedin` | 1920x1080 | Professional context |
Get the exact encoding parameters via `ffmpeg_output_args(get_profile(name))`.
### Step 4: Render Video
Call the `video_compose` tool with:
```
{
"operation": "render",
"edit_decisions": <edit_decisions artifact>,
"asset_manifest": <asset_manifest artifact>,
"output_profile": "youtube_landscape",
"output_path": "renders/output.mp4",
"options": {
"subtitle_burn": true,
"audio_normalize": true,
"two_pass_encode": true
}
}
```
If using Remotion for animated segments:
1. Generate Remotion composition data from edit decisions
2. Call `video_compose` with `operation: "remotion_render"` for animated segments
3. Assemble Remotion outputs with remaining segments via FFmpeg
### Step 5: Audio Post-Processing
Call the `audio_mixer` tool to:
1. Layer narration segments in order
2. Mix background music at playbook volume
3. Apply ducking (music dips during narration)
4. Normalize overall audio levels
5. Output the final mixed audio track
The video_compose tool will mux this with the video.
### Step 6: Verify Output
**File verification:**
- [ ] Output file exists at declared path
- [ ] File size is reasonable (not 0 bytes, not suspiciously small)
- [ ] File is a valid container (ffprobe succeeds)
**Content verification:**
- [ ] Duration within ±5% of target
- [ ] Resolution matches selected profile
- [ ] Audio channels present (stereo)
- [ ] No audio clipping or silence gaps > 1s
**Quality check:**
- [ ] Visual: scrub through at 25%, 50%, 75% marks — images display correctly
- [ ] Audio: narration is audible and clear throughout
- [ ] Subtitles: visible and correctly timed
### Step 7: Build Render Report
```json
{
"version": "1.0",
"outputs": [
{
"path": "renders/output.mp4",
"format": "mp4",
"codec": "h264",
"resolution": "1920x1080",
"fps": 30,
"duration_seconds": 62.4,
"file_size_mb": 45.2,
"audio_codec": "aac",
"audio_channels": 2,
"render_strategy": "ffmpeg",
"render_time_seconds": 180
}
],
"render_summary": {
"total_cuts_rendered": 12,
"subtitles_burned": true,
"audio_tracks_mixed": 3,
"target_duration_seconds": 60,
"actual_duration_seconds": 62.4
}
}
```
### Step 8: Self-Evaluate
Score (1-5):
| Criterion | Question |
|-----------|----------|
| **Playability** | Does the video play without errors in a standard player? |
| **Duration accuracy** | Is actual duration within ±5% of target? |
| **Audio quality** | Is narration clear, music balanced, no clipping? |
| **Visual quality** | Are images sharp, transitions smooth, no artifacts? |
| **Subtitle accuracy** | Are subtitles present, readable, and synced? |
If any dimension scores below 3, investigate and re-render.
### Step 9: Submit
Validate the render_report against the schema and persist via checkpoint.
## Common Pitfalls
- **Missing asset files**: Always verify every referenced file exists before starting the render. A missing file mid-render wastes time.
- **Audio sync drift**: Accumulated timing errors across narration segments cause audio-visual desync. Use absolute timestamps, not relative offsets.
- **Subtitle encoding**: Burn subtitles into the video (hardcoded) for maximum compatibility. Don't rely on soft subtitles for social media.
- **Single-pass encode**: Two-pass encoding produces better quality at the same file size. Worth the extra render time.
- **Ignoring media profile**: YouTube and TikTok have very different requirements. Always check the target profile.
skills/pipelines/explainer/edit-director.md
+170
View File
@@ -0,0 +1,170 @@
# Edit Director — Explainer Pipeline
## When to Use
You are the Editor for a generated explainer video. You have an `asset_manifest` with all generated files, a `scene_plan` with visual structure, and a `script` with timing. Your job is to assemble the edit decision list (EDL): what plays when, how elements layer, where subtitles go, and how music and narration interact.
This is where raw assets become a coherent video. Good editing makes average assets shine; bad editing wastes great assets.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/edit_decisions.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["assets"]["asset_manifest"]`, `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]` | Assets, visual plan, timing |
| Playbook | Active style playbook | Transitions, pacing rules, overlay styles |
## Process
### Step 1: Map Assets to Timeline
For each scene in the scene plan:
1. Find the matching assets from the asset manifest (by `scene_id`)
2. Find the matching narration audio (by script section)
3. Note the scene's timing (`start_seconds`, `end_seconds`)
Build a timeline map:
```
0s-10s: scene-1 (talking_head) | narration-s1 | img-intro.png
10s-18s: scene-2 (diagram) | narration-s2 | diagram-flow.svg
18s-22s: scene-3 (text_card) | narration-s3 | [text overlay]
...
```
### Step 2: Define Cuts
Each cut defines what visual is shown and when:
```json
{
"id": "cut-1",
"source": "img-scene-1",
"in_seconds": 0,
"out_seconds": 10,
"layer": "primary",
"transform": {
"scale": 1.0,
"position": "center",
"animation": "ken-burns-slow-zoom"
},
"transition_in": "fade",
"transition_out": "dissolve",
"transition_duration": 0.4
}
```
**Layering rules:**
- `primary` — main visual (one at a time)
- `overlay` — text cards, stat cards, key terms (on top of primary)
- `background` — solid color or texture behind everything
### Step 3: Configure Subtitles
Subtitles are mandatory for all explainer content:
```json
{
"subtitles": {
"enabled": true,
"style": "word-by-word",
"font": "Inter",
"font_size": 48,
"color": "#FFFFFF",
"background": "#00000088",
"position": "bottom-center",
"max_words_per_line": 8
}
}
```
**Subtitle timing**: Derive from narration audio timestamps. Each word should highlight as it's spoken (word-by-word style) or display in phrase chunks (phrase style).
Use the playbook's typography for font choices.
### Step 4: Configure Audio Layers
```json
{
"audio": {
"narration": {
"segments": [
{ "asset_id": "narration-s1", "start_seconds": 0 },
{ "asset_id": "narration-s2", "start_seconds": 10 }
]
},
"music": {
"asset_id": "music-bg",
"volume": 0.08,
"fade_in_seconds": 2,
"fade_out_seconds": 3,
"ducking": {
"enabled": true,
"threshold_db": -3,
"reduction_db": -8,
"attack_ms": 200,
"release_ms": 500
}
},
"sfx": []
}
}
```
**Music ducking**: Music volume drops when narration plays, rises during pauses. Use playbook's `audio.ducking_threshold_db`.
### Step 5: Apply Pacing Rules
Check the playbook's `motion.pacing_rules`:
- No cut shorter than `min_scene_hold_seconds`
- No cut longer than `max_scene_hold_seconds`
- Text cards hold for `text_card_hold_seconds`
- Transitions use `transition_duration_seconds`
Adjust cut timing if any violates these rules.
### Step 6: Verify Edit Completeness
**Timeline coverage:**
- [ ] Cuts span full video duration (no black frames)
- [ ] No overlapping primary cuts
- [ ] Every scene in scene_plan has at least one corresponding cut
**Asset references:**
- [ ] Every cut's `source` references a valid asset_id from the manifest
- [ ] Every narration segment references a valid audio asset
- [ ] Music asset exists
**Audio sync:**
- [ ] Narration segments are ordered and non-overlapping
- [ ] Narration timing aligns with corresponding visual cuts
- [ ] Music ducking is configured
**Subtitles:**
- [ ] Subtitles enabled
- [ ] Subtitle style uses playbook-compatible fonts and colors
### Step 7: Self-Evaluate
Score (1-5):
| Criterion | Question |
|-----------|----------|
| **Continuity** | Does every second of the video have a visual? |
| **Pacing** | Do cuts follow the playbook's timing rules? |
| **Audio-visual sync** | Does what you see match what you hear at every moment? |
| **Subtitle quality** | Are subtitles readable and correctly timed? |
| **Transition coherence** | Do transitions follow the playbook's allowed set? |
If any dimension scores below 3, revise.
### Step 8: Submit
Validate the edit_decisions artifact against the schema and persist via checkpoint.
## Common Pitfalls
- **Forgetting gaps**: If scene-1 ends at 10s and scene-2 starts at 10.5s, there's a 0.5s black frame. Check for gaps.
- **Audio drift**: Narration audio may be slightly longer/shorter than planned. Adjust visual cuts to match actual narration durations, not planned durations.
- **No ducking**: Music playing at full volume under narration makes the video unwatchable. Always configure ducking.
- **Same transition everywhere**: Varying transitions creates rhythm. Use the playbook's allowed set, but don't use the same one for every cut.
- **Subtitle font mismatch**: Subtitles should use the playbook's body font, not a random default.
skills/pipelines/explainer/executive-producer.md
+423
View File
@@ -0,0 +1,423 @@
# Executive Producer — Explainer Pipeline
## When to Use
You are the **Executive Producer (EP)** for a generated explainer video. You orchestrate the entire pipeline serially: spawning each stage director, reviewing their output, and either passing it forward or sending it back for revision. You are the stateful brain; the directors are stateless workers.
**You replace the default parallel/sequential execution model.** Instead of running all stages blindly, you exercise judgment at every gate.
## Why This Exists
The parallel pipeline produces "technically correct" but low-quality videos because:
- No feedback loop when TTS narration is too long for the video duration
- No style consistency enforcement across image generation calls
- No A/V sync validation before the final render
- No budget reallocation when early stages overspend
- No ability to send a single stage back without re-running everything
The EP solves all of these by maintaining cumulative state and applying judgment at each gate.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/animated-explainer.yaml` | Stage definitions, review focus, success criteria |
| Skills | All 7 director skills + `meta/reviewer` | Stage execution knowledge |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
| Tools | Full tool registry | Available capabilities |
## Cumulative State
The EP maintains a running state object that flows through the entire pipeline:
```
EP_STATE:
pipeline: animated-explainer
playbook: <selected playbook name>
target_duration_seconds: <from proposal_packet.selected_concept>
budget_total_usd: <from proposal_packet.approval.approved_budget_usd or configured limit>
budget_spent_usd: 0.0
budget_remaining_usd: <budget_total>
# Accumulated from each stage (8 stages)
artifacts:
research: null # → research_brief
proposal: null # → proposal_packet (includes approval gate)
script: null # → script
scene_plan: null # → scene_plan
assets: null # → asset_manifest
edit: null # → edit_decisions
compose: null # → render_report
publish: null # → publish_log
# Pre-production context (carried forward from research + proposal)
research_brief: null # full research_brief artifact — available to all downstream stages
selected_concept: null # the approved concept from proposal_packet
production_plan: null # the approved tool/provider plan
approved_budget_usd: null # explicit user-approved spend cap
# Cross-stage tracking
narration_durations: {} # section_id → actual_seconds
total_narration_seconds: 0
total_visual_seconds: 0
style_anchors: {} # consistency tokens carried forward
revision_counts: {} # stage_name → number of revisions
issues_log: [] # all issues found, with resolution status
```
## Execution Protocol
### Phase 0: Initialize
1. Load the pipeline manifest (`animated-explainer.yaml`)
2. Load the playbook (from user selection or default)
3. Set budget from configuration or user input (default: $2.00)
4. Initialize EP_STATE
### Phase 1: Execute Stages Serially
For each stage in order: `research → proposal → script → scene_plan → assets → edit → compose → publish`
**Pre-production stages (research, proposal)** run before any money is spent:
- **research** gathers raw data via web search — zero cost, no tools
- **proposal** presents concepts and costs to the user — zero cost, but contains the **approval gate**
- The pipeline MUST NOT proceed past proposal without `approval.status == "approved"` or `"approved_with_changes"`
After proposal approval, extract and store in EP_STATE:
- `selected_concept` from `proposal_packet.selected_concept` (drives script, scene, visual decisions)
- `production_plan` from `proposal_packet.production_plan` (drives tool selection in assets stage)
- `approved_budget_usd` from `proposal_packet.approval.approved_budget_usd` (overrides default budget)
- `playbook` from `proposal_packet.selected_concept → concept_options[selected].suggested_playbook`
```
EXECUTE_STAGE(stage_name):
1. PREPARE
- Load the director skill for this stage
- Inject EP_STATE as context (prior artifacts, budget remaining, style anchors)
- Inject any EP feedback from previous revision attempts
2. SPAWN DIRECTOR
- The director executes its full process (as defined in its skill MD)
- Director produces an artifact
3. REVIEW (EP performs this, not a separate reviewer)
- Schema validation against artifact schema
- Check review_focus items from pipeline manifest
- Check success_criteria from pipeline manifest
- Cross-check against playbook constraints
- Run EP-SPECIFIC CROSS-STAGE CHECKS (see below)
4. GATE DECISION
If PASS:
- Store artifact in EP_STATE
- Update cumulative tracking (budget, durations, etc.)
- Log: "[stage] PASSED — moving to next stage"
- Continue to next stage
If REVISE:
- Increment revision_counts[stage_name]
- If revision_counts[stage_name] >= 3:
- PASS WITH WARNINGS (never block forever)
- Log unresolved issues
- Else:
- Compose specific feedback for the director
- Re-run SPAWN DIRECTOR with feedback injected
- Re-run REVIEW
If SEND_BACK(target_stage):
- This is the EP's special power: send work BACK to a prior stage
- Only used when a downstream discovery invalidates upstream work
- Example: TTS returns 16s audio for a scene planned at 10s
→ Send back to script director: "Rewrite section 3. Max 25 words."
- Re-execute from target_stage forward (artifacts after target are invalidated)
- Max 1 send-back per stage pair (prevent infinite loops)
```
### Phase 2: Final Quality Assurance
After all 7 stages complete, the EP performs a holistic review:
```
FINAL_QA:
1. PROBE the output video:
- Duration: within ±5% of target?
- Resolution: matches media profile?
- Audio: narration audible throughout? Music balanced?
- File: valid container, reasonable size?
2. A/V SYNC CHECK:
- Compare narration timestamps to visual cut points
- Flag any section where narration plays over the wrong visual
- Tolerance: ±0.5 seconds
3. STYLE CONSISTENCY:
- Review all generated images: do they look like the same video?
- Check color palette adherence
- Check typography consistency
4. BUDGET RECONCILIATION:
- Total actual spend vs. budget
- Log per-stage cost breakdown
5. DECISION:
If all checks pass → APPROVE for publish stage
If issues found → Send back to the specific stage(s) that can fix them
- Audio issues → compose director
- Visual issues → asset director (regenerate) or scene director (replan)
- Duration issues → script director (rewrite)
- Sync issues → edit director (re-cut)
```
## EP-Specific Cross-Stage Checks
These checks use information accumulated across stages — something no individual director can do.
### After RESEARCH stage:
```
CHECK: Research depth
- At least 3 data_points with source URLs?
- At least 3 angles_discovered with grounded_in references?
- At least 5 sources cited?
- If any minimum not met: REVISE research
- Note: Do NOT checkpoint with user — research is informational, not a decision point
```
### After PROPOSAL stage:
```
CHECK: Approval gate (CRITICAL — the entire point of pre-production)
- Is approval.status == "approved" or "approved_with_changes"?
- If "pending" or "rejected": STOP. Present to user and wait.
- If "approved_with_changes": apply modifications to selected_concept before proceeding
- Extract: target_duration_seconds, playbook, budget, tool selections
- Initialize budget from approved_budget_usd (not default)
CHECK: Production feasibility
- Does the production plan reference tools that are actually available?
- Cross-check production_plan.stages[].tools[].available against registry
- If any required tool is unavailable: alert user, offer alternatives
```
### After SCRIPT stage:
```
CHECK: Word count vs. duration target
- Calculate: total_words / 150 = estimated_minutes (at 150 WPM speaking rate)
- If estimated_minutes > target_duration * 1.15:
REVISE script: "Script is {X} words. At 150 WPM, that's {Y} minutes.
Target is {Z} minutes. Cut {N} words."
- If estimated_minutes < target_duration * 0.7:
REVISE script: "Script is too short. Add {N} words of content."
```
### After SCENE_PLAN stage:
```
CHECK: Total scene duration covers full script
- Sum all scene durations
- Compare to script's total duration
- If gaps > 1 second: REVISE scene_plan
- If overlaps: REVISE scene_plan
CHECK: Visual variety
- Count consecutive same-type scenes
- If > 3 consecutive: REVISE scene_plan
CHECK: Asset feasibility
- For each required_asset, verify the tool exists in registry
- If any asset requires a tool that's unavailable:
REVISE scene_plan: "Tool {X} is unavailable. Use {alternative} instead."
```
### After ASSETS stage:
```
CHECK: Narration duration feedback loop (CRITICAL)
- For each TTS audio file, probe actual duration
- Store in EP_STATE.narration_durations
- For each section:
If actual_duration > planned_duration * 1.15:
Option A: SEND_BACK to script director:
"Section {id} narration is {X}s but scene is {Y}s.
Rewrite to max {N} words."
Option B (if within 25% over): Adjust scene_plan durations to fit
- Update EP_STATE.total_narration_seconds
CHECK: Budget gate
- If budget_spent > budget_total * 0.9 and stages remain:
Alert: "90% budget consumed with {N} stages remaining"
Adjust remaining stages to use free/cheap alternatives
CHECK: Style consistency
- Compare image descriptions/styles across all generated images
- Store style_anchors for downstream use
```
### After EDIT stage:
```
CHECK: Timeline completeness
- Verify edit decisions cover 0 to total_duration with no gaps
- Verify all asset references point to existing files
- Verify audio ducking is configured for all narration segments
CHECK: A/V sync pre-validation
- For each cut: narration_start aligns with visual_start (±0.5s)
- For each scene: narration_duration ≤ visual_duration
```
### After COMPOSE stage:
```
CHECK: Output validation
- ffprobe the output: duration, resolution, codec, audio channels
- If duration drift > 5%: investigate which stage caused it
- If audio missing: check audio_mixer configuration
- If resolution wrong: check media profile selection
```
## Feedback Message Templates
When sending work back to a director, use these structured feedback messages:
### To Script Director:
```
EP FEEDBACK — Script Revision Required
Reason: {reason}
Specific issue: {detail}
Constraint: {word_count_limit / duration_target / etc.}
Keep: {what was good about the current script}
Change: {what specifically needs to change}
```
### To Scene Director:
```
EP FEEDBACK — Scene Plan Revision Required
Reason: {reason}
Affected scenes: {scene_ids}
Constraint: {feasibility / variety / duration / etc.}
Available tools: {current tool registry status}
```
### To Asset Director:
```
EP FEEDBACK — Asset Regeneration Required
Reason: {reason}
Affected assets: {asset_ids}
Style anchors: {consistency requirements from prior successful assets}
Budget remaining: ${remaining}
```
### To Compose Director:
```
EP FEEDBACK — Re-render Required
Reason: {reason}
Specific issue: {audio_sync / duration / quality / etc.}
Expected: {what the output should be}
Actual: {what was produced}
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | research | Data depth, source quality, angle diversity | Revise research |
| G2 | proposal | Concept quality, cost accuracy, user approval | Revise proposal OR wait for user |
| G3 | script | Word count vs duration, narrative arc, research integration | Revise script |
| G4 | scene_plan | Coverage, variety, feasibility against production plan | Revise scene_plan |
| G5 | assets | File existence, narration duration, budget, style | Revise assets OR send-back to script |
| G6 | edit | Timeline completeness, A/V pre-sync | Revise edit |
| G7 | compose | Output probe, duration, audio quality | Revise compose OR send-back to edit/assets |
| G8 | publish | Metadata, packaging | Revise publish |
| FINAL | all | Holistic video review | Send-back to specific stage |
## Execution Limits (Anti-Loop Protection)
| Limit | Value | Rationale |
|-------|-------|-----------|
| Max revisions per stage | 3 | Prevent perfectionism loops |
| Max send-backs per stage pair | 1 | Prevent ping-pong between stages |
| Max total send-backs | 3 | Cap total pipeline re-work |
| Max total budget | Configurable (default $2) | Hard stop on spending |
| Max total wall-time | 15 minutes | Timeout for entire pipeline |
After any limit is hit: **proceed with warnings**, never block indefinitely.
## Integration with Existing Skills
The EP doesn't replace any director skill — it wraps them. Each director skill continues to work exactly as documented. The EP adds:
1. **Context injection**: Directors receive EP_STATE with cross-stage information they couldn't access before
2. **Feedback injection**: Directors receive specific revision instructions when sent back
3. **Budget awareness**: Directors receive remaining budget and can adjust tool choices accordingly
4. **Style anchors**: Directors receive consistency tokens from prior stages
## Example EP Run (Abbreviated)
```
[EP] Starting pipeline: animated-explainer v2.0
[EP] Default budget: $2.00 | Target: TBD (set after proposal)
[EP] === STAGE 1: research ===
[EP] Spawning research-director... Topic: "How DNS Works"
[EP] Research director executed 18 web searches.
[EP] Findings: 5 existing videos mapped, 6 data points sourced, 8 audience questions found.
[EP] Top insight: "1.1.1.1 handles 13.5% of queries — most people assume Google dominates."
[EP] G1 PASS — 6 data points, 4 angles discovered, 12 sources cited.
[EP] Budget: $0.00 spent (research is free)
[EP] === STAGE 2: proposal ===
[EP] Spawning proposal-director with research_brief...
[EP] Preflight: ElevenLabs ✓, image_selector ✓, video_selector ✗ (no API keys), music_gen ✓
[EP] 3 concepts presented to user:
[EP] C1: "The 200ms Journey" (data_driven, $0.64)
[EP] C2: "Your ISP Knows Everything" (contrarian, $0.58)
[EP] C3: "The Internet's Phone Book" (analogy, $0.52)
[EP] Awaiting user approval...
[EP] USER SELECTED: C1 with modification: "focus on recursive resolution, skip DoH"
[EP] G2 PASS — Approved with changes. Budget: $0.64 approved.
[EP] Extracted: target=90s, playbook=minimalist-diagram, budget=$0.64
[EP] === STAGE 3: script ===
[EP] Spawning script-director with proposal_packet + research_brief...
[EP] Script director produced script. Reviewing...
[EP] Word count: 210 words → ~84s at 150 WPM. Target: 90s.
[EP] Script references 3 data points from research. ✓
[EP] G3 PASS — Within duration, research integrated.
[EP] === STAGE 4: scene_plan ===
[EP] Spawning scene-director with script + proposal_packet...
[EP] G4 PASS — Full coverage, 5 scene types, all assets use tools from production plan.
[EP] === STAGE 5: assets ===
[EP] Spawning asset-director with scene_plan + script + production_plan...
[EP] Asset director generated 14 assets. Reviewing...
[EP] Narration check: Section 3 is 8.2s audio for 6s scene.
[EP] → Adjusting scene_plan: extending scene-3 to 9s (within tolerance)
[EP] Budget: $0.52 spent, $0.12 remaining
[EP] Style check: All images use consistent palette. ✓
[EP] G5 PASS (with scene duration adjustment)
[EP] === STAGE 6: edit ===
[EP] Spawning edit-director with adjusted scene_plan + asset_manifest...
[EP] G6 PASS — Timeline complete, audio ducking configured.
[EP] === STAGE 7: compose ===
[EP] Spawning compose-director with edit_decisions + asset_manifest...
[EP] Output probe: 88.7s (target 90s, within 5%). Resolution: 1920x1080. Audio: stereo. ✓
[EP] G7 PASS
[EP] === STAGE 8: publish ===
[EP] Spawning publish-director with render_report + proposal_packet...
[EP] G8 PASS — SEO metadata complete, chapters present, research citations included.
[EP] === FINAL QA ===
[EP] Duration: 88.7s ✓ | A/V sync: within tolerance ✓ | Style: consistent ✓
[EP] Budget: $0.52 / $0.64 approved ✓
[EP] PIPELINE COMPLETE — 0 revisions, 0 send-backs
[EP] Output: renders/output.mp4
```
## Common Pitfalls
- **Over-revising**: The EP should be pragmatic. A "pretty good" script that's within duration is better than a "perfect" script after 5 rounds. Use the limits.
- **Ignoring budget**: Don't let early stages consume all budget. Reserve at least 30% for assets + compose.
- **Sending back too eagerly**: Minor issues (±10% duration) should be handled by adjusting downstream, not re-running upstream. Only send back for structural problems.
- **Not probing outputs**: Always ffprobe the final video. Never trust metadata alone.
- **Losing style context**: The EP must carry style anchors forward. If image 1 uses a specific palette, image 5 must match. Pass this explicitly to the asset director.
skills/pipelines/explainer/idea-director.md
+183
View File
@@ -0,0 +1,183 @@
# Idea Director — Explainer Pipeline
## When to Use
You are the Idea Explorer for a generated explainer video. The user has provided a **topic or idea** (not raw footage). Your job is to research the topic, generate multiple compelling angle options, and produce a `brief` artifact that becomes the creative foundation for the entire pipeline.
This is the most important stage — a weak brief produces a weak video regardless of how good the tools are. Invest time here.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/brief.schema.json` | Artifact validation |
| Playbooks | `styles/*.yaml` | Visual/audio style options |
| Skills | `skills/meta/skill-creator.md` | If you encounter unfamiliar domain |
## Process
### Step 1: Understand the Request
Before doing anything, clarify the user's intent:
- **Topic**: What is the core subject? (e.g., "vector databases", "how HTTPS works", "why the sky is blue")
- **Audience**: Who is this for? (developers, general public, students, executives)
- **Platform**: Where will this be published? (YouTube, TikTok, Instagram, LinkedIn) — this constrains duration and style
- **Duration**: Target length. Defaults by platform: TikTok 30-60s, Instagram Reels 60-90s, YouTube 60-180s, LinkedIn 60-120s
- **Tone**: Casual, professional, educational, provocative, playful
If the user's request is vague (e.g., "make a video about AI"), ask targeted questions. Never guess when you can ask.
### Step 2: Research the Topic
**This step is mandatory.** Do not skip it. The research dossier is what separates a generic explainer from a compelling one.
Use web search to investigate:
1. **Existing content landscape**: Search YouTube and blogs for existing explainer videos on this topic. What angles have been covered? What's missing? What's been done to death?
2. **Trending discussions**: Search Reddit, X/Twitter, Hacker News, Stack Overflow for what people are currently asking or debating about this topic. What misconceptions exist? What surprises people?
3. **Key facts and data**: Find 3-5 surprising statistics, quotes, or facts that could anchor the video. Cite your sources.
4. **Visual inspiration**: How have the best creators visualized this concept? What analogies work? What diagrams are commonly used?
5. **Audience knowledge gaps**: What do most people get wrong about this topic? Where does the "aha moment" live?
**Output of this step**: A mental research dossier. You don't need to write it all down, but reference specific findings in your angle options.
### Step 3: Generate Angle Options
Generate **at least 3 genuinely different angles**. Not rewordings — structurally different approaches to the same topic.
For each angle, specify:
| Field | What | Quality Bar |
|-------|------|-------------|
| `name` | Short title (5-8 words) | Specific, not generic. "Why Vector Search Beats SQL LIKE" not "About Vector Databases" |
| `hook` | Opening line/question (under 15 words) | Must create curiosity or surprise in one sentence |
| `narrative_structure` | How the story unfolds | One of: analogy, problem-solution, journey, debate, myth-busting, timeline, comparison |
| `visual_approach` | Primary visual style | e.g., "animated diagrams with vector space visualizations" |
| `suggested_playbook` | Best-matching style playbook | Reference available playbooks in `styles/` |
| `target_audience` | Who this angle serves best | Specific: "mid-level developers evaluating databases" not "developers" |
| `why_this_works` | Rationale | Reference your research — why is this angle compelling right now? |
**Angle diversity checklist:**
- [ ] At least one angle is technical/detailed
- [ ] At least one angle is intuitive/accessible (uses analogy or story)
- [ ] At least one angle is provocative/surprising (challenges assumptions)
- [ ] No two angles use the same narrative structure
- [ ] Each angle suggests a different visual approach
### Step 4: Present to User and Select
Present all angle options clearly. Let the user:
- Select one as-is
- Ask you to combine elements from multiple angles
- Describe a custom direction entirely
If the user provides a custom direction, use it — but apply the research and quality bar from Steps 2-3.
### Step 5: Assemble the Brief
Build the `brief` artifact with all required and relevant optional fields:
```json
{
"version": "1.0",
"title": "...",
"hook": "...",
"key_points": ["...", "...", "..."],
"core_message": "...",
"cta": "...",
"tone": "...",
"style": "...",
"target_audience": "...",
"target_platform": "youtube|instagram|tiktok|linkedin|generic",
"target_duration_seconds": 60,
"reference_material": ["..."],
"angle_options": [
{"name": "...", "description": "..."},
{"name": "...", "description": "..."},
{"name": "...", "description": "..."}
],
"selected_angle": "..."
}
```
**Field quality bar:**
| Field | Excellent | Mediocre |
|-------|-----------|----------|
| `title` | "How Vector Databases Find Your Data in 1ms" | "Vector Databases Explained" |
| `hook` | "Your database searches every single row. What if it didn't have to?" | "Today we'll learn about vector databases" |
| `key_points` | Concrete, specific claims the video will prove | Vague topics like "how it works" |
| `core_message` | One sentence the viewer should remember tomorrow | Absent or too broad |
| `cta` | Actionable and relevant: "Try building a similarity search with 10 lines of Python" | Generic: "Like and subscribe" |
| `tone` | Matches audience and platform | Mismatched (e.g., corporate tone on TikTok) |
### Step 6: Self-Evaluate
Before submitting, score your brief on this rubric (1-5 each):
| Criterion | Question |
|-----------|----------|
| **Hook strength** | Would someone stop scrolling for this? Does it create an information gap? |
| **Specificity** | Are key_points concrete claims, not vague topics? |
| **Research depth** | Does the brief reference real data, trends, or insights from Step 2? |
| **Audience fit** | Is the tone, complexity, and duration right for the target audience? |
| **Playbook match** | Does the selected style genuinely fit the content? |
| **Uniqueness** | Does this angle offer something the existing content landscape doesn't? |
If any dimension scores below 3, iterate before submitting. The reviewer will check the same criteria.
### Step 7: Submit
Call `handle_explainer_idea(state, {"brief": brief_json})` to validate and persist.
## Playbook Selection Guide
| Content Type | Recommended Playbooks | Why |
|--------------|----------------------|-----|
| Technical architecture | `minimalist-diagram` | Clean diagrams, whiteboard feel |
| Business/SaaS concept | `clean-professional` | Polished, trustworthy |
| Social media / quick explainer | `flat-motion-graphics` | Eye-catching, data-driven |
| Storytelling / narrative | Warm playbooks (Ghibli, Watercolor) | Emotional connection |
| Developer tutorial | `minimalist-diagram` or custom | Focus on code/diagrams |
If no existing playbook fits, describe the desired style in `brief.style` and the pipeline can create a custom playbook later.
## Common Pitfalls
- **Skipping research**: The #1 failure mode. Without research, angles are generic and hooks are weak.
- **Reworded angles**: Three variations of "explain how X works" are not three angles. Change the narrative structure.
- **Wrong duration for platform**: A 3-minute explainer doesn't work on TikTok. A 30-second video can't explain Kubernetes.
- **Ignoring the audience**: A video for CTOs needs different framing than one for junior developers, even on the same topic.
- **Vague key_points**: "How vector databases work" is a topic, not a key point. "Vector databases use high-dimensional math to find similar items in milliseconds" is a key point.
## Examples
### Good Angle Set (Topic: "How HTTPS Works")
**Angle 1: The Spy Analogy**
- Hook: "Every time you visit a website, you're having a secret conversation. Here's how."
- Structure: Analogy (spy/espionage metaphor)
- Visual: Animated characters passing secret messages
- Playbook: `flat-motion-graphics`
- Audience: General public, non-technical
**Angle 2: The Handshake Deep Dive**
- Hook: "The TLS handshake takes 100 milliseconds and involves 4 messages. Here's what each one does."
- Structure: Timeline/process walkthrough
- Visual: Technical diagram with packet animations
- Playbook: `minimalist-diagram`
- Audience: CS students, junior developers
**Angle 3: The Myth Buster**
- Hook: "The padlock icon doesn't mean what you think it means."
- Structure: Myth-busting (challenge assumption, then reveal truth)
- Visual: Split-screen before/after misconception
- Playbook: `clean-professional`
- Audience: Business professionals, security-aware users
### Bad Angle Set (same topic)
- Angle 1: "HTTPS Explained" — generic, no hook
- Angle 2: "How HTTPS Works" — same thing, reworded
- Angle 3: "Understanding HTTPS" — still the same, no structural difference
skills/pipelines/explainer/proposal-director.md
+343
View File
@@ -0,0 +1,343 @@
# Proposal Director — Explainer Pipeline
## When to Use
You are the **Proposal Director** for a generated explainer video. You sit between the Research Director and the Script Director. You receive a `research_brief` full of raw findings and transform it into a concrete, reviewable proposal that the user approves before any money is spent.
**This is the approval gate.** Nothing downstream runs until the user says "go." Your job is to make that decision easy by presenting clear options, honest costs, and explicit tradeoffs.
Think of yourself as a creative agency pitching to a client: you present concepts backed by research, show what it'll cost, explain the tradeoffs, and let the client choose.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/proposal_packet.schema.json` | Artifact validation |
| Prior artifact | `research_brief` from Research Director | Raw research findings |
| Pipeline manifest | `pipeline_defs/animated-explainer.yaml` | Stage and tool definitions |
| Tool registry | `support_envelope()` output | What's actually available right now |
| Cost tracker | `tools/cost_tracker.py` | Cost estimation data |
| Style playbooks | `styles/*.yaml` | Available visual styles |
| User input | Topic, any preferences expressed | Creative direction |
## Process
### Step 1: Absorb the Research
Read the `research_brief` thoroughly. Extract:
- **`research_summary`** — read this first. This is the researcher's single most important finding.
- **`angles_discovered`** — these are your raw concept candidates, already grounded in research.
- **`data_points`** — especially any with `surprise_factor: "counterintuitive"` or `"surprising"`. These become hooks.
- **`audience_insights.misconceptions`** — myth-busting is a proven engagement pattern.
- **`landscape.underserved_gaps`** — this is where the opportunity lives. Our video should fill a gap, not repeat what exists.
- **`trending`** — if there's a timeliness window, factor it into concept urgency.
### Step 2: Run Preflight
Before designing concepts, know what tools are available:
```bash
python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.support_envelope(), indent=2))"
```
Also check the capability catalog:
```bash
python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.capability_catalog(), indent=2))"
```
Record:
- Which TTS providers are available — run `registry.get_by_capability("tts")` and check status
- Which video generation providers are available — run `registry.get_by_capability("video_generation")` and check status
- Which enhancement tools are available
- Image generation status — run `registry.get_by_capability("image_generation")` and check status
- **Remotion render engine status** — check `video_compose.get_info()["render_engines"]["remotion"]`. If `true`, Remotion is available for animated text cards, stat cards, charts, spring-physics transitions, and image-to-video rendering. This is a major quality upgrade over Ken Burns pan-and-zoom.
This directly affects what you can promise in the production plan. **Do not propose a concept that requires tools you don't have.**
**Setup offers:** If critical tools are UNAVAILABLE but fixable with a simple configuration, read each tool's `install_instructions` from the registry and offer the user setup help before designing around the limitation. See AGENT_GUIDE.md "Provider Menu" protocol for the approach. Group related tools that share the same env var dependency.
### Step 3: Design Concept Options
Build **at least 3 genuinely different concepts.** Start from the `angles_discovered` in the research brief, but elevate them into full production concepts.
For each concept, specify all fields in the `proposal_packet.concept_options` schema:
#### 3a: Title and Hook
The title and hook are the most important two lines. They determine whether the user gets excited or scrolls past.
**Hook construction patterns** (use the research to fill these):
| Pattern | Template | When to Use |
|---------|----------|-------------|
| **Surprising stat** | "[Counterintuitive number]. Here's why." | When you have a data point with high surprise factor |
| **Misconception flip** | "You've been told [myth]. The truth is [reality]." | When audience_insights.misconceptions has a strong entry |
| **Recency** | "[Thing] just changed everything about [topic]. Here's what happened." | When trending.recent_developments has a timely event |
| **Question** | "Why does [thing everyone experiences] actually happen?" | When audience_insights.common_questions has a strong entry |
| **Contrast** | "[Thing A] takes [big number]. [Thing B] takes [small number]. Here's the trick." | When data_points has comparison data |
| **Insider knowledge** | "The thing about [topic] that nobody explains." | When landscape.underserved_gaps reveals a strong gap |
**Rules:**
- Hook must be under 20 words
- Hook must create an information gap — the viewer needs to watch to close it
- Hook must be grounded in a specific research finding (cite it in `grounded_in`)
- Never use: "In this video we'll...", "Hey guys...", "Let me explain..."
#### 3b: Narrative Structure
Choose the structure that best fits the research findings:
| Structure | Best When | Research Signal |
|-----------|-----------|-----------------|
| `myth_busting` | Strong misconceptions found | `audience_insights.misconceptions` has 2+ entries |
| `problem_solution` | Clear pain points | `audience_insights.pain_points` is rich |
| `data_narrative` | Strong surprising data | Multiple data_points with high surprise_factor |
| `comparison` | Two approaches to compare | Data_points contain comparative data |
| `timeline` | Topic has evolution/history | Landscape shows topic changing over time |
| `journey` | Complex topic needs progressive reveal | `audience_insights.knowledge_level` shows big gaps |
| `analogy` | Abstract topic needs grounding | Audience is non-technical |
| `debate` | Community is divided | `trending.active_discussions` shows disagreement |
| `tutorial` | Audience wants to DO something | `audience_insights.common_questions` are how-to |
| `story` | Human interest angle exists | Expert voices or real-world cases available |
#### 3c: Visual Approach and Playbook
Match the visual approach to the content. **Check Remotion availability first** — if `video_compose` reports `render_engines.remotion: true`, the Remotion render path unlocks animated text cards, stat cards, charts, spring-physics transitions, and component-based scenes. This should change your visual design:
| Content Type | Visual Approach (Remotion available) | Visual Approach (FFmpeg only) | Playbook |
|--------------|--------------------------------------|-------------------------------|----------|
| Technical architecture/process | Remotion animated diagrams, flowcharts with spring transitions | Static diagrams with Ken Burns motion | `minimalist-diagram` |
| Data-heavy narrative | Remotion stat cards, animated charts, comparison cards | Static image cards with zoom-in | `flat-motion-graphics` |
| Professional/business | Remotion text cards with clean typography | Image-based title cards | `clean-professional` |
| Storytelling/analogy | Remotion scenes with animated character cards | Image sequence with pan | Warm/narrative playbook |
| Tutorial/how-to | Screen captures + Remotion callout overlays | Screen captures + static overlays | `minimalist-diagram` |
**Remotion components available** (when Remotion engine is active):
- `text_card` — animated text with spring entrance
- `stat_card` — number + label with count-up animation
- `callout` — highlighted explanation box
- `comparison` — side-by-side with animated reveal
- `progress` — animated progress bar
- `chart` — bar, line, pie charts with animated data entry
- `kpi_grid` — multi-stat dashboard layout
**Important:** When Remotion is available and the playbook is `flat-motion-graphics`, **always design for Remotion component scenes** rather than static AI-generated images with Ken Burns pan. This is the difference between a professional motion graphics video and a slideshow.
#### 3d: Duration and Platform
Set realistic duration based on platform and content depth:
| Platform | Duration Range | Word Budget (150 WPM) |
|----------|---------------|----------------------|
| TikTok | 30-60s | 65-150 words |
| Instagram Reels | 30-90s | 65-225 words |
| YouTube Shorts | 30-60s | 65-150 words |
| YouTube | 60-180s | 150-450 words |
| LinkedIn | 60-120s | 150-300 words |
#### 3e: Concept Diversity Check
Before finalizing, verify diversity:
- [ ] No two concepts use the same narrative structure
- [ ] No two concepts use the same hook pattern
- [ ] At least one concept targets a different audience segment
- [ ] At least one concept leverages the most surprising data point
- [ ] At least one concept addresses the biggest content gap found
- [ ] Each concept's `grounded_in` references different research findings
### Step 4: Present Concepts and Get Selection
Present all concepts clearly to the user. For each concept, show:
1. **Title** and **hook** — the creative pitch
2. **Why this works** — the research backing, in one sentence
3. **What it'll look like** — visual approach in plain language
4. **Duration** — how long the video will be
Let the user:
- Select one as-is
- Combine elements from multiple concepts
- Request modifications
- Describe a completely different direction (in which case, use the research to strengthen it)
Record the selection in `selected_concept` with rationale and any modifications.
### Step 5: Build the Production Plan
For the selected concept, design the stage-by-stage production plan.
For each stage in the pipeline manifest (`animated-explainer.yaml`), specify:
1. **Which tools will be used** — specific provider names, not just selectors
2. **Whether each tool is available** — from the preflight check
3. **Estimated cost per tool** — from the tool's cost metadata
4. **Why this provider** — explain the choice ("ElevenLabs for narration because voice quality is critical for this topic" or "Piper TTS because running local-only and free")
5. **Fallback if unavailable** — what happens if the primary tool is down
**Tool selection rationale must be honest:**
- If using a free/local tool because the cloud tool is unavailable, say so
- If using a cloud tool when a local alternative exists, explain the quality tradeoff
- If a capability is entirely missing, say what the video will lack
#### Quality/Cost Tradeoff Matrix
For each meaningful choice, present the tradeoff:
```
TRADEOFF: TTS Provider
├── Premium: ElevenLabs ($0.18-0.30) — natural voice, emotional delivery
├── Standard: OpenAI TTS ($0.05-0.15) — good quality, less expressive
└── Free: Piper local ($0.00) — robotic but works offline
TRADEOFF: Visual Assets
├── Premium: AI video clips ($0.10-0.50/clip) — motion, dynamic
├── Standard: AI images ($0.02-0.04/image) — static, reliable
└── Free: Diagrams/code ($0.00) — text-based, technical feel
TRADEOFF: Render Path (check video_compose render_engines)
├── Remotion ($0.00, local): Animated text cards, stat cards, charts,
│ spring-physics transitions, component-based scenes. Professional
│ motion graphics feel. Requires Node.js.
└── FFmpeg ($0.00, local): Ken Burns pan-and-zoom on images, video
concat. Functional but less engaging for explainer content.
```
**If Remotion is available:** Design the scene plan around Remotion component types (text_card, stat_card, chart, etc.) rather than generating AI images for every scene. This is both cheaper (fewer image gen calls) and higher quality (animated motion graphics vs. static images with pan).
Also present **alternative production paths** — complete packages at different price points:
| Path | Quality | Cost | What Changes |
|------|---------|------|-------------|
| Premium | Best TTS + video clips + music | ~$1.50-2.50 | Full production value |
| Standard | Good TTS + images + music | ~$0.50-1.00 | Static visuals, still professional |
| Budget | Local TTS + images | ~$0.05-0.15 | Robotic voice, image-only |
| Free | Local TTS + diagrams | $0.00 | Functional but minimal |
### Step 6: Build the Cost Estimate
Itemize every paid operation:
```
COST ESTIMATE
├── TTS Narration: tts_selector × 1 run (~150 words) $0.18
├── Image Generation: image_selector × 6 scenes $0.24
├── Music: music_gen × 1 track (30s) $0.10
├── Video Generation: video_selector × 2 clips (optional) $0.00 (local)
├── Audio Enhancement: audio_enhance × 1 pass $0.00 (local)
└── TOTAL ESTIMATED $0.52
Budget cap: $2.00
Verdict: within_budget ✓
Headroom: $1.48 for revisions/regeneration
```
**Rules:**
- Always show per-item costs, not just the total
- Always show the budget cap comparison
- If over budget, list specific savings options (e.g., "Switch to a cheaper TTS provider: saves $0.18" — check each provider's `estimate_cost` via the registry)
- Include headroom note — some budget should remain for revisions
### Step 7: Assemble the Approval Gate
The approval section is where the user commits. Present it as a clear decision point:
```
────────────────────────────────────────
PROPOSAL READY FOR APPROVAL
Concept: [selected title]
Duration: [X] seconds for [platform]
Estimated cost: $[X.XX] of $[budget] budget
Production path: [premium/standard/budget/free]
Proceed? (approve / approve with changes / reject)
────────────────────────────────────────
```
Set `approval.status: "pending"` in the artifact. The EP or the user updates this to `approved` before the pipeline continues.
**Critical rule:** The pipeline MUST NOT proceed past this stage without explicit approval. This is the last free exit. Everything after this costs money and time.
### Step 8: Submit
Validate the `proposal_packet` artifact against `schemas/artifacts/proposal_packet.schema.json` and submit.
## How This Connects Downstream
| Downstream Stage | What It Takes From proposal_packet |
|------------------|------------------------------------|
| Script Director | `selected_concept` (title, hook, key_points, core_message, tone, narrative_structure) + research_brief data points |
| Scene Director | `selected_concept.visual_approach` + `production_plan.playbook` |
| Asset Director | `production_plan.stages[assets].tools` — knows exactly which providers to use |
| Executive Producer | `cost_estimate` — initializes budget tracking |
| All stages | `approval.approved_budget_usd` — hard spending cap |
The `selected_concept` in the proposal_packet effectively replaces what the old `brief` artifact used to be — but it's grounded in research and comes with an explicit production plan attached.
## Common Pitfalls
- **Presenting concepts without research grounding**: Every concept's `why_this_works` must reference specific research findings. "This is a popular topic" is not grounding. "Cloudflare Radar shows 13.5% of DNS queries hit 1.1.1.1, which contradicts the common belief that Google DNS dominates" is grounding.
- **Hiding costs**: Be transparent. If ElevenLabs will cost $0.30, say $0.30. Don't round down or omit items. The user trusts you more when you're honest.
- **Over-promising tool availability**: If the preflight shows only Piper TTS available, don't design a concept that depends on expressive voice acting. Design around constraints.
- **Three versions of the same concept**: "Kubernetes Explained", "Understanding Kubernetes", and "Kubernetes Guide" are not three concepts. They're one concept with three titles. Structural diversity means different narrative structures, different hooks, different audiences.
- **Skipping the approval gate**: This is the whole point of pre-production. No shortcuts.
- **Not showing alternatives**: The user should always see at least 2 production paths at different price points. Let them make an informed choice.
## Example: Full Proposal Flow
### Input: research_brief on "How DNS Works"
**Concept 1: "The 200ms Journey" (data_driven)**
- Hook: "Every website you visit starts with a 200-millisecond treasure hunt across the internet."
- Structure: journey — follow a DNS query step by step
- Visual: minimalist-diagram, animated packet flow
- Duration: 90s (YouTube)
- Grounded in: recursive resolution timing data, audience gap about multi-step process
- Why it works: Most viewers think DNS is instant and singular. Showing the real journey is the aha moment.
**Concept 2: "Your ISP Knows Everything" (contrarian)**
- Hook: "Your internet provider logs every website you visit. Here's the 40-year-old system that makes it possible."
- Structure: myth_busting — challenge "private browsing = private" belief
- Visual: clean-professional, privacy-focused with dark tones
- Duration: 75s (YouTube)
- Grounded in: DNS privacy misconception (audience research), DoH trending signal
- Why it works: Privacy is emotionally charged. The misconception that HTTPS = full privacy is widespread.
**Concept 3: "The Internet's Phone Book" (analogy)**
- Hook: "DNS is a phone book designed in 1983 that somehow still runs the modern internet."
- Structure: analogy — phone book metaphor through historical evolution
- Visual: flat-motion-graphics, retro-to-modern visual timeline
- Duration: 60s (LinkedIn)
- Grounded in: audience knowledge gap about DNS age, landscape gap (no historical angle found)
- Why it works: Simplest on-ramp for non-technical audience. The "still works after 40 years" angle is inherently surprising.
**Production plan (for selected concept 1, Remotion available):**
```
script → no tools, no cost
scene → no tools, no cost — design 4 Remotion component scenes + 4 AI image scenes
assets → tts_selector ($0.22), image_selector × 4 ($0.16), music_gen ($0.10)
edit → no tools, no cost
compose → video_compose/Remotion render (free) — animated text cards, stat cards,
spring transitions, image scenes with animation. NOT Ken Burns.
publish → no tools, no cost
TOTAL: $0.48 of $2.00 budget (saved $0.16 by using Remotion components instead of
generating images for text/data scenes)
```
**Production plan (for selected concept 1, FFmpeg only):**
```
script → no tools, no cost
scene → no tools, no cost
assets → tts_selector ($0.22), image_selector × 8 ($0.32), music_gen ($0.10)
edit → no tools, no cost
compose → video_compose/FFmpeg (free) — Ken Burns pan-and-zoom on images
publish → no tools, no cost
TOTAL: $0.64 of $2.00 budget
```
**Alternative paths:**
- Premium (Remotion): Best available TTS + 4 AI images + 4 Remotion animated scenes = $0.48
- Standard: Mid-tier TTS + images = $0.40
- Free: Local TTS + Remotion component scenes only = $0.00 (no images, pure motion graphics)
skills/pipelines/explainer/publish-director.md
+152
View File
@@ -0,0 +1,152 @@
# Publish Director — Explainer Pipeline
## When to Use
You are the Publisher for a generated explainer video. You have a `render_report` with the final video file. Your job is to prepare the video for distribution: generate SEO metadata, create thumbnails, package exports, and log the publish event.
This is where a great video reaches its audience. Without proper metadata and packaging, even the best content gets buried.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/publish_log.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["compose"]["render_report"]`, `state.artifacts["idea"]["brief"]` | Video file and original brief |
| Playbook | Active style playbook | Visual style for thumbnail |
## Process
### Step 1: Gather Context
Collect everything needed for metadata:
- **Brief**: title, hook, key points, target platform, tone
- **Render report**: output path, duration, resolution
- **Script**: section summaries for description/chapters
### Step 2: Generate SEO Metadata
**Title** (max 60 characters for YouTube):
- Include the primary keyword from the brief
- Lead with a hook or number
- Avoid clickbait but be compelling
- Examples: "Vector Databases Explained in 60 Seconds" > "About Vector Databases"
**Description** (first 150 chars are critical — shown in search):
- Opening line: restate the hook with the main value proposition
- Body: key topics covered, with relevant keywords naturally included
- Chapters: timestamp markers for each major section (from script sections)
- Call to action: subscribe/like/follow
- Links: relevant resources mentioned in the video
**Tags/Keywords** (platform-dependent):
- 5-10 specific tags derived from brief's key_points
- Mix broad and specific: "machine learning" + "vector database tutorial"
- Include the topic, format ("explainer"), and related terms
**Hashtags** (for social platforms):
- 3-5 relevant hashtags
- Mix trending and niche
### Step 3: Generate Thumbnail Concept
Describe a thumbnail that:
1. Uses the playbook's visual style
2. Features the video's core concept visually
3. Includes 3-5 words of text (the hook or key stat)
4. Has high contrast and is readable at small sizes
5. Uses the playbook's accent colors for text
```json
{
"thumbnail": {
"concept": "Split screen: left side shows slow SQL query (red X), right shows fast vector search (green check). Large text: '100x FASTER'",
"text_overlay": "100x FASTER",
"style_notes": "Use playbook accent colors, bold Inter font, dark background"
}
}
```
*Note: Actual thumbnail generation happens via image_selector if available, otherwise it's a concept for manual creation.*
### Step 4: Create Chapter Markers
From the script sections, generate YouTube-style chapters:
```
0:00 - Introduction
0:15 - What are Vector Databases?
0:45 - How Embeddings Work
1:20 - The Search Algorithm
1:55 - Real-World Examples
2:30 - When to Use Vector DBs
```
Each chapter maps to a script section's `start_seconds`.
### Step 5: Package Export
Create the export directory structure:
```
exports/
<project_name>/
video/
output.mp4 # Final rendered video
metadata/
metadata.json # All SEO metadata
chapters.txt # Chapter markers
description.txt # Ready-to-paste description
tags.txt # One tag per line
thumbnails/
concept.json # Thumbnail concept (or generated image)
```
### Step 6: Build Publish Log
```json
{
"version": "1.0",
"entries": [
{
"platform": "youtube",
"status": "draft",
"timestamp": "2024-01-15T10:30:00Z",
"metadata": {
"title": "Vector Databases Explained in 60 Seconds",
"description_length": 450,
"tags_count": 8,
"chapters_count": 6,
"thumbnail_ready": false
},
"export_path": "exports/vector-db-explainer/",
"video_path": "renders/output.mp4"
}
]
}
```
### Step 7: Self-Evaluate
Score (1-5):
| Criterion | Question |
|-----------|----------|
| **SEO quality** | Would this title and description rank well for the topic? |
| **Description completeness** | Does the description include chapters, CTA, and keywords? |
| **Thumbnail concept** | Would this thumbnail stand out in a feed? |
| **Export package** | Is everything a creator needs in the export directory? |
| **Platform fit** | Is metadata tailored to the target platform? |
If any dimension scores below 3, revise.
### Step 8: Submit
Validate the publish_log against the schema and persist via checkpoint.
## Common Pitfalls
- **Generic titles**: "Video About X" loses to "X Explained in 60 Seconds" every time. Be specific and compelling.
- **No chapters**: YouTube rewards videos with chapters. Always include them.
- **Description keyword stuffing**: Write for humans first, search engines second. Natural language with keywords woven in.
- **Forgetting the CTA**: Every description should end with a call to action.
- **Wrong platform format**: YouTube descriptions differ from TikTok captions. Tailor to the target platform.
skills/pipelines/explainer/research-director.md
+311
View File
@@ -0,0 +1,311 @@
# Research Director — Explainer Pipeline
## When to Use
You are the **Research Director** for a generated explainer video. You are the first stage in the pipeline — before any creative decisions, before any script, before any money is spent. Your job is to **deeply research the topic** using web search and produce a `research_brief` artifact that grounds the entire video in real data, real trends, and real audience insights.
This stage is what separates an OpenMontage video from generic AI slop. Without research, the agent produces vague platitudes. With research, it produces content that has authority, specificity, and timeliness.
**You do NOT make creative decisions.** You gather raw material. The Proposal Director downstream will use your findings to craft concept options.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/research_brief.schema.json` | Artifact validation |
| User input | Topic, audience hint, platform hint | Research scope |
| Tools | Web search, web fetch | Research execution |
## Process
### Step 1: Scope the Research
Before searching anything, establish boundaries:
- **Topic**: What is the core subject? Extract from user input.
- **Audience hint**: Did the user mention who this is for? (developers, general public, executives, students)
- **Platform hint**: Did the user mention where this will go? (YouTube, TikTok, LinkedIn)
- **Depth**: Is this a well-known topic (HTTPS, React) or niche (vector clock CRDTs, QUIC protocol)?
If the user's request is a single phrase like "make a video about kubernetes," that's fine — you have enough to research. Do NOT ask clarifying questions at this stage. Research first, clarify later (in the Proposal stage).
### Step 2: Content Landscape Scan
**Goal:** Understand what already exists so we can find gaps.
Execute these searches in parallel:
```
SEARCH BATCH 1 — Landscape (run all in parallel)
Q1: "[topic] explained" site:youtube.com
→ Find: Top existing explainer videos. Note titles, view counts, angles used.
Q2: "[topic]" (guide OR tutorial OR explained OR breakdown) -site:youtube.com
→ Find: Blog posts and articles covering this topic.
Q3: "[topic] [current month] [current year]"
→ Find: The freshest content. What's being published RIGHT NOW?
Q4: "best [topic category] [current year]"
→ Find: Listicles and comparisons — reveals the competitive landscape.
```
**Parse results for:**
- Which angles have been done to death (saturated)
- Which questions remain unanswered (gaps)
- What the top-performing content looks like (benchmarks)
- When the most recent quality content was published (freshness)
Record at least 3 entries in `landscape.existing_content` with specific titles, sources, and gap analysis.
### Step 3: Trending Pulse
**Goal:** Find what's happening RIGHT NOW — news, debates, controversies, launches.
```
SEARCH BATCH 2 — Trending (run all in parallel)
Q5: "[topic]" (announcement OR launch OR update OR controversy) after:[current year]-01-01
→ Find: Recent events that make this topic timely.
Q6: "[topic]" site:reddit.com after:[6 months ago]
→ Find: Active community discussions, pain points, hot takes.
Q7: "[topic]" site:news.ycombinator.com
→ Find: Tech-literate opinions, contrarian takes, deeper analysis.
Q8: "why is [topic]" (trending OR popular OR important OR everywhere) [current year]
→ Find: Meta-commentary on why people care about this right now.
```
**Parse results for:**
- Recent developments that could be the hook ("X just happened, here's what it means")
- Active debates where people disagree (debate = engagement)
- Sentiment — is the community excited, frustrated, confused, divided?
- Timeliness window — is this a "publish this week" moment or evergreen?
If no trending signal exists, that's fine — note `timeliness_window: "evergreen"` and move on. Not every topic has a news hook, and that's okay.
### Step 4: Data and Evidence Gathering
**Goal:** Find specific, citable facts that will anchor the script.
```
SEARCH BATCH 3 — Data (run all in parallel)
Q9: "[topic]" statistics [current year]
→ Find: Hard numbers — market size, adoption rates, performance benchmarks.
Q10: "[topic]" (study OR research OR survey OR report) [current year - 1] OR [current year]
→ Find: Academic or industry research with credible methodology.
Q11: "[topic]" "according to" (report OR study OR survey)
→ Find: Cited claims with named sources.
Q12: "[topic]" "surprisingly" OR "counterintuitively" OR "most people don't know"
→ Find: Surprising facts — these become hooks and retention anchors.
Q13: "[topic]" (comparison OR benchmark OR "vs") data
→ Find: Comparative data that can become visual stat cards.
```
**For each data point found, record:**
- The specific claim (not vague — "73% of developers use X" not "most developers use X")
- Source URL and source name
- Credibility rating: `primary_source` (original research), `secondary_source` (reporting on research), `anecdotal` (blog post, opinion)
- Surprise factor: would the target audience find this expected or counterintuitive?
- How it could be used: `hook`, `stat_card`, `script_anchor`, `closing_punch`
**Minimum: 3 data points. Target: 5-8.** If the topic is data-poor (e.g., philosophical or creative), find expert quotes instead.
### Step 5: Audience Mining
**Goal:** Understand what real people ask, believe, and get wrong about this topic.
```
SEARCH BATCH 4 — Audience (run all in parallel)
Q14: "[topic]" site:reddit.com "help" OR "confused" OR "why does" OR "ELI5"
→ Find: Real questions from real people struggling with this topic.
Q15: "[topic]" site:quora.com OR site:stackoverflow.com
→ Find: Structured Q&A — what do beginners ask?
Q16: "why is [topic] so" (hard OR confusing OR expensive OR slow OR popular)
→ Find: Pain points and frustrations.
Q17: "[topic]" "common mistakes" OR "myths" OR "misconceptions" OR "wrong about"
→ Find: What people get wrong — myth-busting is powerful engagement.
Q18: "[topic]" "wish I knew" OR "before you start" OR "nobody tells you"
→ Find: Insider knowledge that feels valuable.
```
**Parse results for:**
- Top 5+ real questions (not generated — sourced from actual forum posts)
- Common misconceptions with the real answer (myth vs reality)
- Knowledge level of the target audience (what they already know, what's new)
- Pain points and frustrations
### Step 6: Expert Voices (Optional but High-Value)
**Goal:** Find named experts and their positions — adds authority.
```
SEARCH BATCH 5 — Experts (run if topic has known figures)
Q19: "[topic]" (creator OR inventor OR pioneer OR expert) (interview OR talk OR keynote)
→ Find: The key voices on this topic.
Q20: "[topic]" "unpopular opinion" OR "hot take" OR "controversial"
→ Find: Contrarian positions that create debate framing.
```
**For each expert, record:**
- Name and affiliation
- Their position or notable quote
- Whether they're mainstream or contrarian (contrarian views make great "but..." moments in scripts)
### Step 7: Visual Reference Scan (Quick Pass)
**Goal:** See how others visualize this concept — inform the Proposal Director's visual approach.
```
Q21: "[topic]" (explainer OR animation OR infographic OR diagram)
→ Find: Visual treatments that work for this topic.
```
Record 2-3 visual references with what works about each approach.
### Step 8: Angle Synthesis
**This is where you earn your keep.** Using everything from Steps 2-7, identify at least 3 genuinely different angle candidates.
For each angle, specify:
| Field | What | Quality Bar |
|-------|------|-------------|
| `name` | Short title (5-8 words) | Specific. "Why Vector Search Beats SQL LIKE" not "About Vector Databases" |
| `hook` | One-sentence grabber | Must create an information gap or surprise |
| `type` | `trending`, `evergreen`, `contrarian`, `narrative`, `data_driven` | Categorize honestly |
| `why_now` | Why this angle is compelling right now | **Must cite specific research findings** — not vibes |
| `grounded_in` | Which data points or audience insights support it | Cross-reference your findings |
**Angle diversity checklist:**
- [ ] At least one angle leverages trending/recent findings (if available)
- [ ] At least one angle is evergreen (works in 6 months too)
- [ ] At least one angle is surprising or contrarian
- [ ] No two angles use the same hook structure
- [ ] Each angle is grounded in different research findings
### Step 9: Source Bibliography
Compile all URLs used, organized by which section of the brief they support. Minimum 5 sources.
**Source quality rules:**
- Primary sources (original studies, official docs) > secondary (news articles, blog posts) > anecdotal (forum comments, tweets)
- At least 2 sources should be primary
- Every data_point must have a source_url
- Flag any source older than 2 years — it may be outdated
### Step 10: Assemble and Submit
Build the `research_brief` artifact per the schema. Include:
1. `research_summary` — one paragraph capturing the single most important insight. This is what the Proposal Director reads first.
2. All sections from Steps 2-9
Validate against `schemas/artifacts/research_brief.schema.json` before submitting.
## Search Query Construction Rules
These rules ensure your searches actually find useful results:
### Use the Current Date
Always include time context in queries where freshness matters:
- `[topic] [current year]` for general freshness
- `[topic] [current month] [current year]` for trending signals
- `after:[YYYY-MM-DD]` filters when supported
### Topic Decomposition
For compound topics, search both the whole and the parts:
- Topic: "how kubernetes autoscaling works"
- Search 1: `kubernetes autoscaling explained`
- Search 2: `kubernetes HPA` (the specific mechanism)
- Search 3: `container orchestration autoscaling` (the broader category)
### Audience-Aware Query Variants
The same topic needs different queries for different audiences:
- For developers: `[topic] implementation` / `[topic] architecture` / `[topic] code example`
- For executives: `[topic] ROI` / `[topic] business impact` / `[topic] case study`
- For general public: `[topic] explained simply` / `what is [topic]` / `[topic] for beginners`
### Quote Mining
To find specific quotable content:
- `"[topic]" "the problem is"` — finds people articulating problems
- `"[topic]" "the key insight"` — finds distilled wisdom
- `"[topic]" "what surprised me"` — finds surprise reactions
### The Negative Space
Search for what's NOT being said:
- `[topic] "nobody talks about"` — finds underserved angles
- `[topic] "overlooked"` — finds hidden aspects
- `[topic] -[obvious_subtopic]` — filters out saturated content
## Quality Bar
Before submitting your research_brief, verify:
| Criterion | Minimum | Target |
|-----------|---------|--------|
| Existing content surveyed | 3 pieces | 5-8 pieces |
| Data points with sources | 3 | 5-8 |
| Audience questions sourced | 3 | 5-10 |
| Misconceptions identified | 1 | 2-3 |
| Angle candidates | 3 | 4-5 |
| Total sources cited | 5 | 10-15 |
| Searches executed | 10 | 15-21 |
**If you can't find data points:** The topic may be too niche or too new. That's useful information — record it in `research_summary` and note that the angle should lean narrative/analogy rather than data-driven.
**If you can't find existing content:** That's a strong signal — a content gap IS the opportunity. Note this prominently.
## Execution Constraints
| Constraint | Value | Why |
|------------|-------|-----|
| Max time on research | 3-5 minutes | Research is valuable but has diminishing returns |
| Max searches | 25 | Prevent infinite rabbit holes |
| Min searches | 10 | Ensure adequate coverage |
| No paid tools | — | Research uses web search only — zero cost |
## Common Pitfalls
- **Skipping to angles without research**: The angles_discovered must be grounded in findings from the other sections. If you can't point to specific data_points or audience_insights that support an angle, the angle is just a guess.
- **Recording vague data**: "Most companies use AI" is not a data point. "87% of Fortune 500 companies have active AI projects (McKinsey 2025)" is a data point.
- **Only searching one way**: If `[topic] statistics` returns nothing, try `[topic] survey`, `[topic] report`, `[topic] data`, `[topic] benchmark`. Vary your query terms.
- **Ignoring negative results**: If searches for trending content return nothing recent, that IS a finding — it means this topic is evergreen, not trending. Record it.
- **Treating all sources equally**: A peer-reviewed study and a random blog post are not equal. Label credibility honestly.
- **Stopping at surface-level**: The first page of Google results is what everyone sees. Dig into specific discussions, specific studies, specific data. The value is in specificity.
## Example: Good vs Bad Research
### Topic: "How DNS Works"
**Bad research output:**
- "DNS is important for the internet"
- "There are many DNS providers"
- Angles: "DNS Explained", "How DNS Works", "Understanding DNS"
**Good research output:**
- Landscape: "Fireship's 'DNS in 100 seconds' has 2.1M views and covers basics but skips DNSSEC entirely. Cloudflare's blog series is comprehensive but text-only. Gap: no visual explainer covers DNS-over-HTTPS controversy."
- Data point: "1.1.1.1 handles 13.5% of all DNS queries globally (Cloudflare Radar 2025, primary source). Surprise factor: counterintuitive — most people think Google's 8.8.8.8 is #1."
- Audience: "Top Reddit question: 'Why does DNS take so long sometimes?' (r/networking, 847 upvotes). Misconception: people think DNS is a single lookup, not a recursive chain."
- Trending: "Cloudflare just launched DNS-over-QUIC support (March 2026). DoH vs DoT debate is active on HN."
- Angles: "The 200ms Journey Your Browser Takes Before Loading Anything" (data_driven, grounded in recursive resolution timing data), "Why Your ISP Knows Every Website You Visit — And How to Stop It" (contrarian, grounded in DNS privacy research + DoH trending signal), "DNS is a 40-Year-Old Phone Book Running the Modern Internet" (narrative/analogy, grounded in audience knowledge gap about DNS age + simplicity)
skills/pipelines/explainer/scene-director.md
+183
View File
@@ -0,0 +1,183 @@
# Scene Director — Explainer Pipeline
## When to Use
You are the Scene Planner for a generated explainer video. You have a `script` artifact with timestamped sections and enhancement cues. Your job is to transform the script into a visual plan: what the viewer sees at every moment, what assets need to be created, and how scenes transition.
This is where words become visuals. A great script with a bad scene plan produces a confusing video.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/scene_plan.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Script sections and creative brief |
| Playbook | Active style playbook | Visual language, transitions, motion rules |
| Layer 3 | `.agents/skills/flux-best-practices/`, `.agents/skills/beautiful-mermaid/`, `.agents/skills/manim-composer/` | Image gen, diagram, animation knowledge |
## Process
### Step 1: Analyze the Script
Read every section. For each, note:
- What concept is being explained?
- What enhancement cues did the script writer embed?
- What's the emotional beat? (curiosity, revelation, emphasis, humor, conclusion)
- How much time is available? (end_seconds - start_seconds)
### Step 2: Research Visual Approaches
**Use web search** to find visual techniques for this topic:
1. **How do top creators visualize this?** Search YouTube thumbnails, blog diagrams, conference slides for the topic.
2. **What visual metaphors work?** Some concepts have well-known visual representations (e.g., neural networks as node graphs, encryption as locks/keys). Use these — viewers recognize them instantly.
3. **What's novel?** Is there a visual approach nobody has tried? A fresh visualization can make an explainer memorable.
4. **What's feasible?** Match your ambitions to available tools: `image_selector` (static images), `diagram_gen` (Mermaid flowcharts/sequences), `code_snippet` (syntax-highlighted code), Remotion (motion graphics, text animations), Manim (mathematical animations).
If you encounter a visualization need that no existing skill covers, use the **Skill Creator** (`skills/meta/skill-creator.md`) to create a new skill.
### Step 3: Decompose into Scenes
Transform each script section into 1-3 visual scenes. Each scene is a distinct visual moment.
```json
{
"id": "scene-3",
"type": "diagram",
"description": "Mermaid flowchart showing query → encode → vector search → rank → return results. Nodes appear one by one as narrator describes each step.",
"start_seconds": 15,
"end_seconds": 22,
"script_section_id": "s3",
"framing": "full-screen diagram, centered",
"movement": "progressive reveal left-to-right",
"transition_in": "fade",
"transition_out": "dissolve",
"overlay_notes": "Label each node as it appears",
"required_assets": [
{
"type": "diagram",
"description": "Mermaid flowchart: query → encode embedding → vector search (ANN) → rank by cosine similarity → return top-k results",
"source": "generate"
}
]
}
```
#### Scene Types and When to Use Them
| Type | Best For | Available Tools | Duration Guidance |
|------|----------|-----------------|-------------------|
| `animation` | Concepts needing motion (data flow, transformations, math) | Remotion, Manim | 4-10s |
| `diagram` | Processes, architecture, relationships | `diagram_gen` (Mermaid), `image_selector` (stylized) | 4-8s |
| `text_card` | Key terms, definitions, statistics, quotes | Remotion TextCard component | 3-5s |
| `generated` | Illustrations, metaphors, real-world imagery | `image_selector` (FLUX/DALL-E) | 3-6s |
| `talking_head` | AI avatar speaking (if HeyGen available) | HeyGen tools | 5-15s |
| `broll` | Context, real-world examples | Stock or generated footage | 3-6s |
| `transition` | Dedicated transition moment between topics | Remotion transition | 1-2s |
| `screen_recording` | Code demos, UI walkthroughs | Recorded or simulated | 5-15s |
### Step 4: Apply the Visual Technique Library
These are proven patterns for explainer visuals. Reference them by name in scene descriptions:
**Diagram Reveal**
Build a diagram progressively — start empty, add components with labels as the narrator describes each part. Perfect for architecture, processes, and systems.
- Tools: Mermaid + Remotion animation or FLUX-generated diagram
- Example: "Show the vector database architecture. Add the encoder node when narrator says 'embeddings'. Add the index when narrator says 'search'."
**Analogy Visualization**
Show the abstract concept alongside its real-world analogy. Split screen or side-by-side.
- Tools: `image_selector` for both sides
- Example: "Left: actual vector space with dots. Right: a library with books sorted by topic."
**Stat Card Punch**
Full-screen number or comparison. Appears with impact animation (scale up, slight bounce). Hold for 2-3 seconds.
- Tools: Remotion TextCard component
- Example: "1ms" in large text, then smaller text below: "vs 500ms with traditional search"
**Before/After Split**
Show the problem, then the solution. Can be sequential (problem → transition → solution) or split-screen.
- Tools: `image_selector` for both states
- Example: "Before: SQL query scanning millions of rows (slow). After: vector search finding nearest neighbors (fast)."
**Timeline Progression**
Left-to-right or top-to-bottom sequence showing evolution or process steps. Each step appears as narrator describes it.
- Tools: Remotion with animated elements or Mermaid timeline
- Example: "1990: keyword search → 2010: semantic search → 2020: vector databases → 2024: multimodal search"
**Zoom and Focus**
Start with a wide view of a system, then zoom into a specific component to explain it in detail. Creates spatial context.
- Tools: Remotion with scale animation on a generated image
- Example: "Show full system architecture. Zoom into the 'embedding model' component."
**Code Walkthrough**
Show code with syntax highlighting. Highlight specific lines as the narrator explains them. Can animate typing or progressive reveal.
- Tools: `code_snippet` tool + Remotion
- Example: "Python code: `results = collection.query(embedding, n_results=5)`. Highlight `embedding` parameter when narrator says 'vector'."
### Step 5: Validate Against Playbook
The style playbook constrains your visual choices:
| Playbook Field | Scene Impact |
|----------------|-------------|
| `visual_language.color_palette` | All generated images and diagrams must use these colors |
| `visual_language.composition` | Framing rules (rule-of-thirds, centered, etc.) |
| `motion.transitions` | Allowed transition types (e.g., `gentle-fade`, `soft-dissolve`) |
| `motion.animation_style` | Animation feel (e.g., `ease-in-out, organic curves`) |
| `motion.pacing_rules` | Minimum hold times (e.g., "hold establishing shots for 2s minimum") |
| `asset_generation.image_prompt_prefix` | Prepend to all image generation prompts |
| `asset_generation.consistency_anchors` | What must stay consistent across all images (color palette, lighting, style) |
**Checklist before submitting:**
- [ ] Every scene uses playbook-compatible transitions
- [ ] All required_asset descriptions include style cues from the playbook
- [ ] No scene violates pacing rules (min/max duration)
- [ ] Image descriptions reference playbook's color palette and texture
### Step 6: Verify Coverage and Variety
**Coverage check:**
- [ ] Scenes span the full script duration (first scene starts at 0s, last scene ends at total_duration)
- [ ] Every script section has at least one corresponding scene
- [ ] No gaps > 1s between scenes (unless intentional beat)
- [ ] All enhancement cues from the script are addressed by a scene or required_asset
**Variety check:**
- [ ] No more than 3 consecutive scenes of the same type
- [ ] At least 3 different scene types used in the video
- [ ] Visual pacing alternates between high-information scenes (diagrams, animations) and breathing room (text cards, generated images)
**Feasibility check:**
- [ ] Every `required_asset` with `source: "generate"` is achievable with available tools
- [ ] Diagram descriptions are specific enough for Mermaid syntax generation
- [ ] Image descriptions are specific enough for FLUX/DALL-E prompt engineering
- [ ] No scene requires tools that aren't in the tool registry
### Step 7: Self-Evaluate
Score (1-5):
| Criterion | Question |
|-----------|----------|
| **Visual storytelling** | Does each scene advance understanding, not just decorate? |
| **Script alignment** | Does every scene match what the narrator is saying at that moment? |
| **Technique variety** | Did you use multiple visual techniques, not just one? |
| **Playbook fidelity** | Would every scene look like it belongs to the same video? |
| **Asset feasibility** | Can every required_asset actually be generated with available tools? |
| **Pacing** | Does the visual rhythm feel natural? High-info scenes balanced with breathing room? |
If any dimension scores below 3, revise.
### Step 8: Submit
Call `handle_explainer_scene_plan(state, {"scene_plan": scene_plan_json})` to validate and persist.
## Common Pitfalls
- **One scene per section**: Script sections often cover multiple concepts. A 10-second section might need 2-3 visual scenes to avoid boring stasis.
- **Ignoring enhancement cues**: The script writer embedded visual hints in `enhancement_cues`. Don't ignore them — they represent the writer's visual intent.
- **Overly ambitious animations**: "Photorealistic 3D fly-through of a data center" can't be generated with current tools. Keep it achievable.
- **No transition strategy**: Random transitions feel chaotic. Use the playbook's transition rules consistently. Reserve special transitions for topic shifts.
- **Vague required_assets**: "An image about databases" is useless for prompt engineering. "Isometric illustration of a vector database with embedding vectors floating in 3D space, using the playbook's blue-green palette" is actionable.
- **Static scenes for dynamic concepts**: If the narrator describes a process or transformation, the visual should move. Use animation or progressive reveal, not a static image.
skills/pipelines/explainer/script-director.md
+223
View File
@@ -0,0 +1,223 @@
# Script Director — Explainer Pipeline
## When to Use
You are the Script Writer for a generated explainer video. You have a `brief` artifact from the Idea Explorer. Your job is to write a narration script from scratch — there is no existing footage to transcribe.
The script is the backbone of the video. Every visual, every scene, every audio cue flows from what you write here. A mediocre script cannot be saved by great visuals.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/script.schema.json` | Artifact validation |
| Prior artifact | `proposal_packet` | Selected concept with title, hook, key_points, core_message, tone, narrative_structure, duration |
| Prior artifact | `research_brief` (optional but high-value) | Data points, audience insights, expert quotes — ground the script in real facts |
| Playbook | Active style playbook from `proposal_packet.selected_concept.suggested_playbook` | Voice style, pacing rules |
| Layer 3 | TTS provider skills (check `agent_skills` on the selected TTS tool) | TTS capabilities for speaker directions |
## Process
### Step 1: Absorb the Proposal and Research
Read the `proposal_packet.selected_concept` carefully. Extract:
- **Target duration** — this is your word budget (see timing table below)
- **Hook** — your opening must deliver on this promise
- **Key points** — these must all be covered in the script
- **Core message** — the one thing the viewer should remember
- **Tone** — shapes word choice, sentence length, formality
- **Target audience** — shapes complexity and assumed knowledge
- **Narrative structure** — the structural approach (myth_busting, journey, data_narrative, etc.)
Then read the `research_brief` for grounding material:
- **`data_points`** — specific statistics and facts to weave into the script. Use claims with `surprise_factor: "surprising"` or `"counterintuitive"` as retention anchors.
- **`audience_insights.misconceptions`** — if the narrative structure is `myth_busting`, these are your myth/reality pairs.
- **`audience_insights.common_questions`** — address these directly in the script where they naturally fit.
- **`expert_voices`** — quotable experts add authority. Use sparingly — one or two per script.
- **`trending.recent_developments`** — if timely, reference them to make the content feel current.
**The research_brief is your cheat sheet.** Every fact, every surprising stat, every misconception is pre-verified and sourced. Use them. A script that cites "73% of developers..." (from research) is more compelling than one that says "many developers..."
### Step 2: Deepen Research Where Needed
The Research Director has already done the heavy lifting — you have a `research_brief` full of sourced facts. Your job here is targeted:
1. **Verify and update**: If any data point from the research_brief feels stale or uncertain, re-search to confirm.
2. **Fill script-specific gaps**: The research gives you broad facts. You may need a specific analogy, a precise technical detail, or a better example for a particular section.
3. **Find the best explanation**: How do the best educators (3Blue1Brown, Kurzgesagt, Fireship, Veritasium) explain this concept? What analogies work?
4. **Source quotable moments**: If the research_brief's expert_voices section has useful quotes, use them. If not, search for one strong quote to anchor a key section.
**Do NOT duplicate the Research Director's work.** If the research_brief already has 6 data points, you don't need to find 6 more. Focus on script-level needs: the right word, the right analogy, the right sequence.
### Step 3: Plan the Narrative Arc
Before writing prose, plan the structure. Every explainer script follows a dramatic arc:
```
HOOK (0-5s) → Grab attention. Question, bold claim, or surprising fact.
NEVER: "In this video, we'll learn about..."
NEVER: "Hey guys, welcome back..."
SETUP (5-15s) → Why should the viewer care? Create a knowledge gap.
Show the problem or the question. Make them NEED the answer.
BUILD (15-Xs) → Progressive revelation. Each section builds on the last.
Use "therefore / but" transitions, NOT "and then."
South Park rule: "This happened, THEREFORE that happened,
BUT then this complication arose..."
CLIMAX (X-5s before end) → The "aha" moment. Everything clicks into place.
This is the payoff for the setup's knowledge gap.
LANDING (last 5s) → Quick recap of core message + CTA.
Don't introduce new information here.
```
Map each of the brief's `key_points` to a specific section in the BUILD phase.
### Step 4: Write the Script
Write each section with these fields:
```json
{
"id": "s1",
"label": "Hook",
"text": "Your database searches every single row. Every. Single. One. What if it didn't have to?",
"start_seconds": 0,
"end_seconds": 5,
"speaker_directions": "Emphasize 'every single row' with measured pacing. Brief pause before the question.",
"enhancement_cues": [
{
"type": "animation",
"description": "Database table with rows highlighted one by one, slowing down as count increases",
"timestamp_seconds": 1
}
],
"pronunciation_guides": []
}
```
#### Timing Estimation
| Pace | Words/minute | Use when |
|------|-------------|----------|
| Conversational | ~150 wpm | Default for most explainers |
| Contemplative | ~120 wpm | Complex topics, need processing time |
| Energetic | ~180 wpm | Short-form, high-energy, TikTok/Reels |
| Technical | ~130 wpm | Code walkthroughs, architecture deep-dives |
**Word budget by duration:**
- 30s video → ~65-75 words
- 60s video → ~130-150 words
- 90s video → ~195-225 words
- 120s video → ~260-300 words
Count your words. If you're 20%+ over budget, the TTS will either rush or exceed duration. Cut ruthlessly.
#### Speaker Directions
Write directions that TTS can actually implement. Reference ElevenLabs capabilities:
| Direction | TTS Implementation |
|-----------|-------------------|
| "Speak slowly, with emphasis" | Lower speed setting, stability boost |
| "Excited, picking up pace" | Higher speed, higher style setting |
| "Pause for 1 second" | SSML `<break time="1s"/>` |
| "Whisper" | SSML whisper tag (model-dependent) |
| "Emphasize THIS word" | Note for post-processing or SSML emphasis |
Avoid directions TTS can't do: "smile while speaking", "gesture toward screen", "look at camera."
#### Enhancement Cues
Every section should have at least one enhancement cue. These tell the Scene Planner and Asset Generator what visuals to create.
| Cue Type | When to Use | Example |
|----------|-------------|---------|
| `overlay` | Key term, definition, label | "Show 'embedding' definition overlay" |
| `diagram` | Process, architecture, flow | "Mermaid flowchart: query → encode → search → rank" |
| `stat_card` | Surprising number or comparison | "Display: 1ms vs 500ms search time" |
| `animation` | Concept that needs motion to understand | "Animate vectors moving through high-dimensional space" |
| `code_snippet` | Code example | "Show Python: `results = collection.query(embedding)`" |
| `broll` | Real-world context | "Show examples of apps using vector search" |
**Density rule**: At least one enhancement cue every 8-10 seconds. A 60-second video should have 6-8 cues minimum. Viewers disengage if the visual doesn't change.
#### Pronunciation Guides
For technical terms, acronyms, and non-English words:
```json
{"word": "FAISS", "phonetic": "FACE"},
{"word": "Qdrant", "phonetic": "kuh-DRANT"},
{"word": "cosine", "phonetic": "CO-sign"}
```
### Step 5: Validate Against Playbook
Read the active style playbook and verify:
| Playbook Field | Script Impact |
|----------------|---------------|
| `identity.pace` | Match word density. `contemplative` = fewer words, longer pauses |
| `audio.voice_style` | Shape tone of speaker directions |
| `motion.pacing_rules` | E.g., "hold establishing shots for 2s minimum" affects section timing |
| `identity.mood` | Word choice: `warm` uses casual language; `professional` uses precise language |
### Step 6: Self-Evaluate
Score your script (1-5):
| Criterion | Question |
|-----------|----------|
| **Hook power** | Would someone stop scrolling in the first 3 seconds? |
| **Word count accuracy** | Within ±10% of target for the duration? |
| **Narrative flow** | Does each section build on the last? "Therefore/but" not "and then"? |
| **Enhancement density** | At least one cue every 8-10 seconds? |
| **Jargon management** | Technical terms explained or have pronunciation guides? |
| **Climax payoff** | Does the aha moment deliver on the hook's promise? |
| **CTA relevance** | Is the call to action specific and actionable? |
If any dimension scores below 3, revise before submitting.
### Step 7: Submit
Call `handle_explainer_script(state, {"script": script_json})` to validate and persist.
## Common Pitfalls
- **Writing too many words**: The #1 failure. TTS pacing is fixed. If you write 250 words for a 60-second video, either the audio will be rushed or the video will be 100 seconds. Count your words.
- **Front-loading information**: The hook should create curiosity, not dump information. "HTTPS uses TLS 1.3 with AEAD ciphers" is a terrible opening. "The padlock icon doesn't mean what you think it means" is compelling.
- **Missing enhancement cues**: A script without visual direction is a podcast script. Every section needs at least one cue telling the visual team what to show.
- **Generic speaker directions**: "Read naturally" is useless. "Start measured and precise, then accelerate through the list to convey scale" is actionable.
- **Forgetting the audience**: A script for CTOs should use different words than one for high schoolers, even if covering the same concept.
- **No transitions between sections**: Each section should have a logical bridge to the next. The viewer should never think "wait, why are we talking about this now?"
## Example: Well-Written Section
```json
{
"id": "s3",
"label": "The Core Idea",
"text": "Instead of matching keywords, vector databases convert everything — text, images, audio — into lists of numbers called embeddings. Similar things get similar numbers. So finding related content becomes a math problem: which numbers are closest?",
"start_seconds": 15,
"end_seconds": 28,
"speaker_directions": "Measured pace through 'text, images, audio' with slight pause between each. Speed up slightly on 'similar things get similar numbers' — it should feel like a revelation. Brief pause before the final question.",
"enhancement_cues": [
{
"type": "animation",
"description": "Show text/image/audio icons transforming into number arrays (embeddings). Arrays cluster by similarity in a 2D space.",
"timestamp_seconds": 16
},
{
"type": "stat_card",
"description": "Display: 'Everything becomes numbers. Similar things → similar numbers.'",
"timestamp_seconds": 22
}
],
"pronunciation_guides": [
{"word": "embeddings", "phonetic": "em-BED-ings"}
]
}
```
skills/pipelines/hybrid/asset-director.md
+70
View File
@@ -0,0 +1,70 @@
# Asset Director - Hybrid Pipeline
## When To Use
This stage prepares the support kit around the anchor edit: subtitles, diagrams, generated inserts, narration, music, and reusable overlay systems.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Support needs and variant plan |
| Tools | `subtitle_gen`, `tts_selector`, `image_selector`, `video_selector`, `diagram_gen`, `code_snippet`, `music_gen`, `audio_enhance` — selectors auto-discover all available providers from the registry | Optional support asset production |
| Playbook | Active style playbook | Consistency rules |
## Process
### 1. Build Shared Support Assets First
Start with reusable systems:
- subtitle treatment,
- lower-third or label system,
- stat-card system,
- CTA container,
- diagram style.
### 1b. Sample Preview (Prevents Wasted Spend)
Before batch-generating support assets, produce one sample of each expensive generated type and show the user:
1. **TTS sample** (if narration is needed): Generate one section. Confirm voice and tone before batching.
2. **Image/video sample** (if generating inserts): Generate one representative visual. Confirm style fits the source footage before batching.
If rejected, adjust parameters and retry (max 3 iterations). Do not batch until approved.
### 2. Generate Only The Support Assets You Need
Support assets should fill identified needs from the script and scene plan, not speculative possibilities.
### 3. Preserve Anchor Truth
Keep the metadata clear about which assets are:
- source-derived,
- provided,
- recorded,
- generated.
### 4. Use Metadata For The Support Map
Recommended metadata keys:
- `shared_support_assets`
- `scene_asset_index`
- `source_vs_generated_map`
- `variant_assets`
### 5. Quality Gate
- support assets map to real narrative needs,
- reusable kits are present,
- source and generated assets are clearly separated,
- every referenced file exists.
## Common Pitfalls
- Overbuilding support assets before the anchor cut is proven.
- Losing track of which assets are generated versus supplied.
- Creating inconsistent overlay systems across one project.
skills/pipelines/hybrid/compose-director.md
+48
View File
@@ -0,0 +1,48 @@
# Compose Director - Hybrid Pipeline
## When To Use
Render the hybrid project so source media, support graphics, and audio all remain coherent across outputs.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | Edit logic and support assets |
| Tools | `video_compose`, `audio_mixer`, `video_stitch`, `video_trimmer`, `color_grade`, `audio_enhance` | Final assembly and polish |
| Playbook | Active style playbook | Output consistency |
## Process
### 1. Verify Source And Support Balance
The final render should still look like a source-led video with support, not a collage of unrelated systems.
### 2. Check Variant Integrity
For each output variant, verify:
- crop safety,
- text safety,
- subtitle legibility,
- audio consistency.
### 3. Keep Audio Coherent
Source dialogue, narration, music, and effects should feel like one mix, not separate layers fighting for space.
### 4. Use Render Metadata
Recommended metadata keys:
- `variant_outputs`
- `balance_checks`
- `subtitle_checks`
- `audio_notes`
## Common Pitfalls
- Good master cut, broken platform variants.
- Support graphics clipping in vertical exports.
- Audio loudness shifting between source and generated sections.
skills/pipelines/hybrid/edit-director.md
+55
View File
@@ -0,0 +1,55 @@
# Edit Director - Hybrid Pipeline
## When To Use
This stage creates the layered edit logic for a source-led video with support elements. The order matters: anchor cut first, support layers second.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/edit_decisions.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["assets"]["asset_manifest"]`, `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]` | Source/support assets and timeline intent |
| Playbook | Active style playbook | Typography and motion consistency |
## Process
### 1. Lock The Anchor Cut First
The viewer should understand the story before support overlays are added. If the anchor cut is weak, support layers will not save it.
### 2. Add Support In Priority Order
Typical order:
1. subtitles,
2. speaker or context labels,
3. diagrams or stat cards,
4. optional inserts,
5. CTA elements.
### 3. Protect Readability
Never stack too many support layers in one moment. If subtitles, labels, charts, and overlays collide, simplify.
### 4. Use Metadata For Layering Logic
Recommended metadata keys:
- `anchor_cut_notes`
- `layer_order`
- `overlay_windows`
- `variant_edit_rules`
### 5. Quality Gate
- the anchor cut works on its own,
- support layers clarify instead of distract,
- mobile readability survives,
- variants remain consistent.
## Common Pitfalls
- Trying to fix a weak cut with extra graphics.
- Letting support layers compete with the source.
- Building each platform variant as a separate editorial philosophy.
skills/pipelines/hybrid/executive-producer.md
+129
View File
@@ -0,0 +1,129 @@
# Executive Producer — Hybrid Pipeline
## When to Use
You are the **Executive Producer (EP)** for a hybrid video that combines source footage with designed or generated support assets. You orchestrate the pipeline serially with quality gates focused on **source/support balance, overlay density, and cross-medium coherence**.
**No pre-production stages.** The user provides direction and source material. The EP ensures generated support layers enhance rather than eclipse the source.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/hybrid.yaml` | Stage definitions |
| Skills | All 7 director skills + `meta/reviewer` | Stage execution |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
## Cumulative State
```
EP_STATE:
pipeline: hybrid
playbook: <selected>
target_duration_seconds: <from brief>
budget_total_usd: <configured>
budget_spent_usd: 0.0
# Hybrid-specific
anchor_medium: null # source footage type (interview, product, screen, etc.)
support_layers: [] # planned support types (diagrams, overlays, graphics, etc.)
source_to_support_ratio: null # target balance (e.g., 70/30 source/support)
artifacts:
idea: null
script: null
scene_plan: null
assets: null
edit: null
compose: null
publish: null
revision_counts: {}
issues_log: []
```
## EP-Specific Cross-Stage Checks
### After IDEA stage:
```
CHECK: Anchor medium clarity
- Is the anchor medium explicitly identified?
- Are support layers justified (filling real gaps, not decorating)?
- Is the source inventory realistic?
```
### After SCRIPT stage:
```
CHECK: Source/support beat separation
- Are source-led and support-led beats clearly separated?
- Does the script avoid relying on unsupported assets?
- Is narration/dialogue plan realistic?
```
### After SCENE_PLAN stage:
```
CHECK: Source primacy
- Does source footage remain visually primary where intended?
- Are overlay and support layers not overloading the frame?
- Max concurrent overlay layers: 2
CHECK: Variant planning
- If platform variants planned: are they realistic?
- Do aspect-ratio variants maintain readability?
```
### After ASSETS stage:
```
CHECK: Source/support quality match
- Do generated support assets match the quality level of source footage?
- Are shared template assets reused across scenes?
- Budget gate: 90% threshold warning
```
### After EDIT stage:
```
CHECK: Anchor-cut coherence
- Is the anchor cut coherent BEFORE support layers are added?
- Do support visuals clarify rather than distract?
- Is variant logic consistent across deliverables?
```
### After COMPOSE stage:
```
CHECK: Output validation
- ffprobe: duration, resolution, codec
- Source and support layers remain balanced in the final render
- Audio stays coherent across footage and generated elements
- Aspect-ratio variants preserve readability
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | idea | Anchor medium, support justification | Revise |
| G2 | script | Source/support separation, narration plan | Revise |
| G3 | scene_plan | Source primacy, overlay density, variants | Revise |
| G4 | assets | Quality match, reuse, budget | Revise |
| G5 | edit | Anchor-cut coherence, support clarity | Revise |
| G6 | compose | Balance, variants, audio coherence | Revise or send-back |
| G7 | publish | Metadata, source-mix labeling | Revise |
| FINAL | all | Source/support balance, readability | Send-back |
## Execution Limits
| Limit | Value |
|-------|-------|
| Max revisions per stage | 3 |
| Max send-backs per stage pair | 1 |
| Max total send-backs | 3 |
| Max total budget | Configurable (default $2) |
| Max total wall-time | 12 minutes |
## Common Pitfalls
- **Support eclipsing source**: Generated graphics should not dominate. Source footage is the anchor.
- **Overlay overload**: Max 2 concurrent overlay layers. More creates visual noise.
- **Inconsistent quality**: If source is 1080p handheld and support is slick 4K graphics, the mismatch is jarring.
- **Ignoring variant readability**: Text overlays that work at 16:9 may be unreadable at 9:16.
skills/pipelines/hybrid/idea-director.md
+73
View File
@@ -0,0 +1,73 @@
# Idea Director - Hybrid Pipeline
## When To Use
Use this pipeline when the project combines real source media with support visuals: interviews plus diagrams, footage plus overlays, screen recording plus branded graphics, or source-led edits with generated inserts.
Hybrid is not a catch-all. Your first job is to define what stays primary.
## Reference Inputs
- `docs/hybrid-video-best-practices.md`
- `skills/creative/storytelling.md`
- `skills/creative/video-editing.md`
## Process
### 1. Choose The Anchor Medium
Pick the storytelling anchor:
- `talking_head`
- `broll_footage`
- `screen_recording`
- `still_sequence`
- `narration_led_graphics`
### 2. Define Support Layers
Possible support layers:
- subtitles,
- diagrams,
- code visuals,
- stat cards,
- generated inserts,
- narration,
- music.
Each support layer should solve a specific problem, not just decorate the timeline.
### 3. Decide The Deliverable Mix
Common outputs:
- hero cut,
- vertical cutdown,
- square cutdown,
- chaptered version,
- ad variant.
### 4. Build The Brief
Recommended metadata keys:
- `anchor_medium`
- `source_inventory`
- `support_layers`
- `deliverable_mix`
- `missing_capabilities`
- `fallback_policy`
### 5. Quality Gate
- the anchor medium is explicit,
- support layers are justified,
- the deliverable mix fits the source inventory,
- missing capabilities are surfaced early.
## Common Pitfalls
- Calling everything hybrid without defining a primary medium.
- Planning support layers before understanding the source.
- Treating optional generated inserts as guaranteed.
skills/pipelines/hybrid/publish-director.md
+50
View File
@@ -0,0 +1,50 @@
# Publish Director - Hybrid Pipeline
## When To Use
Package the hybrid outputs so the hero cut and its derivatives stay organized and the source/support mix remains clear.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/publish_log.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["compose"]["render_report"]`, `state.artifacts["idea"]["brief"]`, `state.artifacts["script"]["script"]` | Final outputs and hybrid framing |
| Playbook | Active style playbook | Tone consistency |
## Process
### 1. Distinguish Master And Variants
Group outputs as:
- master cut,
- short-form derivatives,
- format variants,
- chaptered or contextual variants.
### 2. Preserve Source Truth In Packaging
If the project uses interview footage, screen recording, or product footage as its anchor, the metadata should reflect that instead of packaging it like a pure generated piece.
### 3. Store Cross-Output Notes
Recommended metadata keys:
- `master_output`
- `derivative_outputs`
- `source_mix_notes`
- `platform_copy_map`
### 4. Quality Gate
- master and variants are clearly labeled,
- metadata matches the true source mix,
- export folders are organized by purpose,
- the package is ready to use without manual cleanup.
## Common Pitfalls
- Hiding which output is the hero cut.
- Packaging a source-led project like a generic generated asset.
- Losing platform-specific copy and labeling across variants.
skills/pipelines/hybrid/scene-director.md
+62
View File
@@ -0,0 +1,62 @@
# Scene Director - Hybrid Pipeline
## When To Use
You are translating the hybrid structure into a visual system that keeps the source visible and the support layers under control.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/scene_plan.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Hybrid structure and source truth |
| Tools | `frame_sampler`, `scene_detect` | Optional source inspection |
| Playbook | Active style playbook | Layout consistency |
## Process
### 1. Keep The Anchor Medium Visible
If the piece is source-led, the source must remain visually primary in the scene plan. Do not hide the anchor behind constant overlays.
### 2. Reserve Support For Clear Jobs
Use support scenes for:
- chapter transitions,
- clarifying diagrams,
- stat emphasis,
- CTA or summary moments,
- gap-filling inserts.
### 3. Plan Variant Safety
If the project needs multiple aspect ratios, define where:
- subtitles live,
- speaker labels live,
- chart or code safe zones live,
- crop-sensitive source media becomes unsafe.
### 4. Use Metadata For Balance Rules
Recommended metadata keys:
- `anchor_rules`
- `support_rules`
- `safe_zones`
- `variant_rules`
- `overlay_density_limits`
### 5. Quality Gate
- the anchor medium stays primary where intended,
- support layers are limited and purposeful,
- aspect-ratio planning is explicit,
- no scene relies on invisible future magic.
## Common Pitfalls
- Turning source-led scenes into overlay soup.
- Forgetting variant-safe zones until compose.
- Using generated inserts for every transition.
skills/pipelines/hybrid/script-director.md
+59
View File
@@ -0,0 +1,59 @@
# Script Director - Hybrid Pipeline
## When To Use
This stage maps the story across source-led beats and support-led beats. You are deciding where the source carries the message and where support assets clarify it.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/script.schema.json` | Artifact validation |
| Prior artifact | `state.artifacts["idea"]["brief"]` | Anchor medium and deliverable mix |
| Tools | `transcriber`, `scene_detect`, `audio_enhance` | Optional source analysis |
## Process
### 1. Mark Source-Led Versus Support-Led Beats
For each section, state whether it is:
- carried by source dialogue or footage,
- carried by narration,
- carried by diagrams or overlays,
- carried by text only.
### 2. Use Source Speech When It Is Better Than Rewriting
If the supplied footage already contains strong lines, use `transcriber` and keep the authenticity. Do not replace good source material with unnecessary narration.
### 3. Use Support Only To Clarify
Support-led beats should answer:
- what is not visible,
- what needs summarizing,
- what needs emphasis,
- what changes for a different platform.
### 4. Use Metadata For Structure
Recommended metadata keys:
- `anchor_sections`
- `support_sections`
- `narration_sections`
- `required_support_assets`
### 5. Quality Gate
- source-led beats are clearly marked,
- support-led beats are justified,
- the script does not depend on fake or unavailable assets without saying so,
- the structure can produce the intended deliverables.
## Common Pitfalls
- Rewriting strong source dialogue into weaker narration.
- Adding diagrams or cards where the footage already explains the point.
- Hiding unsupported requirements until asset generation.
skills/pipelines/localization-dub/asset-director.md
+52
View File
@@ -0,0 +1,52 @@
# Asset Director - Localization Dub Pipeline
## When To Use
This stage produces the localized asset kit: translated subtitle files, dubbed audio, optional lip-sync renders, and any language-specific replacements needed for the final outputs.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Language plan and transcript package |
| Tools | `tts_selector`, `subtitle_gen`, `lip_sync`, `audio_enhance` — `tts_selector` auto-discovers all available TTS providers from the registry | Dubbed audio, subtitle, and optional lip-sync production |
| Playbook | Active style playbook | Subtitle and replacement-text rules |
## Process
### 1. Produce Subtitle Assets First
Create the subtitle or caption package for each language. This gives a reviewable fallback even if dubbed-audio generation or lip sync is blocked.
### 2. Generate Dubbed Audio Per Language
Use the approved translated script package, not raw machine output. Record which voice or synthesis path was used for each language.
### 3. Treat Lip Sync As Optional
Only generate lip-sync assets for scenes and languages that actually need it. If the tool path is blocked, record that and keep the dub-audio path alive.
### 4. Use Metadata For Localization Truth
Recommended metadata keys:
- `subtitle_assets_by_language`
- `dub_audio_assets_by_language`
- `lip_sync_assets_by_language`
- `voice_map`
- `pronunciation_warnings`
- `blocked_assets`
### 5. Quality Gate
- subtitle assets exist,
- dubbed audio assets exist for planned dub outputs,
- lip-sync remains explicitly optional,
- every referenced file exists.
## Common Pitfalls
- Generating dubbed audio before finalizing translation review.
- Treating lip sync as mandatory for every language.
- Failing to record which language asset maps to which voice and subtitle set.
skills/pipelines/localization-dub/compose-director.md
+57
View File
@@ -0,0 +1,57 @@
# Compose Director - Localization Dub Pipeline
## When To Use
Render the localized outputs. The quality bar is intelligibility, timing coherence, and clear version labeling across every language package.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | Locale-specific render instructions |
| Tools | `video_compose`, `audio_mixer`, `video_trimmer`, `audio_enhance` | Final render and audio finishing |
| Playbook | Active style playbook | Subtitle placement and output quality |
## Process
### 1. Render By Locale
Treat each target language as its own deliverable set. Keep names and output directories explicit.
### 2. Expect Timing Adjustments
Allow for:
- subtitle reflow,
- dub-audio duration drift,
- longer CTA holds,
- optional trims or coverage sections.
### 3. Verify Every Locale
Record important findings in:
- `render_report.verification_notes`
- `render_report.warnings`
- `render_report.metadata.locale_notes`
Check:
- intelligibility,
- subtitle fit,
- obvious sync drift,
- version labeling.
### 4. Quality Gate
- each locale output exists,
- the dub and subtitle timing are acceptable,
- labels and filenames are unambiguous,
- warnings are preserved.
## Common Pitfalls
- Rendering all locales as if they were timing-identical.
- Forgetting to re-check subtitle line length after translation.
- Naming outputs in ways that hide the locale or treatment mode.
skills/pipelines/localization-dub/edit-director.md
+46
View File
@@ -0,0 +1,46 @@
# Edit Director - Localization Dub Pipeline
## When To Use
Translate the scene plan and localized asset kit into concrete timeline decisions for each language output. The goal is to preserve the source structure where possible without pretending all languages land on the same timing.
## Process
### 1. Preserve Structure By Default
Keep the original scene order and major timing unless the translated audio clearly requires extension, compression, or coverage.
### 2. Apply The Chosen Dub Mode
Per deliverable, decide where to:
- keep original picture with new subtitles,
- replace only the audio,
- use lip-sync output,
- cover mismatch with graphics or B-roll.
### 3. Keep Language Variants Organized
Separate timeline decisions by locale so versioning stays clear all the way into compose and publish.
### 4. Use Metadata For Variant Control
Recommended metadata keys:
- `locale_timeline_map`
- `timing_adjustments`
- `coverage_sections`
- `subtitle_strategy_by_locale`
### 5. Quality Gate
- language variants are explicit,
- timing changes are recorded,
- coverage decisions are deliberate,
- the original structure is only changed where necessary.
## Common Pitfalls
- Forcing every language to match source timing exactly.
- Mixing locale-specific notes into one ambiguous edit list.
- Hiding sections where the dub treatment is visually weak.
skills/pipelines/localization-dub/executive-producer.md
+133
View File
@@ -0,0 +1,133 @@
# Executive Producer — Localization Dub Pipeline
## When to Use
You are the **Executive Producer (EP)** for a localization/dubbing project. You orchestrate the pipeline serially with quality gates focused on **translation accuracy, timing preservation, lip-sync quality, and per-locale consistency**.
**No pre-production stages.** Source video exists in one language. The EP manages the complexity of producing multiple language variants while preserving the original's timing and quality.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/localization-dub.yaml` | Stage definitions |
| Skills | All 7 director skills + `meta/reviewer` | Stage execution |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
## Cumulative State
```
EP_STATE:
pipeline: localization-dub
playbook: <selected>
budget_total_usd: <configured>
budget_spent_usd: 0.0
# Localization-specific
source_language: null
target_languages: []
dub_mode_per_locale: {} # language → subtitle_only / dub / dub_with_lipsync
glossary_terms: [] # protected terms that must not be translated
timing_drift_tolerance: 0.5 # seconds
artifacts:
idea: null
script: null
scene_plan: null
assets: null
edit: null
compose: null
publish: null
revision_counts: {}
issues_log: []
```
## EP-Specific Cross-Stage Checks
### After IDEA stage:
```
CHECK: Scope definition
- Source and target languages explicit?
- Deliverable mode clear per language (subtitle / dub / dub+lipsync)?
- Glossary and protected terms captured?
- Review requirements noted?
```
### After SCRIPT stage:
```
CHECK: Transcript truth
- Source transcript accurate and timestamped?
- Glossary terms preserved in translations?
- Translated scripts reviewable before synthesis?
- Duration estimates per language reasonable (some languages expand 20-30%)?
```
### After SCENE_PLAN stage:
```
CHECK: Dub mode feasibility
- Is the chosen dub mode realistic per locale?
- Lip-sync limited to shots that can support it (front-facing, clear mouth)?
- Timing drift risks mapped (which languages will run long)?
- On-screen text replacement planned if needed?
```
### After ASSETS stage:
```
CHECK: Locale asset completeness
- Subtitle files exist for every target language?
- Dubbed audio generated for every dub-mode language?
- TTS voice quality acceptable for each language?
- Lip-sync applied only where planned?
- Budget gate: 90% threshold (localization can be expensive with many languages)
```
### After EDIT stage:
```
CHECK: Timing preservation
- Source structure preserved unless timing forces change?
- CTA and legal copy survive translation?
- Language variants organized consistently?
- Timing drift within tolerance per segment?
```
### After COMPOSE stage:
```
CHECK: Per-locale validation
- Each language output rendered and intelligible?
- Subtitle timing matches speech in each locale?
- Version labeling unambiguous (language code in filename)?
- Audio quality consistent across locales?
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | idea | Scope, languages, dub modes | Revise |
| G2 | script | Transcript accuracy, glossary, translations | Revise |
| G3 | scene_plan | Dub mode feasibility, timing risks | Revise |
| G4 | assets | Locale completeness, TTS quality, budget | Revise |
| G5 | edit | Timing preservation, structure | Revise |
| G6 | compose | Per-locale probe, subtitle timing | Revise or send-back |
| G7 | publish | Locale packaging, metadata | Revise |
| FINAL | all | Translation quality, timing, lip-sync | Send-back |
## Execution Limits
| Limit | Value |
|-------|-------|
| Max revisions per stage | 3 |
| Max send-backs per stage pair | 1 |
| Max total send-backs | 3 |
| Max total budget | Configurable (default $3 — localization is costlier) |
| Max total wall-time | 15 minutes |
## Common Pitfalls
- **Ignoring language expansion**: Some languages are 20-30% longer than English. The dubbed audio won't fit the original timing without adjustments.
- **Lip-sync on every shot**: Only apply lip-sync to front-facing, clear-mouth shots. Side angles and distant shots don't need it.
- **Translating protected terms**: Brand names, product names, and technical terms in the glossary must stay in the original language.
- **Inconsistent locale labeling**: Use ISO language codes in filenames. "Spanish" is ambiguous (es-ES vs es-MX).
- **Degrading source video**: Re-encoding the source video for each locale should preserve quality. Never downgrade resolution.
skills/pipelines/localization-dub/idea-director.md
+70
View File
@@ -0,0 +1,70 @@
# Idea Director - Localization Dub Pipeline
## When To Use
Use this pipeline when the user has a source video and wants translated deliverables: subtitles, dubbed audio, or localized videos in one or more target languages.
Your first responsibility is to define what kind of localization is actually required, because subtitle-only, dubbed-audio, and lip-synced translation are different jobs.
## Reference Inputs
- `docs/localization-dubbing-best-practices.md`
- `skills/creative/short-form.md`
- `skills/creative/long-form.md`
## Process
### 1. Define The Localization Scope
Capture:
- source language,
- target languages,
- review owner,
- whether glossary or legal review is required,
- whether the user needs subtitles, dubbed audio, lip-sync, or a mix.
### 2. Classify The Source
Record the source mode:
- `single_speaker`
- `multi_speaker`
- `voiceover_led`
- `speaker_led_on_camera`
Also record whether on-screen text or motion graphics will need manual replacement or coverage.
### 3. Pick Deliverables That Match Reality
Possible deliverables:
- subtitle package only,
- dubbed video without lip sync,
- lip-synced localized video,
- per-language export bundle.
### 4. Build The Brief
Recommended metadata keys:
- `source_language`
- `target_languages`
- `deliverable_mode_map`
- `glossary_terms`
- `protected_terms`
- `review_requirements`
- `timing_risks`
### 5. Quality Gate
- localization scope is explicit,
- target outputs are realistic,
- glossary and review requirements are captured,
- risk increases from speaker count or visible mouths are surfaced.
## Common Pitfalls
- Calling every translation request a dubbing request.
- Ignoring glossary control until after audio is generated.
- Promising lip sync on visually difficult source footage without warning.
skills/pipelines/localization-dub/publish-director.md
+44
View File
@@ -0,0 +1,44 @@
# Publish Director - Localization Dub Pipeline
## When To Use
Package the completed localization outputs so downstream teams can find the right video, subtitle, and metadata bundle for each language without manual cleanup.
## Process
### 1. Package By Locale
Each language package should clearly separate:
- video output,
- subtitle files,
- transcript or approved script copy,
- review notes,
- metadata.
### 2. Keep Naming Precise
Recommended metadata keys:
- `locale`
- `language_name`
- `deliverable_mode`
- `subtitle_included`
- `review_owner`
### 3. Preserve Review Context
If a language output has pronunciation caveats, timing warnings, or missing lip sync, keep that note in the published package.
### 4. Quality Gate
- locale packages are clearly labeled,
- metadata matches the actual treatment,
- supporting text assets are present,
- warnings and review notes are not lost.
## Common Pitfalls
- Shipping localized videos without the matching subtitle or transcript files.
- Mixing audio-dub and subtitle-only variants under the same generic filename.
- Removing the QA notes that explain known issues.
skills/pipelines/localization-dub/scene-director.md
+65
View File
@@ -0,0 +1,65 @@
# Scene Director - Localization Dub Pipeline
## When To Use
Plan how each localized deliverable will handle timing, visible speech, subtitles, and on-screen text. This is where the pipeline decides whether to preserve the original cut, cover mouth-visible sections, or attempt lip sync.
## Reference Inputs
- `docs/localization-dubbing-best-practices.md`
- `skills/creative/video-editing.md`
## Process
### 1. Choose The Dub Mode Per Deliverable
Use one of:
- `subtitle_only`
- `dub_audio_only`
- `lip_synced`
- `hybrid_covered`
`hybrid_covered` means using B-roll, graphics, or text coverage during sections where visible mouth mismatch would be distracting.
### 2. Map Timing Risk
Identify scenes likely to drift because of:
- fast speech,
- dense legal copy,
- multiple speakers,
- fast cuts,
- visible close-up mouths.
### 3. Note On-Screen Language Dependencies
Record scenes that contain:
- UI text,
- lower thirds,
- title cards,
- baked-in subtitles,
- charts or labels that may need replacement or coverage.
### 4. Use Metadata For Variant Planning
Recommended metadata keys:
- `dub_mode_map`
- `timing_risk_map`
- `on_screen_text_replacement_map`
- `language_variant_notes`
### 5. Quality Gate
- every deliverable has a defined localization treatment,
- timing risks are mapped,
- lip-sync usage is selective,
- text replacement needs are not hidden.
## Common Pitfalls
- Assuming dubbed audio will fit the source timing exactly.
- Choosing lip sync for every shot instead of only the shots that justify it.
- Forgetting about baked-in text until compose time.
skills/pipelines/localization-dub/script-director.md
+54
View File
@@ -0,0 +1,54 @@
# Script Director - Localization Dub Pipeline
## When To Use
Turn the approved localization brief into a transcript-backed, reviewable script package for every target language. This stage should create text truth before any dubbing audio is generated.
## Reference Inputs
- `docs/localization-dubbing-best-practices.md`
- `skills/creative/storytelling.md`
## Process
### 1. Build Source Transcript Truth
Start with the source transcript and fix obvious errors in:
- names,
- terminology,
- speaker allocation,
- numbers,
- CTA phrasing.
### 2. Produce Reviewable Target Copy
For each target language, generate text that can be reviewed before synthesis. Record where terms should remain unchanged.
### 3. Preserve Structure Where Practical
Keep section timing and sequence aligned to the source unless the translation clearly needs a different pacing strategy.
### 4. Use Metadata For Localization Control
Recommended metadata keys:
- `source_transcript_status`
- `target_language_sections`
- `glossary_terms`
- `protected_terms`
- `pronunciation_notes`
- `review_status_by_language`
### 5. Quality Gate
- the source transcript is strong enough to trust,
- target-language copy exists for every planned deliverable,
- glossary terms are preserved,
- the script package can be reviewed before audio generation.
## Common Pitfalls
- Generating audio from an unreviewed transcript.
- Letting product names drift across languages.
- Treating translation text as final timing without acknowledging length drift.

Some files were not shown because too many files have changed in this diff Show More