Initial release — OpenMontage: the first open-source agentic video production system

11 production pipelines, 47 tools, 124 agent skills.
Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and
free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
calesthio
2026-03-29 08:25:17 -07:00
commit a3e735cc7a
1147 changed files with 240221 additions and 0 deletions
@@ -0,0 +1,109 @@
# Asset Director - Screen Demo Pipeline
## When To Use
This stage produces the minimal but high-leverage assets that make a screen demo easier to follow: subtitles, audio cleanup, reusable overlays, masks, and optional light-weight support cards.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | What to produce |
| Tools | `subtitle_gen`, `audio_enhance`, `tts_selector`, `image_selector`, `diagram_gen` — selectors auto-discover all available providers from the registry | Generation capabilities |
| Playbook | Active style playbook | Typography and overlay styling |
## Process
### 1. Prioritize Utility Over Decoration
Screen demos do not need a large asset pile. They need the right few assets:
- mandatory: subtitles
- usually mandatory: cleaned primary audio
- usually helpful: reusable highlight box, arrow, step label, blur mask kit
- optional: one intro card, one outro card, sparse diagram overlays
- optional only if preflight allows it: generated narration for silent recordings
### 2. Generate Subtitles First
Rules:
- high contrast over unknown UI backgrounds,
- never cover the text the viewer needs to read,
- prefer phrase-level chunks unless word-by-word highlighting materially helps,
- prepare position override notes in `asset_manifest.metadata.subtitle_zones`.
### 3. Build A Reusable Overlay Kit
Do not generate bespoke assets for every click. Build a small shared kit:
- `highlight_box_primary`
- `arrow_primary`
- `step_label_primary`
- `keystroke_badge_primary`
- `blur_mask_template`
These should be reusable across scenes, with timing and placement handled downstream.
### 4. Clean Or Generate Audio Pragmatically
Goals:
- remove distracting keyboard and room noise,
- normalize speech,
- preserve timing,
- do not over-process into robotic audio.
If the recording is silent:
- only generate narration if TTS passed preflight,
- otherwise keep the asset plan text-led and note the limitation in metadata.
### 5. Only Generate Supplementary Visuals When They Earn It
Use `image_selector` or `diagram_gen` only for:
- a short opening card,
- a step transition card,
- a simple diagram that clarifies a hidden process,
- an outro card.
Do not create decorative artwork for a workflow the screen already explains.
### 6. Build The Asset Manifest Cleanly
Every asset must have a valid schema type and `scene_id`.
Use `asset_manifest.metadata` for details like:
- `subtitle_zones`
- `overlay_kit`
- `audio_settings`
- `narration_mode`
- `sensitive_regions`
### 7. Quality Gate
**Existence check:**
- [ ] Subtitle file exists at declared path and parses without errors
- [ ] Cleaned audio file exists and has the expected duration
- [ ] Reusable overlay kit exists and covers planned callout types
- [ ] All supplementary visuals exist at declared paths
**Timing check:**
- [ ] Subtitle timestamps align with script section timestamps
- [ ] If narration was generated, timing matches section duration closely enough for editing
**Quality check:**
- [ ] Subtitles are readable at output resolution
- [ ] Cleaned audio has no remaining distracting noise
- [ ] Callout colors have sufficient contrast
- [ ] Blur masks fully cover the sensitive content
## Common Pitfalls
- Generating too many one-off overlay files instead of a reusable kit.
- Using subtitles that sit directly on top of terminal output or bottom navigation.
- Assuming silent recordings will magically gain narration without checking TTS.
- Spending image generation budget on visuals the raw screen already provides.
@@ -0,0 +1,86 @@
# Compose Director - Screen Demo Pipeline
## When To Use
Render the final screen-demo outputs. The quality bar is simple: the UI must be readable, the pacing must feel intentional, and the result must match the planned platform shapes.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | What to render |
| Tools | `video_compose`, `audio_mixer`, `video_trimmer` | Rendering capabilities |
| Playbook | Active style playbook | Quality targets |
## Process
### 1. Render For Legibility First
Prefer the simplest reliable render chain:
- trim and speed-adjust source footage,
- compose overlays and subtitles,
- mix audio only as much as needed,
- encode at a bitrate suitable for text-heavy content.
### 2. Choose Output Shapes Pragmatically
| Platform | Aspect Ratio | Resolution | Notes |
|----------|--------------|------------|-------|
| YouTube / docs | `16:9` | 1920x1080 or higher | safest default for dense UI |
| LinkedIn feed | `1:1` | 1080x1080 | good compromise when vertical is too tight |
| Shorts / Reels / TikTok | `9:16` | 1080x1920 | only if crop plan is actually readable |
If the source is 4K and text is tiny, keep a higher resolution when practical.
### 3. Compose In The Right Order
1. apply trims and speed changes,
2. apply crop and framing strategy,
3. place masks and overlays,
4. burn subtitles,
5. mix audio,
6. encode with text-preserving settings.
Use sharp scaling and avoid aggressive compression. Screen text is the first thing viewers notice when encode quality drops.
### 4. Keep Audio Honest
- preserve original speech clarity,
- do not overcompress,
- mute or simplify useless sped-up noise,
- use music sparingly, if at all.
### 5. Verify Every Output
**File checks:**
- [ ] Output file exists and is a valid MP4 container
- [ ] Duration matches effective target within +/-5%
- [ ] Resolution matches selected profile
**Visual spot checks:**
- [ ] Text is sharp and readable at sampled frames
- [ ] Crop transitions are smooth enough to follow
- [ ] Callout overlays appear and disappear cleanly
- [ ] Blur masks fully cover sensitive data
- [ ] No black frames or timing glitches
- [ ] Subtitles do not sit on top of critical UI
**Audio spot checks:**
- [ ] Narration/voiceover is clear and consistent volume
- [ ] Music, if used, is not competing with speech
- [ ] No obvious audio glitches at speed boundaries
- [ ] No clipping or distortion
Record important findings in:
- `render_report.verification_notes`
- `render_report.warnings`
- `render_report.metadata.variant_notes`
## Common Pitfalls
- Rendering `9:16` versions that are technically exported but practically unreadable.
- Encoding screen text with generic low-bitrate social defaults.
- Letting decorative backgrounds or padding reduce usable UI area too far.
@@ -0,0 +1,87 @@
# Edit Director - Screen Demo Pipeline
## When To Use
This stage turns the plan into a concrete, schema-valid edit: trims, speeds, overlays, subtitles, and transitions. Keep the edit simple enough to execute with the current tooling and explicit enough that composition is predictable.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/edit_decisions.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["assets"]["asset_manifest"]`, `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]` | Assets, visual plan, timing |
| Playbook | Active style playbook | Overlay and pacing rules |
## Process
### 1. Start With The Smallest Clear Edit
Screen demos get worse when over-edited. Build the timeline in this order:
1. trim or cut dead time,
2. apply speed changes,
3. place overlays,
4. set subtitle behavior,
5. define audio behavior,
6. capture detailed crop/ramp notes in `edit_decisions.metadata`.
### 2. Keep The Schema Clean
Use `cuts[]` for actual source segments and speed changes. Use `overlays[]`, `subtitles`, `music`, and `transitions` only for things the schema already models. Put screen-demo-specific detail in metadata:
- `crop_keyframes`
- `speed_plan`
- `subtitle_position_overrides`
- `audio_notes`
- `variant_notes`
### 3. Editing Rules
- the viewer should see useful motion or result within the first seconds,
- result moments stay at normal speed,
- typing, installs, and waiting should be accelerated or removed,
- no cut starts mid-word or ends before the payoff lands,
- do not introduce more motion through editing than the scene plan asked for.
### 4. Overlay Rules
- hook or step label can appear immediately,
- callouts should appear slightly before the action,
- blur masks must be treated as critical, not optional,
- subtitles and callouts must not compete for the same space.
### 5. Audio Rules
- keep primary speech clear and centered,
- mute or greatly reduce meaningless sped-up noise,
- only use background music if it adds value and survives ducking gracefully,
- if narration was generated, ensure it fits the tightened timeline.
### 6. Quality Gate
**Timeline integrity:**
- [ ] Cuts cover the full intended timeline
- [ ] No accidental black gaps
- [ ] Speed ramps don't overlap
- [ ] Effective duration matches the brief closely
**Overlay integrity:**
- [ ] Every planned callout or mask is represented
- [ ] No overlay collisions
- [ ] UI-anchored overlays are documented clearly enough to position during compose
**Audio integrity:**
- [ ] Primary audio or narration covers the entire timeline
- [ ] Speed-up segments have intentional audio treatment
- [ ] Music, if present, will not compete with instruction
**Subtitle integrity:**
- [ ] Subtitles are present for all narrated sections
- [ ] Position overrides protect important UI content
- [ ] Subtitle timing still works after planned speed changes
## Common Pitfalls
- Overbuilding the edit with cinematic transitions the workflow does not need.
- Letting sped-up audio become a wall of harsh clicks and typing.
- Forgetting that crop and speed plans live in metadata, not arbitrary schema fields.
@@ -0,0 +1,277 @@
# Executive Producer — Screen-Demo Pipeline
## When to Use
You are the **Executive Producer (EP)** for a screen-demo video. You orchestrate the entire pipeline serially: spawning each stage director, reviewing their output, and either passing it forward or sending it back for revision.
**This pipeline has no pre-production stages** (no research, no proposal). Source footage already exists. The EP adds cross-stage quality gates that catch legibility, audio clarity, and pacing issues early — before the expensive compose step.
## Why This Exists
Screen-demo videos have specific failure modes that parallel execution misses:
- Text in screen recordings becomes unreadable after crops and scaling
- Zoom-crop regions that looked fine in planning obscure critical UI elements in practice
- Keyboard noise and background hum survive into the final render
- Dead time (loading screens, typing pauses) makes videos unwatchable without speed adjustments
- Callout overlays block the very UI they're trying to highlight
- Subtitle positioning conflicts with screen content
The EP catches all of these at the earliest possible stage.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Pipeline | `pipeline_defs/screen-demo.yaml` | Stage definitions, review focus, success criteria |
| Skills | All 7 director skills + `meta/reviewer` | Stage execution knowledge |
| Schemas | All artifact schemas | Validation |
| Playbook | Active style playbook | Quality constraints |
| Tools | Full tool registry | Available capabilities |
## Cumulative State
```
EP_STATE:
pipeline: screen-demo
playbook: <selected playbook name>
target_duration_seconds: <from brief or estimated from source>
budget_total_usd: <configured limit>
budget_spent_usd: 0.0
budget_remaining_usd: <budget_total>
# Screen-demo specific state
source_resolution: null # original recording resolution
target_resolution: null # output resolution
has_voiceover: false # does source have narration audio?
has_keyboard_noise: false # flagged during script/asset stage
zoom_regions: [] # crop regions from scene plan, for cross-checking
# Accumulated from each stage (7 stages)
artifacts:
idea: null # → brief
script: null # → script
scene_plan: null # → scene_plan
assets: null # → asset_manifest
edit: null # → edit_decisions
compose: null # → render_report
publish: null # → publish_log
# Cross-stage tracking
narration_durations: {}
style_anchors: {}
revision_counts: {}
issues_log: []
```
## Execution Protocol
### Phase 0: Initialize
1. Load the pipeline manifest (`screen-demo.yaml`)
2. Load the playbook (from user selection or default)
3. Set budget from configuration or user input (default: $1.00 — screen-demo is typically low-cost)
4. Initialize EP_STATE
### Phase 1: Execute Stages Serially
For each stage in order: `idea → script → scene_plan → assets → edit → compose → publish`
```
EXECUTE_STAGE(stage_name):
1. PREPARE
- Load the director skill for this stage
- Inject EP_STATE as context
- Inject any EP feedback from previous revision attempts
2. SPAWN DIRECTOR
- Director executes its full process
- Director produces an artifact
3. REVIEW
- Schema validation
- Check review_focus from pipeline manifest
- Check success_criteria from pipeline manifest
- Run EP-SPECIFIC CROSS-STAGE CHECKS (see below)
4. GATE DECISION
If PASS → store artifact, update tracking, continue
If REVISE → increment revision count, re-run with feedback (max 3)
If SEND_BACK(target_stage) → re-execute from target forward (max 1 per pair)
```
### Phase 2: Final Quality Assurance
```
FINAL_QA:
1. PROBE the output video:
- Duration: reasonable for the demo content?
- Resolution: matches target?
- Audio: voiceover clear? Keyboard noise removed?
- File: valid container, reasonable size?
2. LEGIBILITY CHECK (SCREEN-DEMO CRITICAL):
- Is UI text readable at the output resolution?
- Are zoom-crop regions showing the intended UI elements?
- Are callout overlays not obscuring critical content?
3. PACING CHECK:
- Are loading/typing pauses sped up or cut?
- Does the demo flow logically?
- Are transitions between workflow steps smooth?
4. SUBTITLE CHECK:
- Do subtitles not overlap with screen content?
- Is subtitle timing accurate to speech?
5. BUDGET RECONCILIATION:
- Total actual spend vs. budget
- Log per-stage cost breakdown
6. DECISION:
If all pass → APPROVE for publish
If legibility issues → send back to compose (re-render) or scene (replan crops)
If audio issues → send back to compose (re-mix)
If pacing issues → send back to edit (re-time)
```
## EP-Specific Cross-Stage Checks
### After IDEA stage:
```
CHECK: Source assessment
- Is source footage referenced and accessible?
- Is target platform and duration realistic?
- Are callout/zoom needs identified?
- If no source footage: STOP — this pipeline requires source footage
```
### After SCRIPT stage:
```
CHECK: Transcript quality
- If source has voiceover: is transcript accurate and timestamped?
- Are key UI actions annotated with timestamps?
- Are workflow steps clearly segmented?
- Flag keyboard noise presence for asset stage
CHECK: Duration estimate
- Estimated output duration reasonable for the content?
- If demo is > 5 minutes: suggest trimming or speed adjustments
```
### After SCENE_PLAN stage:
```
CHECK: Zoom-crop feasibility
- For each crop region: does it capture the intended UI element?
- Are crop regions at least 50% of source resolution? (avoid extreme zooms that pixelate)
- Store zoom_regions in EP_STATE for compose verification
CHECK: Callout placement
- Do callout overlays (arrows, highlights, masks) avoid obscuring the UI element they reference?
- Are callouts sparse? (max 2-3 concurrent callouts)
CHECK: Pacing plan
- Are dead-time segments (loading, typing) flagged for speed-up or cut?
- Are speed changes smooth (not jarring jumps)?
```
### After ASSETS stage:
```
CHECK: Subtitle positioning
- Do subtitles avoid overlapping with key screen content?
- Is subtitle font readable against screen background?
CHECK: Audio quality
- If audio_enhance was used: is keyboard noise reduced?
- If TTS was generated: does narration timing match screen actions?
CHECK: Budget gate
- If budget_spent > budget_total * 0.9 and stages remain:
Alert and adjust remaining stages
```
### After EDIT stage:
```
CHECK: Timeline completeness
- All edit decisions reference valid source files and assets
- Audio ducking configured if background music added
- Speed adjustments are smooth (ramp, not jump)
CHECK: Dead time handling
- Loading screens and typing pauses either cut or sped up
- Total dead time < 10% of output duration
```
### After COMPOSE stage:
```
CHECK: Output validation
- ffprobe: duration, resolution, codec, audio channels
- Text readability at output resolution
- Audio clarity — voiceover intelligible throughout
CHECK: Screen sharpness (SCREEN-DEMO CRITICAL)
- UI text in the recording must be readable
- If crops caused pixelation: flag for scene plan revision
- Anti-aliased text must survive compression
```
## Feedback Message Templates
### To Script Director:
```
EP FEEDBACK — Script Revision Required
Reason: {reason}
Specific issue: {transcript_accuracy / segmentation / timing}
Keep: {what was good}
Change: {what specifically needs to change}
```
### To Scene Director:
```
EP FEEDBACK — Scene Plan Revision Required
Reason: {reason}
Affected scenes: {scene_ids}
Constraint: {crop_feasibility / callout_placement / pacing}
Source resolution: {W}x{H} — minimum crop: {W/2}x{H/2}
```
### To Compose Director:
```
EP FEEDBACK — Re-render Required
Reason: {reason}
Specific issue: {legibility / audio / pacing}
Expected: {what the output should look/sound like}
Actual: {what was produced}
```
## Quality Gates Summary
| Gate | After Stage | What's Checked | Fail Action |
|------|-------------|---------------|-------------|
| G1 | idea | Source assessment, feasibility | Revise idea |
| G2 | script | Transcript accuracy, duration estimate | Revise script |
| G3 | scene_plan | Crop feasibility, callout placement, pacing plan | Revise scene_plan |
| G4 | assets | Subtitle positioning, audio quality, budget | Revise assets |
| G5 | edit | Timeline completeness, dead time handling | Revise edit |
| G6 | compose | Output probe, screen sharpness, audio clarity | Revise compose OR send-back |
| G7 | publish | Metadata, chapters, export packaging | Revise publish |
| FINAL | all | Legibility, pacing, subtitles, audio | Send-back to specific stage |
## Execution Limits
| Limit | Value | Rationale |
|-------|-------|-----------|
| Max revisions per stage | 3 | Prevent perfectionism loops |
| Max send-backs per stage pair | 1 | Prevent ping-pong |
| Max total send-backs | 3 | Cap total re-work |
| Max total budget | Configurable (default $1) | Hard stop on spending |
| Max total wall-time | 10 minutes | Screen-demo is simpler than generated pipelines |
## Common Pitfalls
- **Ignoring text readability**: The #1 screen-demo issue. Always verify UI text is readable after crops.
- **Over-cropping**: Extreme zooms pixelate. Minimum crop should be 50% of source resolution.
- **Leaving dead time**: Loading screens and typing pauses must be handled. Speed-up or cut.
- **Callout overload**: More than 2-3 concurrent callouts creates visual chaos.
- **Ignoring keyboard noise**: If the source has keyboard sounds, flag it early for audio cleanup.
@@ -0,0 +1,112 @@
# Idea Director - Screen Demo Pipeline
## When To Use
Use this pipeline when the source is already a screen recording: product walkthrough, software tutorial, coding demo, browser flow, or troubleshooting capture.
Your job is to turn raw capture into a clear procedural video. The main deliverable at this stage is a schema-valid `brief`, with pipeline-specific detail stored in `brief.metadata`.
## Operating Principles
Screen-demo best practices are consistent:
- prioritize procedure over theory,
- keep scope to one workflow or one outcome,
- map narration to visible action,
- plan attention guidance with restraint,
- optimize for legibility before style.
Reference docs:
- `docs/screen-demo-best-practices.md`
- `skills/creative/screen-recording.md`
## Process
### 1. Inspect The Source
Use the available analysis tools before writing the brief:
- `frame_sampler` for representative frames and dense samples around likely key moments
- `scene_detect` for window switches, page changes, and major layout changes
- `transcriber` to determine whether the recording has narration, system audio only, or silence
Identify:
- software and surfaces shown,
- the single workflow being taught,
- critical interactions: click, type, scroll, submit, result,
- moments that obviously need zoom or highlight support,
- dead time: installs, builds, loading, repetitive typing,
- whether `9:16` is even feasible without losing meaning.
### 2. Classify The Demo
Choose one dominant archetype:
- `tutorial`: step-by-step task completion
- `feature_showcase`: show what a feature does
- `troubleshooting`: reproduce and fix a problem
- `walkthrough`: explain a multi-step flow across tools
- `comparison`: compare two approaches or outcomes
If the footage mixes several archetypes, pick the one that should drive pacing and packaging.
### 3. Set Deliverable Intent
Screen demos should stay narrow and outcome-led:
- `30-60s`: quick tip or feature reveal
- `60-120s`: focused product walkthrough or bug fix
- `120-300s`: chaptered tutorial
Default to the shortest duration that still teaches the task cleanly. Do not preserve raw duration unless the user explicitly wants training footage with minimal compression.
### 4. Choose A Viable Output Shape
Plan the platform around readability, not trend pressure:
- use `youtube` or `linkedin` for dense desktop UI,
- use `instagram` or `tiktok` only if the active area can survive a narrow crop,
- prefer `1:1` or `16:9` when the interface has multiple panels or code windows.
### 5. Build The Brief
Use the schema fields for the concise creative contract and store the richer production detail in `metadata`.
Recommended `metadata` keys:
- `source_path`
- `source_duration_seconds`
- `source_resolution`
- `has_voiceover`
- `software_shown`
- `demo_archetype`
- `critical_moments`
- `dead_time_segments`
- `recommended_aspect_ratios`
- `notes_for_scene_planner`
The brief should answer:
- what the viewer will learn,
- who this is for,
- what proof/result the video should land on,
- what the must-show actions are,
- which crop directions are safe.
### 6. Quality Gate
Before checkpointing, verify:
- the workflow is narrow enough for the chosen duration,
- the "aha" result is clearly identified,
- the target platform matches the UI density,
- the brief names the actual software rather than describing it vaguely,
- the metadata gives downstream stages enough production truth.
## Common Pitfalls
- Treating a 7-minute recording as a 7-minute deliverable by default.
- Choosing `9:16` for a dense desktop capture just because the user asked for Shorts.
- Writing a concept-heavy brief when the user really needs task completion.
- Failing to note silence; if there is no voiceover, downstream stages must know immediately.
@@ -0,0 +1,80 @@
# Publish Director - Screen Demo Pipeline
## When To Use
Package the finished demo so the user can publish it quickly and so the metadata reflects the actual task, result, and tools involved.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/publish_log.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["compose"]["render_report"]`, `state.artifacts["idea"]["brief"]`, `state.artifacts["script"]["script"]` | Video, brief, and sections |
| Playbook | Active style playbook | Thumbnail and copy tone |
## Process
### 1. Build Searchable Metadata
Screen-demo titles work best when they combine:
- task,
- tool,
- outcome.
Good patterns:
- `How to deploy on Vercel from Next.js`
- `Fix CORS in React + Express`
- `Set up GitHub Actions for Python tests`
Pull keywords from:
- software names,
- frameworks,
- commands,
- exact error text,
- outcome words such as `deploy`, `fix`, `connect`, `publish`, `ship`.
### 2. Use Chapter Markers As Navigation
Use script sections as the basis for chapter markers and packaging bullets. A good screen-demo package makes the workflow skimmable before the user even presses play.
### 3. Thumbnail Strategy
If a thumbnail concept is needed, it should show:
- the result state, not a generic setup screen,
- the recognizable tool surface,
- 2-4 words of value text.
Store the concept in `publish_log.metadata.thumbnail_concepts`.
### 4. Package By Platform
Prepare:
- video file,
- title and description/caption,
- chapter markers where relevant,
- keyword list,
- thumbnail concept notes.
For developer or product-demo content, also package:
- commands shown,
- software/version mentions,
- error terms if it is a troubleshooting demo.
### 5. Quality Gate
- metadata names the real tool and task,
- chapters match the actual rendered flow,
- export folders are clean and reusable,
- copy is tailored to the platform instead of duplicated.
## Common Pitfalls
- Publishing with generic titles that omit the actual software or task.
- Using the same caption for YouTube, LinkedIn, and short-form social.
- Building chapter markers from the script without checking the render.
@@ -0,0 +1,131 @@
# Scene Director - Screen Demo Pipeline
## When To Use
You are planning how the viewer's attention moves through an existing screen capture. The source video already exists; your job is to decide when to stay wide, when to crop in, and when to add minimal guidance.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/scene_plan.schema.json` | Artifact validation |
| Prior artifacts | `state.artifacts["script"]["script"]`, `state.artifacts["idea"]["brief"]` | Script timing and source notes |
| Tools | `frame_sampler`, `scene_detect` | Extract reference frames and transitions |
| Playbook | Active style playbook | Overlay style and pacing rules |
## Process
### 1. Plan Attention, Not Constant Motion
Modern screen-demo tools converge on click-led zoom because it reduces random camera motion. Use that principle here:
- zoom when the viewer truly needs help reading or locating something,
- stay steady during comprehension,
- zoom manually when the key event has no click anchor,
- reset to a wider context between major steps.
Use `frame_sampler` for exact reference frames around each key action and store detailed crop notes in `scene_plan.metadata`.
### 2. Choose Scene Shapes
Use simple scene types that match the schema:
- `screen_recording` for live UI capture sections
- `text_card` for step labels, title cards, or recap slides
- `diagram` only when the UI alone cannot explain the concept
- `transition` sparingly between major workflow phases
### 3. Plan Crop Strategy In Metadata
The schema does not have first-class zoom objects, so keep the scene descriptions concise and place the detailed crop plan in `scene_plan.metadata.crop_regions`.
Recommended `crop_regions` fields:
- `section_id`
- `start_seconds`
- `end_seconds`
- `region`
- `zoom_level`
- `trigger` (`click_cluster`, `typing`, `result`, `manual_focus`)
- `transition_duration`
- `rationale`
Useful heuristics:
| UI Element | Zoom Level | Region Sizing | Notes |
|------------|-----------|---------------|-------|
| Terminal / code | 1.5-2.2x | keep enough surrounding context to orient the viewer |
| Small button / icon | 2.0-3.0x | show padding so the viewer knows where it lives |
| Modal / dialog | 1.4-2.0x | capture the full modal, not a partial crop |
| Full-page result | 1.0-1.3x | show more context before zooming again |
### 4. Plan Callouts With Restraint
Document overlay needs in `required_assets` and `metadata.callout_plan`.
Use only the patterns that clarify the action:
- `highlight_box`
- `arrow`
- `step_label`
- `keystroke_badge`
- `blur_mask`
Rules:
- no more than two attention cues at once,
- show the cue just before the action,
- remove it quickly after the action,
- prefer highlight or zoom; do not stack both unless readability truly demands it.
### 5. Plan Speed Treatment
Mark repetitive segments as either:
- `speed_up`
- `cut`
- `realtime`
Useful visual treatments for sped-up sections:
- progress label,
- small status text,
- simple dissolve over the removed wait.
Do not invent flashy transition behavior for installs, builds, or long typing stretches.
### 6. Choose Aspect Ratio Per Segment
If the whole project targets multiple outputs, note in metadata which scene crops are viable for:
- `16:9`
- `1:1`
- `9:16`
If a step cannot survive vertical, say so. The correct answer is sometimes to ship landscape only or create a separate simplified vertical cut.
### 7. Quality Gate
**Zoom coherence:**
- [ ] motion is intentional, not constant
- [ ] every zoom exists for legibility or orientation
- [ ] every major section gets a wider re-establishing view
- [ ] crops do not cut off the relevant text or control
**Callout coherence:**
- [ ] every critical action has either a crop or a callout
- [ ] callouts do not obscure the UI
- [ ] subtitle and callout zones do not collide
- [ ] sensitive data has a planned mask
**Pacing coherence:**
- [ ] result moments stay at normal speed
- [ ] waiting and repetitive typing are compressed or removed
- [ ] the output duration still matches the brief
## Common Pitfalls
- Staying zoomed in so long that the viewer loses the interface map.
- Planning vertical crops for wide UI without admitting they fail.
- Adding highlight layers everywhere instead of choosing the single clearest cue.
- Ignoring sensitive data revealed in seemingly minor frames.
@@ -0,0 +1,116 @@
# Script Director - Screen Demo Pipeline
## When To Use
You are turning the inspected recording into a timestamped procedural script. Unlike explainer work, you are not inventing the flow. You are synchronizing language to actions the viewer will literally see.
## Prerequisites
| Layer | Resource | Purpose |
|-------|----------|---------|
| Schema | `schemas/artifacts/script.schema.json` | Artifact validation |
| Prior artifact | `state.artifacts["idea"]["brief"]` | Brief with workflow, critical moments, and source notes |
| Tools | `transcriber`, `frame_sampler`, `audio_enhance` | Audio/transcript inspection and spot checks |
## Process
### 1. Decide The Script Mode
Use the brief metadata to choose one of three modes:
| Voiceover Status | Strategy |
|-----------------|----------|
| `voiced` | Transcribe, tighten, and preserve the speaker's phrasing where possible |
| `silent` | Write text-led or optional TTS-ready narration around the actions |
| `partial` | Transcribe the existing speech and bridge only where necessary |
If the recording is silent and TTS was not available in preflight, do not pretend there will be narration later. Write the script so the video can still work with captions, hook cards, and step labels.
### 2. Build The Action Map
The action map is the real backbone of this stage. Use `frame_sampler` and `transcriber` together to log:
- exact task boundaries,
- clicks worth highlighting,
- typed input worth slowing down,
- waits worth speeding up or cutting,
- the result moment to preserve in real time.
Store detailed action information in `script.metadata.interaction_map`. Keep `sections` clean and schema-valid.
Useful `interaction_map` fields:
- `timestamp_seconds`
- `action_type`
- `target`
- `importance`
- `suggested_treatment` (`realtime`, `speed_up`, `cut`, `highlight`, `zoom`)
### 3. Write Sections By Step
Each `script.sections[]` entry should correspond to a real user step, not a thematic paragraph.
Good section labels:
- `Open the settings panel`
- `Paste the API token`
- `Run the build`
- `Verify the live result`
Every section should do three things:
- say what is happening,
- say why it matters,
- leave clear cues for highlights, zooms, or speed changes.
### 4. Keep The Narration Procedural
Use the research-backed rules:
- narrate intent and effect, not obvious cursor motion,
- keep wording short and direct,
- avoid jargon unless the target audience clearly expects it,
- keep the action on screen synchronized with the wording,
- preserve the speaker's natural voice if the source already has narration.
### 5. Mark Pacing Decisions
Use section-level notes and `metadata.speed_plan` to call out:
| Speed Factor | When to Use | Example |
|-------------|-------------|---------|
| `0.75-1.0x` | Important click or result | Small control, key validation moment |
| `1.5-2.0x` | routine typing or navigation | filling obvious fields |
| `3.0-6.0x` | installs, builds, loading | dependency install, compile |
| `cut` | no learning value | long idle wait |
Do not put critical proof moments inside sped-up sections.
### 6. Use Metadata For Screen-Specific Detail
Recommended `script.metadata` fields:
- `interaction_map`
- `speed_plan`
- `chapter_candidates`
- `pronunciation_guides`
- `callout_candidates`
- `sections_needing_zoom`
### 7. Quality Gate
| Criterion | Question |
|-----------|----------|
| **Action coverage** | Is every critical moment from the brief annotated with a timestamp? |
| **Narration sync** | Does each narration segment align with what's happening on screen? |
| **Speed marking** | Are dead-time segments marked for acceleration or removal? |
| **Enhancement density** | Are highlights reserved for true attention shifts rather than every click? |
| **Technical accuracy** | Are all software names, commands, and UI elements named correctly? |
| **Word economy** | Is narration concise and procedural? |
## Common Pitfalls
- Narrating the cursor instead of the outcome.
- Letting spoken timing drift away from the visual action.
- Keeping builds and loading screens in real time.
- Writing a silent-recording script that secretly depends on unavailable TTS.