screen-demo: add synthetic-terminal mode via Remotion TerminalScene

Make synthetic screen recording a first-class, discoverable capability
alongside real OS capture. For CLI / terminal / install-flow demos where
commands and output are predictable, author a `terminal_scene` cut instead
of driving a real screen recorder — deterministic, privacy-safe, pixel-
perfect, and frame-accurate to narration cues.

Components:
- TerminalScene.tsx: window chrome, char-by-char typing, blinking cursor,
  scrolling output, non-blocking floating pills, spring-based reveals
- ProviderChip.tsx: rotating badge overlay that cycles through provider
  names (used in AI-generated-motion scenes)
- BackgroundVideoLayer + source_in_seconds + backgroundVideo props on
  every scene type — supports video-behind-component composition

Discovery chain (six layers, so the next agent finds this without reading
source code):

1. pipeline_defs/screen-demo.yaml — bump to 2.1, declare production_modes
   (real_capture, synthetic_terminal) with required_tools, scene_type, and
   agent_skills pointers
2. skills/pipelines/screen-demo/idea-director.md — mode-selection table
   at brief time; brief.metadata.production_mode contract
3. skills/pipelines/screen-demo/asset-director.md — reads production_mode
   and branches asset production (capture+overlays vs steps+narration+
   pacing check)
4. .agents/skills/synthetic-screen-recording/SKILL.md — Layer 3 skill with
   step kinds (cmd/out/pause/pill), pacing rule, and the frozen-terminal
   failure mode captured from the showcase v3 retune
5. AGENT_GUIDE.md — TerminalScene added to Remotion routing; links
   SCENE_TYPES.md as the authoritative cut-type registry
6. remotion-composer/SCENE_TYPES.md — new cheat sheet of every cut.type
   and overlay.type with required fields, plus a "how to add a new scene
   type" section (candidates: ChatTranscript, EditorScene, PrReview,
   SlackThread, TicketBoard)

Guardrail:
- lib/verify_scene_pacing.py — reusable trace() and assert_alignment()
  helpers that mimic the TerminalScene frame math exactly. Fail loudly
  before render if steps burn through too fast or leave the scene frozen.
This commit is contained in:
calesthio
2026-04-16 19:22:46 -07:00
parent 33ba37704e
commit a36ce99793
11 changed files with 918 additions and 26 deletions
@@ -4,6 +4,23 @@
This stage produces the minimal but high-leverage assets that make a screen demo easier to follow: subtitles, audio cleanup, reusable overlays, masks, and optional light-weight support cards.
## Two production modes — pick before generating assets
**Read the brief's `production_mode` field.** If `idea` didn't set one, decide here:
| Mode | When | Asset production looks like |
|---|---|---|
| **`real_capture`** | Real app UI (browser, design tool, IDE with plugins); live behavior; user asked for their own screen recorded | Clean audio + subtitles + callout overlays (arrows, highlight masks) applied on top of the captured MP4 |
| **`synthetic_terminal`** | CLI, terminal, install flow, make targets, git/npm commands, `.env` config — anything scriptable | **No capture at all.** Author a `terminal_scene` cut for `video_compose` (Remotion). Commands type char-by-char, output scrolls, pills announce completions. See `.agents/skills/synthetic-screen-recording/SKILL.md`. |
**Mode selection heuristic:** *"Can I predict every command and its output before shooting?"* If yes → synthetic. If no → real capture.
For synthetic mode, the asset stage produces:
- **narration (tts_selector)** aligned to the exact video-time each command should type
- **a `steps` list** (cmd/out/pause/pill primitives) that paces with narration cues
- **a pacing verification** via `lib.verify_scene_pacing.assert_alignment(...)` — must pass before render
- **no** screen-recorder footage, no callout arrows, no zoom-crop regions (those are real-capture concepts)
## Prerequisites
| Layer | Resource | Purpose |
+13 -2
View File
@@ -2,9 +2,20 @@
## When To Use
Use this pipeline when the source is already a screen recording: product walkthrough, software tutorial, coding demo, browser flow, or troubleshooting capture.
Use this pipeline whenever the deliverable is a screen-recording-style demo. There are **two production modes** — pick one in the brief:
Your job is to turn raw capture into a clear procedural video. The main deliverable at this stage is a schema-valid `brief`, with pipeline-specific detail stored in `brief.metadata`.
| Mode | Source material | Pick when |
|---|---|---|
| **`real_capture`** | An actual screen recording (MP4) captured via `screen_recorder`, `cap_recorder`, or `playwright-recording` | Real app UI, live behavior, browser flows, IDE plugins, user asked for their own screen |
| **`synthetic_terminal`** | None — nothing is captured. You author a `terminal_scene` cut for Remotion | CLI / terminal / install flow / make targets / git clone / API key config — anything scriptable where every command and output is predictable |
**Decision question:** *"Can I predict every command and its output before shooting?"* If yes → synthetic. If no → real capture.
**Record the mode in `brief.metadata.production_mode`.** The asset-director reads this field to choose between capture+overlay assets vs a `steps` list paced with narration.
For `synthetic_terminal`, also read `.agents/skills/synthetic-screen-recording/SKILL.md` before proceeding — it encodes the pacing rule that killed an earlier showcase render (commands burned through in 40% of scene time, then terminal froze for the remaining 60%).
Your job at this stage is to turn the user's request into a clear procedural video plan. The main deliverable is a schema-valid `brief`, with pipeline-specific detail stored in `brief.metadata`.
## Operating Principles