screen-demo: add synthetic-terminal mode via Remotion TerminalScene
Make synthetic screen recording a first-class, discoverable capability alongside real OS capture. For CLI / terminal / install-flow demos where commands and output are predictable, author a `terminal_scene` cut instead of driving a real screen recorder — deterministic, privacy-safe, pixel- perfect, and frame-accurate to narration cues. Components: - TerminalScene.tsx: window chrome, char-by-char typing, blinking cursor, scrolling output, non-blocking floating pills, spring-based reveals - ProviderChip.tsx: rotating badge overlay that cycles through provider names (used in AI-generated-motion scenes) - BackgroundVideoLayer + source_in_seconds + backgroundVideo props on every scene type — supports video-behind-component composition Discovery chain (six layers, so the next agent finds this without reading source code): 1. pipeline_defs/screen-demo.yaml — bump to 2.1, declare production_modes (real_capture, synthetic_terminal) with required_tools, scene_type, and agent_skills pointers 2. skills/pipelines/screen-demo/idea-director.md — mode-selection table at brief time; brief.metadata.production_mode contract 3. skills/pipelines/screen-demo/asset-director.md — reads production_mode and branches asset production (capture+overlays vs steps+narration+ pacing check) 4. .agents/skills/synthetic-screen-recording/SKILL.md — Layer 3 skill with step kinds (cmd/out/pause/pill), pacing rule, and the frozen-terminal failure mode captured from the showcase v3 retune 5. AGENT_GUIDE.md — TerminalScene added to Remotion routing; links SCENE_TYPES.md as the authoritative cut-type registry 6. remotion-composer/SCENE_TYPES.md — new cheat sheet of every cut.type and overlay.type with required fields, plus a "how to add a new scene type" section (candidates: ChatTranscript, EditorScene, PrReview, SlackThread, TicketBoard) Guardrail: - lib/verify_scene_pacing.py — reusable trace() and assert_alignment() helpers that mimic the TerminalScene frame math exactly. Fail loudly before render if steps burn through too fast or leave the scene frozen.
This commit is contained in:
@@ -4,6 +4,23 @@
|
||||
|
||||
This stage produces the minimal but high-leverage assets that make a screen demo easier to follow: subtitles, audio cleanup, reusable overlays, masks, and optional light-weight support cards.
|
||||
|
||||
## Two production modes — pick before generating assets
|
||||
|
||||
**Read the brief's `production_mode` field.** If `idea` didn't set one, decide here:
|
||||
|
||||
| Mode | When | Asset production looks like |
|
||||
|---|---|---|
|
||||
| **`real_capture`** | Real app UI (browser, design tool, IDE with plugins); live behavior; user asked for their own screen recorded | Clean audio + subtitles + callout overlays (arrows, highlight masks) applied on top of the captured MP4 |
|
||||
| **`synthetic_terminal`** | CLI, terminal, install flow, make targets, git/npm commands, `.env` config — anything scriptable | **No capture at all.** Author a `terminal_scene` cut for `video_compose` (Remotion). Commands type char-by-char, output scrolls, pills announce completions. See `.agents/skills/synthetic-screen-recording/SKILL.md`. |
|
||||
|
||||
**Mode selection heuristic:** *"Can I predict every command and its output before shooting?"* If yes → synthetic. If no → real capture.
|
||||
|
||||
For synthetic mode, the asset stage produces:
|
||||
- **narration (tts_selector)** aligned to the exact video-time each command should type
|
||||
- **a `steps` list** (cmd/out/pause/pill primitives) that paces with narration cues
|
||||
- **a pacing verification** via `lib.verify_scene_pacing.assert_alignment(...)` — must pass before render
|
||||
- **no** screen-recorder footage, no callout arrows, no zoom-crop regions (those are real-capture concepts)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Layer | Resource | Purpose |
|
||||
|
||||
@@ -2,9 +2,20 @@
|
||||
|
||||
## When To Use
|
||||
|
||||
Use this pipeline when the source is already a screen recording: product walkthrough, software tutorial, coding demo, browser flow, or troubleshooting capture.
|
||||
Use this pipeline whenever the deliverable is a screen-recording-style demo. There are **two production modes** — pick one in the brief:
|
||||
|
||||
Your job is to turn raw capture into a clear procedural video. The main deliverable at this stage is a schema-valid `brief`, with pipeline-specific detail stored in `brief.metadata`.
|
||||
| Mode | Source material | Pick when |
|
||||
|---|---|---|
|
||||
| **`real_capture`** | An actual screen recording (MP4) captured via `screen_recorder`, `cap_recorder`, or `playwright-recording` | Real app UI, live behavior, browser flows, IDE plugins, user asked for their own screen |
|
||||
| **`synthetic_terminal`** | None — nothing is captured. You author a `terminal_scene` cut for Remotion | CLI / terminal / install flow / make targets / git clone / API key config — anything scriptable where every command and output is predictable |
|
||||
|
||||
**Decision question:** *"Can I predict every command and its output before shooting?"* If yes → synthetic. If no → real capture.
|
||||
|
||||
**Record the mode in `brief.metadata.production_mode`.** The asset-director reads this field to choose between capture+overlay assets vs a `steps` list paced with narration.
|
||||
|
||||
For `synthetic_terminal`, also read `.agents/skills/synthetic-screen-recording/SKILL.md` before proceeding — it encodes the pacing rule that killed an earlier showcase render (commands burned through in 40% of scene time, then terminal froze for the remaining 60%).
|
||||
|
||||
Your job at this stage is to turn the user's request into a clear procedural video plan. The main deliverable is a schema-valid `brief`, with pipeline-specific detail stored in `brief.metadata`.
|
||||
|
||||
## Operating Principles
|
||||
|
||||
|
||||
Reference in New Issue
Block a user