docs(prompting): adopt 5-aspect video specification across skills
Incorporate the structured taxonomy from Lin et al. "Building a Precise Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2). The paper proves prompts structured around five aspects (Subject / Subject Motion / Scene / Spatial Framing / Camera) unlock controllable cinematography in fine-tuned video generation models. Off-the-shelf VLMs already nail subject and scene; the gains live in motion, spatial, and camera, which prompts routinely omit. Universal layer (skills/creative/video-gen-prompting.md, +125 lines): - 5-aspect prompt skeleton replaces flat formula - Camera movements regrouped (translation / rotation / lens-only) with dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations - New primitive tables: camera height, camera angle, POV, lens distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking), playback speed (6 modes), subject transitions - Order-matters and self-contained-prompt rules - Identity anchoring rule for multi-shot - Strict static-shot rule, anti-subjective callout, overlays-not-depth - Per-model word-count guidance Per-model guides (sora, veo, hunyuan, ltx, seedance): - Add the primitives each model honors literally - Word-count sweet spots per model - Strengthen seedance verbatim-identity and subject-transition guidance Pipeline directors (cinematic / explainer / animation scene-director, cinematic / explainer asset-director): - 5-aspect scene-plan checklist (per-pipeline adapted) - Overlays-not-depth callout - Pre / critique / post self-review loop for generation prompts Reviewer (skills/meta/reviewer.md): - CHAI critique-quality rules: accurate / complete / constructive - Critical findings now require a proposed_fix Storytelling, cinematic, broll, video-reference-analyst: - Anti-subjective rule (replace mood adjectives with visual causes) - Camera-intent-per-beat for script writers - POV column in stock-footage query templates - 5-aspect structured output mandatory for reference-video analysis skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.
This commit is contained in:
@@ -101,6 +101,22 @@ Recommended metadata keys:
|
||||
- `generated_support_assets`
|
||||
- `rights_notes`
|
||||
|
||||
### Pre/Post Self-Review for Generation Prompts
|
||||
|
||||
> Before sending a prompt to any image or video generation tool, run a three-step self-review modeled on the CHAI oversight loop ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2). Cost is small (no extra tool calls); benefit is large (avoids wasted generations). For cinematic, this matters most for **hero-frame prompts** — one bad hero frame ruins the piece, and hero frames are the most expensive shots to regenerate.
|
||||
>
|
||||
> **Step 1 — Pre-caption pass.** Write the prompt the way you'd write it today. Do not over-edit; aim for a complete first draft.
|
||||
>
|
||||
> **Step 2 — Critique pass.** Score the draft against the 5-aspect checklist (Subject / Subject Motion / Scene / Spatial Framing / Camera). For each aspect:
|
||||
> - Is it specified? If not, is the omission deliberate (e.g., "no subject — scenery shot") or accidental?
|
||||
> - Are confusable terms disambiguated? (dolly vs zoom, pan vs truck, bird's-eye vs aerial, fisheye vs barrel, full shot vs close-up)
|
||||
> - Are emotional adjectives ("epic", "moody", "cinematic") replaced with their visual causes (low-key lighting, slow push-in, anamorphic flare, deep shadows)?
|
||||
> - For multi-shot prompts and identity-anchored hero frames: is identity anchored verbatim across shots?
|
||||
>
|
||||
> **Step 3 — Post-caption pass.** Rewrite filling the missing aspects, fixing confusable terms, and replacing subjective language. The post-caption is what gets sent to the generation tool.
|
||||
>
|
||||
> Log the (pre, critique, post) triplet in the asset metadata for traceability. This mirrors the CHAI workflow and creates a record the reviewer can audit.
|
||||
|
||||
### 5. Quality Gate
|
||||
|
||||
- source and support assets are clearly distinguished,
|
||||
|
||||
@@ -49,11 +49,26 @@ Recommended metadata keys:
|
||||
- `title_card_rules`
|
||||
- `support_insert_rules`
|
||||
|
||||
### 5. Quality Gate
|
||||
### 5. 5-Aspect Scene-Plan Checklist
|
||||
|
||||
> Every scene beat — and especially every hero frame — must specify all five aspects. Cinematic relies on a small number of memorable frames; vague hero-frame specs are the single most common failure mode and produce unpredictable model output. Marking an aspect as N/A is allowed but must be explicit (e.g., "no subject — establishing scenery shot"). Silent omission is forbidden.
|
||||
>
|
||||
> 1. **Subject** — type + key visual attributes; if multiple, how to disambiguate. For hero frames, identity must be anchored verbatim across shots.
|
||||
> 2. **Subject Motion** — actions in temporal order; subject↔object / subject↔subject interactions.
|
||||
> 3. **Scene** — overlays (separately!) + POV + setting + time of day + scene dynamics.
|
||||
> 4. **Spatial Framing** — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE across the beat.
|
||||
> 5. **Camera** — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement.
|
||||
>
|
||||
> See `skills/creative/video-gen-prompting.md` for the primitive vocabulary.
|
||||
|
||||
> **Overlays callout.** Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds, name plates, end-tag cards) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (`overlays: [...]`) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output.
|
||||
|
||||
### 6. Quality Gate
|
||||
|
||||
- every beat has a scene treatment,
|
||||
- hero frames are identifiable,
|
||||
- hero frames are identifiable AND fully specified across all 5 aspects,
|
||||
- support inserts are justified,
|
||||
- overlays are recorded under `overlays:`, never inside the depth/framing description,
|
||||
- the visual language stays consistent across the piece.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
Reference in New Issue
Block a user