docs(prompting): adopt 5-aspect video specification across skills
Incorporate the structured taxonomy from Lin et al. "Building a Precise Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2). The paper proves prompts structured around five aspects (Subject / Subject Motion / Scene / Spatial Framing / Camera) unlock controllable cinematography in fine-tuned video generation models. Off-the-shelf VLMs already nail subject and scene; the gains live in motion, spatial, and camera, which prompts routinely omit. Universal layer (skills/creative/video-gen-prompting.md, +125 lines): - 5-aspect prompt skeleton replaces flat formula - Camera movements regrouped (translation / rotation / lens-only) with dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations - New primitive tables: camera height, camera angle, POV, lens distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking), playback speed (6 modes), subject transitions - Order-matters and self-contained-prompt rules - Identity anchoring rule for multi-shot - Strict static-shot rule, anti-subjective callout, overlays-not-depth - Per-model word-count guidance Per-model guides (sora, veo, hunyuan, ltx, seedance): - Add the primitives each model honors literally - Word-count sweet spots per model - Strengthen seedance verbatim-identity and subject-transition guidance Pipeline directors (cinematic / explainer / animation scene-director, cinematic / explainer asset-director): - 5-aspect scene-plan checklist (per-pipeline adapted) - Overlays-not-depth callout - Pre / critique / post self-review loop for generation prompts Reviewer (skills/meta/reviewer.md): - CHAI critique-quality rules: accurate / complete / constructive - Critical findings now require a proposed_fix Storytelling, cinematic, broll, video-reference-analyst: - Anti-subjective rule (replace mood adjectives with visual causes) - Camera-intent-per-beat for script writers - POV column in stock-footage query templates - 5-aspect structured output mandatory for reference-video analysis skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.
This commit is contained in:
+16
-3
@@ -6,6 +6,18 @@ After completing any pipeline stage's work — before checkpointing. You are the
|
||||
|
||||
Every stage gets reviewed. No exceptions. The review quality determines whether the final video is worth watching.
|
||||
|
||||
## Critique Quality (CHAI Rules)
|
||||
|
||||
> Findings ≠ critiques. A finding identifies a problem; a critique tells the next stage how to fix it. The CMU/Harvard CHAI study ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2) showed that critique quality, measured on three axes, directly governs downstream output quality. Apply all three to every reviewer pass.
|
||||
>
|
||||
> **Accurate.** Every finding must reference a concrete artifact field, line number, or visible asset frame. Forbid hallucinated criticism — if you cannot point to where the problem is, you are guessing.
|
||||
>
|
||||
> **Complete.** A reviewer pass that catches one mistake while missing a second is worse than scoring "needs another pass" and continuing. If you find one critical issue, scan for the rest of the same class before returning. Pattern-match: where else in this artifact could the same mistake be hiding?
|
||||
>
|
||||
> **Constructive.** Every "critical" finding MUST propose a concrete fix, not just identify the problem. "Caption is wrong" → "Caption says 'man on the right'; the man is on the left of the frame. Replace with 'the man on the left of the frame.'" If you cannot propose a fix, label the finding as "investigation" not "critical."
|
||||
>
|
||||
> Removing any of these three properties measurably hurts pipeline output. The reviewer is the choke point — be rigorous.
|
||||
|
||||
## Protocol
|
||||
|
||||
### Step 1: Load Review Context
|
||||
@@ -27,9 +39,10 @@ First, the non-negotiable check:
|
||||
For each `review_focus` item from the manifest:
|
||||
1. Evaluate the artifact against this specific criterion
|
||||
2. Assign a severity:
|
||||
- **critical** — Must fix before proceeding. The artifact is broken, incomplete, or dangerously wrong.
|
||||
- **suggestion** — Should fix. Improves quality significantly but doesn't block progress.
|
||||
- **nitpick** — Could fix. Minor polish that's nice-to-have.
|
||||
- **critical** — Must fix before proceeding. The artifact is broken, incomplete, or dangerously wrong. **Per CHAI rules, every critical finding MUST carry a `proposed_fix` (concrete replacement text, exact field value, or specific corrective action). A critical finding without a proposed fix is downgraded to `investigation`.**
|
||||
- **suggestion** — Should fix. Improves quality significantly but doesn't block progress. **Suggestions MUST carry a `proposed_change` describing how to improve.**
|
||||
- **nitpick** — Could fix. Minor polish that's nice-to-have. May stand alone without a proposed change.
|
||||
- **investigation** — A real concern but you cannot pinpoint the fix. Surface it for the next round; do not block on it.
|
||||
3. Write a specific, actionable finding (not vague)
|
||||
|
||||
**Good finding:** "Section 3 narration is 180 words for a 10-second window — that's 1080 wpm, impossible to speak. Cut to 25 words."
|
||||
|
||||
@@ -38,7 +38,8 @@ video_analyzer.execute({
|
||||
```
|
||||
|
||||
Read the resulting VideoAnalysisBrief. Before proceeding, present a summary to the
|
||||
user. This is NOT a raw dump. It's a conversational interpretation:
|
||||
user. This is NOT a raw dump. It's a conversational interpretation, and it MUST be
|
||||
structured by the 5 aspects so downstream stages can lift fields directly:
|
||||
|
||||
```
|
||||
"I've watched the video. Here's what I see:
|
||||
@@ -48,12 +49,22 @@ user. This is NOT a raw dump. It's a conversational interpretation:
|
||||
**Structure:** [X scenes over Y seconds, pacing style]
|
||||
**Motion:** [N of M scenes are motion clips / animated stills / static images.
|
||||
This video uses [AI-generated video clips / still images with pan-zoom / a mix].]
|
||||
|
||||
**5-aspect breakdown (per shot or per shot-group):**
|
||||
- Subject: [type, count, attributes; subject transitions across shots: revealing / disappearing / switching / complex-alternating; or N/A]
|
||||
- Subject Motion: [actions in temporal order; interactions; or N/A]
|
||||
- Scene: [overlays (text/graphics) listed separately; POV (drone/OTS/macro/etc.); setting; time of day; dynamics]
|
||||
- Spatial Framing: [shot size; subject position; depth; height-relative; how it changes]
|
||||
- Camera: [playback speed; lens; height; angle; focus/DoF; steadiness; movement]
|
||||
|
||||
**What makes it work:** [2-3 specific things — the hook technique, the pacing,
|
||||
the visual transitions, the narration style]
|
||||
|
||||
Now let me check what I can do with your current setup..."
|
||||
```
|
||||
|
||||
The 5-aspect block above is the **canonical form** that `proposal-director`, `script-director`, and `scene-director` will read. Do not collapse it back into prose — keep the labels.
|
||||
|
||||
**Motion classification is critical.** The VideoAnalysisBrief now includes per-scene
|
||||
`motion_type` ("motion_clip", "animated_still", "static_image") and `flow_variance`.
|
||||
Use this to determine the production approach:
|
||||
@@ -81,6 +92,22 @@ Update the brief's `content_analysis`, `style_profile`, and `replication_guidanc
|
||||
fields with your visual observations. This is where the analysis becomes truly
|
||||
comprehensive — the tools provide structure; your vision provides understanding.
|
||||
|
||||
### 5-Aspect Structured Output (MANDATORY)
|
||||
|
||||
The analyst's report MUST break down the reference video into the **five aspects** from the CMU/Harvard CHAI study (also the canonical structure used in `skills/creative/video-gen-prompting.md`). A narrative-only summary is no longer sufficient — downstream stages (proposal, script, scene-director) ingest the 5-aspect form directly without re-parsing prose.
|
||||
|
||||
**Decision-tree captioning policy.** For each detected shot, walk all five aspects in order:
|
||||
|
||||
> - **Subject:** type, attributes (count, age, role, costume, distinguishing features), multiple-subject disambiguation, transitions across shots (revealing / disappearing / switching / complex-alternating).
|
||||
> - **Subject Motion:** actions in temporal order; group/interaction patterns (parallel, sequential, reactive); locomotion vs gesture vs facial.
|
||||
> - **Scene:** **overlays separately** (text, lower thirds, graphics, watermark — call these out as their own layer, do not merge into setting) + POV (drone, aerial, OTS, macro, top-down, dashcam, FPV, handheld, locked-off) + setting + time of day + dynamics (weather, particles, crowd movement).
|
||||
> - **Spatial Framing:** shot size (ECU/CU/MS/WS/EWS), subject position in frame, depth (foreground/midground/background usage), height-relative (above/at/below subject) — and how each of these **changes** across the shot if the camera or subject moves.
|
||||
> - **Camera:** playback speed (real-time / slow-mo / time-lapse), lens distortion (anamorphic, fish-eye, tilt-shift), height (ground / eye / overhead), angle (high / low / Dutch), focus / DoF (rack focus, deep focus, shallow), steadiness (locked / handheld / gimbal), movement (push / pull / pan / tilt / dolly / truck / crane / orbit).
|
||||
>
|
||||
> **Mark any aspect explicitly as N/A** if it doesn't apply (e.g., "Subject: N/A — pure scenery shot," or "Scene overlays: N/A — no graphics"). **Silent omission is the most common analyst failure** and produces ambiguous downstream prompts.
|
||||
|
||||
See `skills/creative/video-gen-prompting.md` for primitive definitions and the canonical vocabulary used at every aspect.
|
||||
|
||||
### Step 2: Capability Audit
|
||||
|
||||
Run standard preflight:
|
||||
|
||||
Reference in New Issue
Block a user