Harden cinematic pipeline and add reusable renderer
This commit is contained in:
calesthio
@@ -26,6 +26,12 @@ Start with:
|
||||
|
||||
These are the primary materials. Everything else is support.
|
||||
|
||||
If `brief.metadata.motion_required = true`, actual moving footage or generated video clips are mandatory. In that case:
|
||||
|
||||
- stills may be used only as reference material or backing elements inside a larger motion composition,
|
||||
- stills may not replace the planned motion shots,
|
||||
- a still-image teaser is not an acceptable fallback unless the user explicitly approves an animatic.
|
||||
|
||||
### 1b. Sample Preview (Prevents Wasted Spend)
|
||||
|
||||
Before batch-generating support assets, produce one sample of each expensive generated type and show the user:
|
||||
@@ -33,8 +39,20 @@ Before batch-generating support assets, produce one sample of each expensive gen
|
||||
1. **Generated insert sample** (if using `image_selector` or `video_selector`): Generate one representative visual. Confirm it complements the source footage before batching.
|
||||
2. **Music sample** (if using `music_gen`): Generate a short clip. Confirm mood and energy match the beat plan.
|
||||
|
||||
If `motion_required = true`, the representative visual must be a video clip sample, not a still image sample.
|
||||
|
||||
If rejected, adjust parameters and retry (max 3 iterations). Do not batch until approved.
|
||||
|
||||
Before the sample is generated, tell the user exactly which generation path will be used:
|
||||
|
||||
- tool,
|
||||
- provider,
|
||||
- model or variant,
|
||||
- generation mode,
|
||||
- why it was selected.
|
||||
|
||||
If that path fails, stop and ask before trying a different provider, model, or generation mode.
|
||||
|
||||
### 2. Generate Support Assets Only Where Needed
|
||||
|
||||
Optional generated assets should fill clear gaps:
|
||||
@@ -44,6 +62,8 @@ Optional generated assets should fill clear gaps:
|
||||
- texture or atmosphere cards,
|
||||
- simple textural motion backgrounds.
|
||||
|
||||
For motion-required jobs, use `video_selector` first for generated shots. `image_selector` may support look development, concept frames, or embedded design layers, but it does not satisfy the motion requirement by itself.
|
||||
|
||||
### 3. Prepare A Real Audio Plan
|
||||
|
||||
Store:
|
||||
@@ -70,9 +90,12 @@ Recommended metadata keys:
|
||||
- generated inserts are limited and purposeful,
|
||||
- audio plan matches the beat map,
|
||||
- every referenced file exists.
|
||||
- if motion is required, the asset set contains actual video clips for the motion-led beats.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Generating extra shots before proving the source edit works.
|
||||
- Treating music as a single loop instead of a beat-aware element.
|
||||
- Forgetting rights or provenance notes for supplied assets.
|
||||
- Quietly downgrading from video clips to still images because one provider or renderer failed.
|
||||
- Quietly switching providers or models after the user approved a generation path.
|
||||
|
||||
@@ -15,6 +15,15 @@ Render the cinematic piece with careful attention to grade, audio dynamics, and
|
||||
|
||||
## Process
|
||||
|
||||
### 0. Check Hard Requirements Before Rendering
|
||||
|
||||
If the approved brief or scene plan makes motion a hard requirement, verify that the render path still preserves that promise.
|
||||
|
||||
- If Remotion is required and unavailable or failing, stop and bubble the issue to the user immediately.
|
||||
- Do not switch to an FFmpeg-only still-image fallback for a motion-led trailer, teaser, or agent video.
|
||||
- Do not convert the piece into an animatic unless the user explicitly approves that downgrade.
|
||||
- If the render engine changes materially, tell the user before rendering and explain why.
|
||||
|
||||
### 1. Use Frame Treatment Deliberately
|
||||
|
||||
Only use letterbox, 24fps intent, or heavy grading if they help the piece. Do not apply them because the pipeline name says cinematic.
|
||||
@@ -51,3 +60,4 @@ Recommended metadata keys:
|
||||
- Flattening the audio so the piece loses dynamics.
|
||||
- Applying letterbox to footage that needs every pixel.
|
||||
- Letting grading or sharpening damage faces or text.
|
||||
- Silently swapping a blocked Remotion render for a lower-fidelity still-image export.
|
||||
|
||||
@@ -50,6 +50,28 @@ Same as standard EP: Initialize → Execute stages serially (idea → script →
|
||||
|
||||
Each stage: PREPARE → SPAWN DIRECTOR → REVIEW → GATE DECISION (pass / revise / send-back).
|
||||
|
||||
### User-Facing Decision Flow
|
||||
|
||||
For this pipeline, the EP must make the decision trail visible to the user.
|
||||
|
||||
Before any expensive or consequential generation step, present:
|
||||
|
||||
- selected tool,
|
||||
- provider,
|
||||
- model or variant,
|
||||
- why it was chosen,
|
||||
- whether the run is a sample or a batch.
|
||||
|
||||
If the approved path becomes blocked, the EP must stop and present:
|
||||
|
||||
- the attempted path,
|
||||
- the concrete failure,
|
||||
- the likely class of issue (auth, provider access, tool bug, or creative mismatch),
|
||||
- the available next options,
|
||||
- the recommended next option.
|
||||
|
||||
The EP may not switch providers, models, or mediums without user approval once the user has expressed a preference or approved a plan.
|
||||
|
||||
## EP-Specific Cross-Stage Checks
|
||||
|
||||
### After IDEA stage:
|
||||
@@ -57,6 +79,7 @@ Each stage: PREPARE → SPAWN DIRECTOR → REVIEW → GATE DECISION (pass / revi
|
||||
CHECK: Emotional arc definition
|
||||
- Is the emotional arc explicit (build → reveal → landing)?
|
||||
- Is source mode clear (supplied footage vs generated inserts)?
|
||||
- Does the brief explicitly say whether motion is required?
|
||||
- Is the target mood defined and achievable?
|
||||
```
|
||||
|
||||
@@ -88,6 +111,7 @@ CHECK: Visual consistency
|
||||
CHECK: Music/ambience alignment
|
||||
- Does the music beat map align with the script beat map?
|
||||
- Are generated inserts limited and justified?
|
||||
- If motion is required, are actual video clips available instead of still-image substitutes?
|
||||
- Budget gate: 90% threshold warning
|
||||
|
||||
CHECK: Source selects quality
|
||||
@@ -114,6 +138,7 @@ CHECK: Output validation
|
||||
- Color grade applied and consistent
|
||||
- Audio dynamics controlled — dialogue intelligible, music balanced
|
||||
- Letterbox or frame treatment improves (not harms) the output
|
||||
- If motion was required, does the output still satisfy that promise instead of degrading into a still-led animatic?
|
||||
```
|
||||
|
||||
## Quality Gates Summary
|
||||
@@ -145,3 +170,5 @@ CHECK: Output validation
|
||||
- **Overuse of generated inserts**: Source footage should be primary. Generated content fills gaps, not replaces.
|
||||
- **Ignoring audio dynamics**: Cinematic videos live and die by their audio. Music/dialogue balance is critical.
|
||||
- **Rushing the reveal**: The climax moment needs breathing room. Don't let pacing compress it.
|
||||
- **Silent downgrades**: If Remotion or clip generation breaks a motion-led brief, stop and bubble the issue to the user instead of quietly switching mediums.
|
||||
- **Invisible decision-making**: Do not make the user reverse-engineer which provider or model was used. State it before execution and when anything changes.
|
||||
|
||||
@@ -24,6 +24,16 @@ Capture the source mode:
|
||||
- `generated_support`
|
||||
- `mixed_montage`
|
||||
|
||||
Also classify whether the requested delivery is motion-required. Store this as a boolean in `brief.metadata.motion_required`.
|
||||
|
||||
Set `motion_required = true` when the promise of the video depends on moving shots or animated compositions rather than static frames. This includes:
|
||||
|
||||
- sci-fi trailers,
|
||||
- cinematic teasers,
|
||||
- action or hype edits,
|
||||
- agent/avatar outputs,
|
||||
- any concept whose quality depends on generated video clips.
|
||||
|
||||
Do not assume stock, generated b-roll, or music exists unless the user has provided it or the environment can actually make it.
|
||||
|
||||
### 2. Define The Emotional Arc
|
||||
@@ -53,6 +63,7 @@ Store longer planning detail in `brief.metadata`.
|
||||
Recommended metadata keys:
|
||||
|
||||
- `source_mode`
|
||||
- `motion_required`
|
||||
- `delivery_shape`
|
||||
- `emotional_arc`
|
||||
- `anchor_assets`
|
||||
@@ -65,6 +76,12 @@ Recommended metadata keys:
|
||||
|
||||
If the user has weak source media and no generation path, say so. A cinematic result still needs enough visual or audio material to carry mood.
|
||||
|
||||
If `motion_required = true`, be explicit about the motion path:
|
||||
|
||||
- confirm the planned clip-generation providers,
|
||||
- confirm whether Remotion is required for the intended composition,
|
||||
- if either is unavailable or unstable, mark the treatment as blocked rather than silently redesigning it around still images.
|
||||
|
||||
### 5. Music Plan (Mandatory)
|
||||
|
||||
Cinematic videos live and die by their audio. **Surface the music situation before the user approves the brief.**
|
||||
@@ -95,6 +112,7 @@ Record the decision in `brief.metadata.music_strategy` with the chosen source an
|
||||
### 6. Quality Gate
|
||||
|
||||
- the source truth is explicit,
|
||||
- the brief says whether motion is a hard requirement,
|
||||
- the emotional arc is specific,
|
||||
- the output shape fits the available assets,
|
||||
- the music plan is resolved (source chosen or explicitly deferred),
|
||||
@@ -104,4 +122,5 @@ Record the decision in `brief.metadata.music_strategy` with the chosen source an
|
||||
|
||||
- Calling something cinematic when it is really just a normal edit with black bars.
|
||||
- Assuming generated inserts are available without checking tools.
|
||||
- Quietly turning a motion-led brief into a still-led teaser.
|
||||
- Planning a trailer shape with no reveal or payoff.
|
||||
|
||||
Reference in New Issue
Block a user