Fix gap #22 and add CinematicRenderer captions + music support

- AGENT_GUIDE.md: add "never read source code" rule — skills are the
  interface, not .py files
- animation.yaml: add Layer 2 skill-first guardrail in assets stage
- video-reference-analyst.md: Step 4b now mandates Layer 2 before Layer 3,
  explicitly forbids reading implementation code
- CinematicRenderer: add TikTok-style CaptionOverlay and separate music
  track support (narration + music as independent audio layers)
- cinematic/types.ts: add CinematicCaptionConfig and music prop types
This commit is contained in:
calesthio
2026-04-04 14:23:46 -07:00
parent 370f2f11b7
commit 7b40edb82c
5 changed files with 60 additions and 8 deletions
+4 -2
View File
@@ -580,8 +580,10 @@ OpenMontage has three instruction layers:
Reading order:
1. registry / tool contract — discover what's available
2. relevant pipeline or creative skill — know HOW to use it in this context
3. underlying vendor skill — **mandatory before calling any generation tool**
2. relevant pipeline or creative skill (Layer 2) — know HOW to use it in this context
3. underlying vendor skill (Layer 3)**mandatory before calling any generation tool**
**NEVER read tool source code (`tools/*.py`) to understand how to use a tool.** Skills exist precisely so you don't need implementation details. Layer 2 tells you *what* and *when*. Layer 3 tells you *how*. If you're reading `.py` files to figure out input schemas or provider options, you're doing it wrong — that information belongs in the skill layer.
**Layer 3 is not optional.** Every generation tool (video, image, TTS, music) has an `agent_skills` field listing its Layer 3 skills. These skills contain provider-specific prompt engineering, parameter tuning, and quality techniques. Read them before writing prompts. The difference between a generic prompt and a skill-informed prompt is the difference between "usable" and "cinematic."
+1
View File
@@ -198,6 +198,7 @@ stages:
- "Layer 3 skills read for EVERY generation tool before writing prompts (check agent_skills field)"
- "Clip duration maximized (prefer 10s over 5s) to reduce API calls and cost"
- "Use selector tools (video_selector, tts_selector, image_selector) — never call provider tools directly"
- "Read Layer 2 skills for each tool BEFORE reading source code — skills contain usage guidance, source code is implementation detail"
success_criteria:
- Schema-valid asset_manifest artifact
- All referenced asset files exist on disk
+30 -3
View File
@@ -24,6 +24,7 @@ function resolveAsset(src: string): string {
return staticFile(clean);
}
import { CinematicRendererProps, CinematicTone, CinematicVideoScene } from "./cinematic/types";
import { CaptionOverlay } from "./components/CaptionOverlay";
const FPS = 30;
@@ -339,19 +340,34 @@ export const CinematicRenderer: React.FC<CinematicRendererProps> = ({
titleWidth = 1320,
signalLineCount = 18,
soundtrack,
music,
captions,
}) => {
return (
<AbsoluteFill style={{ backgroundColor: "#000000" }}>
{/* Layer 1: Narration audio */}
{soundtrack ? (
<Soundtrack
src={soundtrack.src}
volume={soundtrack.volume ?? 0.45}
volume={soundtrack.volume ?? 1}
trimBeforeSeconds={soundtrack.trimBeforeSeconds}
trimAfterSeconds={soundtrack.trimAfterSeconds}
fadeInSeconds={soundtrack.fadeInSeconds ?? 1.5}
fadeOutSeconds={soundtrack.fadeOutSeconds ?? 2}
fadeInSeconds={soundtrack.fadeInSeconds ?? 0.3}
fadeOutSeconds={soundtrack.fadeOutSeconds ?? 0.5}
/>
) : null}
{/* Layer 2: Music bed (separate track, ducked) */}
{music ? (
<Soundtrack
src={music.src}
volume={music.volume ?? 0.15}
trimBeforeSeconds={music.trimBeforeSeconds}
trimAfterSeconds={music.trimAfterSeconds}
fadeInSeconds={music.fadeInSeconds ?? 2}
fadeOutSeconds={music.fadeOutSeconds ?? 3}
/>
) : null}
{/* Layer 3: Video scenes */}
{scenes.map((scene) => (
<Sequence
key={scene.id}
@@ -372,6 +388,17 @@ export const CinematicRenderer: React.FC<CinematicRendererProps> = ({
)}
</Sequence>
))}
{/* Layer 4: TikTok-style captions */}
{captions?.words ? (
<CaptionOverlay
words={captions.words}
wordsPerPage={captions.wordsPerPage ?? 5}
fontSize={captions.fontSize ?? 48}
color={captions.color ?? "#F8FAFC"}
highlightColor={captions.highlightColor ?? "#FBBF24"}
backgroundColor={captions.backgroundColor ?? "rgba(0, 0, 0, 0.6)"}
/>
) : null}
</AbsoluteFill>
);
};
+17
View File
@@ -35,6 +35,21 @@ export interface CinematicSoundtrack {
fadeOutSeconds?: number;
}
export interface CinematicWordCaption {
word: string;
startMs: number;
endMs: number;
}
export interface CinematicCaptionConfig {
words: CinematicWordCaption[];
wordsPerPage?: number;
fontSize?: number;
color?: string;
highlightColor?: string;
backgroundColor?: string;
}
export interface CinematicRendererProps {
[key: string]: unknown;
scenes: CinematicScene[];
@@ -42,4 +57,6 @@ export interface CinematicRendererProps {
titleWidth?: number;
signalLineCount?: number;
soundtrack?: CinematicSoundtrack;
music?: CinematicSoundtrack;
captions?: CinematicCaptionConfig;
}
+8 -3
View File
@@ -269,9 +269,14 @@ the user paralyzed with equal choices.
**Before ANY asset generation** (sample or full production), the agent MUST:
1. Check the `agent_skills` field on every tool that will be used
2. Read each referenced skill in `.agents/skills/`
3. Apply the provider-specific prompting guidance to all generation prompts
1. Read the **Layer 2 skill** for each tool from `skills/` directory (usage guidance, input schemas, best practices)
2. Check the `agent_skills` field on every tool that will be used
3. Read each referenced **Layer 3 skill** in `.agents/skills/` (provider-specific prompting)
4. Apply the provider-specific prompting guidance to all generation prompts
**NEVER read tool source code (*.py) to understand how to use a tool.**
Skills exist precisely so the agent doesn't need to read implementation code.
Layer 2 skills describe *what* and *when*. Layer 3 skills describe *how*.
This is NOT optional. The AGENT_GUIDE says: *"Layer 3 is not optional.
Every generation tool has an agent_skills field. Read them before writing