docmontage: end-tag overlay default + Chirp 3 HD as default TTS voice
End-tag now composited over final body footage (ProRes 4444 with alpha) instead of concatenated as a black card. Updated idea-director, edit-director, compose-director, and pipeline manifest with overlay-first instructions. Default Google TTS voice changed from Neural2-D to Chirp3-HD-Orus across tool code and docs.
This commit is contained in:
calesthio
+4
-1
@@ -213,11 +213,14 @@ Google TTS offers 700+ voices across 50+ languages. Voice names follow the patte
|
||||
|
||||
| Type | Example | Quality | Cost |
|
||||
|------|---------|---------|------|
|
||||
| **Chirp 3 HD** | `en-US-Chirp3-HD-Orus` | **Best (2024, most natural)** | **Mid — default** |
|
||||
| Standard | `en-US-Standard-A` | Good | Cheapest |
|
||||
| WaveNet | `en-US-WaveNet-D` | Very good | Mid |
|
||||
| Neural2 | `en-US-Neural2-D` | Excellent | Mid |
|
||||
| Studio | `en-US-Studio-O` | Professional | Highest |
|
||||
| Journey | `en-US-Journey-D` | Conversational | Mid |
|
||||
| Journey | `en-US-Journey-D` | Conversational (long-form) | Mid |
|
||||
|
||||
**Recommended voices:** `en-US-Chirp3-HD-Orus` (male, rich/cinematic), `en-US-Chirp3-HD-Aoede` (female, warm). These are Google's newest tier — most natural-sounding, uses the v1beta1 endpoint automatically.
|
||||
|
||||
**Languages include:** English (US, UK, AU, IN), Spanish, French, German, Italian, Portuguese, Japanese, Korean, Chinese (Mandarin, Cantonese), Arabic, Hindi, Russian, Dutch, Polish, Turkish, Vietnamese, Thai, Indonesian, and 30+ more.
|
||||
|
||||
|
||||
@@ -55,13 +55,13 @@ stages:
|
||||
- Tone register is ONE value from the fixed list
|
||||
- Duration and shape are concrete
|
||||
- Music plan is present (MANDATORY — silent only if user explicitly opted out)
|
||||
- End-tag plan is present (MANDATORY — one philosophical closing line, rendered as Remotion end-card, unless user explicitly opted out)
|
||||
- End-tag plan is present (MANDATORY — one philosophical closing line, rendered as Remotion end-card, default mode "overlay" on final scenes, unless user explicitly opted out)
|
||||
- Narration plan is present (narration itself is OPTIONAL — absence is fine if music + visuals + end-tag carry the register)
|
||||
success_criteria:
|
||||
- Schema-valid brief artifact
|
||||
- thematic_question present in metadata
|
||||
- music_plan present in metadata (source may be `none` ONLY with explicit user opt-out note)
|
||||
- end_tag_plan present in metadata (text, palette, duration — may be `null` ONLY with explicit user opt-out note)
|
||||
- end_tag_plan present in metadata (text, palette, duration, mode — may be `null` ONLY with explicit user opt-out note)
|
||||
|
||||
- name: scene_plan
|
||||
skill: pipelines/documentary-montage/scene-director
|
||||
@@ -129,8 +129,8 @@ stages:
|
||||
- No two adjacent cuts share subject AND scale
|
||||
- Transition vocabulary is at most 4 distinct values
|
||||
- Music config is present (MANDATORY — silent only if brief.metadata.music_plan.source=none with explicit opt-out note)
|
||||
- End-tag cut is present at the tail (MANDATORY — rendered separately via Remotion, concatenated after body, unless brief.metadata.end_tag_plan is null with explicit opt-out note)
|
||||
- total_duration_seconds matches sum of cut durations INCLUDING the end-tag hold
|
||||
- "End-tag timing is present (MANDATORY — overlay mode: end_tag.offset_seconds in edit_decisions; concat mode: end-tag appended after body. Unless brief.metadata.end_tag_plan is null with explicit opt-out note)"
|
||||
- total_duration_seconds matches sum of cut durations (body only — end-tag overlay does not extend duration; concat mode adds tag duration)
|
||||
- Every cut has a reason
|
||||
success_criteria:
|
||||
- Schema-valid edit_decisions artifact
|
||||
@@ -160,15 +160,16 @@ stages:
|
||||
checkpoint_required: true
|
||||
human_approval_default: false
|
||||
review_focus:
|
||||
- Output duration matches planned within 1s (body + end-tag inclusive)
|
||||
- Output duration matches planned within 1s (body duration for overlay mode; body + tag for concat mode)
|
||||
- Resolution matches target_platform canvas
|
||||
- Uniform LUT applied across the timeline
|
||||
- Music is mixed in (MANDATORY — silent output only with explicit user opt-out recorded in brief)
|
||||
- End-tag MP4 rendered via Remotion and concatenated at the tail (MANDATORY — absence only with explicit opt-out)
|
||||
- First and last frames verified (last frame must be the end-tag card unless opted out)
|
||||
- "End-tag rendered via Remotion (MANDATORY — absence only with explicit opt-out). Default mode: overlay (ProRes 4444 with alpha composited on final scenes). Fallback: concat (opaque card appended after body)."
|
||||
- "Overlay mode: extract a frame from the overlay region and verify text is visible over footage, not over black. Concat mode: last frame must be the end-tag card."
|
||||
- No silent fallback from a motion-led promise
|
||||
success_criteria:
|
||||
- Schema-valid render_report artifact
|
||||
- Output file exists and passes ffprobe validation
|
||||
- render_report.end_tag_rendered = true (or explicit opt-out)
|
||||
- render_report.end_tag_mode = "overlay" or "concat"
|
||||
- render_report.music_mixed = true (or explicit opt-out)
|
||||
|
||||
@@ -220,6 +220,24 @@ export const Root: React.FC = () => {
|
||||
fadeOutSeconds: 0.6,
|
||||
} as EndTagProps}
|
||||
/>
|
||||
<Composition
|
||||
id="EndTagOverlay"
|
||||
component={EndTag}
|
||||
// 8.19s at 30fps = 246 frames. Render CLI can override via --props.
|
||||
// Intended to be composited on top of body footage, not concat'd.
|
||||
durationInFrames={246}
|
||||
fps={30}
|
||||
width={1920}
|
||||
height={1080}
|
||||
defaultProps={{
|
||||
text: "EARN THE LIGHT.",
|
||||
palette: "cool_offwhite_on_black",
|
||||
fadeInSeconds: 1.0,
|
||||
holdSeconds: 5.69,
|
||||
fadeOutSeconds: 1.5,
|
||||
overlay: true,
|
||||
} as EndTagProps}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -13,6 +13,12 @@ export interface EndTagProps {
|
||||
fadeInSeconds?: number;
|
||||
holdSeconds?: number;
|
||||
fadeOutSeconds?: number;
|
||||
// Overlay mode: render on a transparent background so the tag can be
|
||||
// composited on top of the body footage in post, instead of being
|
||||
// concatenated as a standalone black card. When `overlay=true` the
|
||||
// AbsoluteFill drops its background fill — caller is responsible for
|
||||
// rendering with an alpha-capable codec (VP9/WebM or ProRes 4444).
|
||||
overlay?: boolean;
|
||||
}
|
||||
|
||||
const PALETTES = {
|
||||
@@ -49,6 +55,7 @@ export const EndTag: React.FC<EndTagProps> = ({
|
||||
fadeInSeconds = 0.6,
|
||||
holdSeconds = 4.3,
|
||||
fadeOutSeconds = 0.6,
|
||||
overlay = false,
|
||||
}) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps } = useVideoConfig();
|
||||
@@ -113,7 +120,7 @@ export const EndTag: React.FC<EndTagProps> = ({
|
||||
return (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
backgroundColor: pal.background,
|
||||
backgroundColor: overlay ? "transparent" : pal.background,
|
||||
justifyContent: "center",
|
||||
alignItems: "center",
|
||||
}}
|
||||
|
||||
@@ -142,13 +142,12 @@ and L-cut sfx layers. Your job is to execute them faithfully:
|
||||
|
||||
Do NOT add ambient noise "to fill the gap".
|
||||
|
||||
### 4b. Render The End-Tag Via Remotion, Concatenate After Body
|
||||
### 4b. Render The End-Tag Via Remotion
|
||||
|
||||
The end-tag is rendered **separately** from the FFmpeg body and
|
||||
concatenated at the tail. This is deliberate — it sidesteps the
|
||||
`video_compose.render` scene-adapter mismatch and keeps the two
|
||||
render engines (FFmpeg for footage, Remotion for typography) cleanly
|
||||
separated.
|
||||
The end-tag is rendered **separately** from the FFmpeg body via
|
||||
Remotion. This keeps the two render engines (FFmpeg for footage,
|
||||
Remotion for typography) cleanly separated. The compositing method
|
||||
depends on `brief.metadata.end_tag_plan.mode`.
|
||||
|
||||
Read `brief.metadata.end_tag_plan`:
|
||||
|
||||
@@ -158,19 +157,79 @@ Read `brief.metadata.end_tag_plan`:
|
||||
"palette": "warm_ivory_on_black",
|
||||
"duration_seconds": 5.5,
|
||||
"render_engine": "remotion",
|
||||
"component": "EndTag"
|
||||
"component": "EndTag",
|
||||
"mode": "overlay"
|
||||
}
|
||||
```
|
||||
|
||||
Execution path:
|
||||
#### Path A — Overlay Mode (default)
|
||||
|
||||
The tag fades in over the final scenes of the body footage. This is
|
||||
the default and produces a more cinematic result — the typography
|
||||
appears on top of live footage rather than cutting to a black card.
|
||||
|
||||
**Execution:**
|
||||
|
||||
1. Compose the body via FFmpeg (cuts + LUT + music + silence window).
|
||||
Save as `projects/<name>/renders/body.mp4`.
|
||||
2. Render the end-tag via Remotion CLI with component-specific props:
|
||||
`npx remotion render EndTag --props='{"text":"...", "palette":"...","durationInFrames":132}' projects/<name>/renders/end_tag.mp4`
|
||||
Save as `projects/<name>/renders/body.mp4`. Note the body fps.
|
||||
2. Compute `durationInFrames = round(duration_seconds × body_fps)`.
|
||||
3. Render the end-tag with alpha via Remotion CLI:
|
||||
```bash
|
||||
npx remotion render src/index.tsx EndTagOverlay \
|
||||
projects/<name>/renders/end_tag_overlay.mov \
|
||||
--codec=prores --prores-profile=4444 \
|
||||
--pixel-format=yuva444p10le --image-format=png \
|
||||
--props='{"text":"...","palette":"...","overlay":true,
|
||||
"fadeInSeconds":1.0,"holdSeconds":3.0,"fadeOutSeconds":1.5}'
|
||||
```
|
||||
Use the `EndTagOverlay` composition with `overlay: true`. This
|
||||
produces a ProRes 4444 MOV with a real alpha channel
|
||||
(pix_fmt=yuva444p12le). Canvas must match body canvas.
|
||||
4. Compute the overlay offset:
|
||||
- Read `edit_decisions.end_tag.offset_seconds` if present.
|
||||
- Otherwise auto-compute: `offset = body_duration - tag_duration`.
|
||||
The tag's fade-out should align with the body's closing fade-out.
|
||||
5. Composite via FFmpeg overlay with `-itsoffset`:
|
||||
```bash
|
||||
ffmpeg -y \
|
||||
-i body.mp4 \
|
||||
-itsoffset {offset} -i end_tag_overlay.mov \
|
||||
-filter_complex "[0:v][1:v]overlay=0:0:format=auto:eof_action=pass[v]" \
|
||||
-map "[v]" -map "0:a" \
|
||||
-c:v libx264 -preset medium -crf 18 -pix_fmt yuv420p \
|
||||
-c:a aac -b:a 192k \
|
||||
projects/<name>/renders/final.mp4
|
||||
```
|
||||
`eof_action=pass` means the body video continues after the overlay
|
||||
ends. The overlay's own alpha handles the fade-in/hold/fade-out.
|
||||
|
||||
**Verification:** Extract a frame from the overlay region (e.g.
|
||||
`offset + 2s`) and confirm text is visible over footage, not over
|
||||
black. If the frame shows a black background behind the text, the
|
||||
alpha channel was lost — re-render with `--image-format=png`.
|
||||
|
||||
#### Path B — Concat Mode
|
||||
|
||||
Classic tail-card: opaque black card appended after the body. Use
|
||||
this only when `end_tag_plan.mode == "concat"`.
|
||||
|
||||
**Execution:**
|
||||
|
||||
1. Compose the body as above.
|
||||
2. Render the end-tag as opaque MP4:
|
||||
```bash
|
||||
npx remotion render src/index.tsx EndTag \
|
||||
projects/<name>/renders/end_tag.mp4 \
|
||||
--props='{"text":"...","palette":"...","durationInFrames":132}'
|
||||
```
|
||||
(5.5s at 24fps = 132 frames). Canvas must match body canvas.
|
||||
3. Concat body + end_tag with `ffmpeg -f concat -safe 0 -i list.txt -c copy final.mp4`
|
||||
or, if encoders don't match, re-encode with the documentary spec.
|
||||
3. Concat body + end_tag:
|
||||
```bash
|
||||
ffmpeg -f concat -safe 0 -i list.txt -c copy final.mp4
|
||||
```
|
||||
Or re-encode if codecs don't match.
|
||||
|
||||
#### Common Rules (Both Modes)
|
||||
|
||||
**End-tag is MANDATORY.** The ONLY way to skip it is an explicit user
|
||||
opt-out recorded as `end_tag_plan: null` with an `end_tag_opt_out_reason`.
|
||||
@@ -179,7 +238,9 @@ a contract violation. Stop and surface before finalizing.
|
||||
|
||||
Record in `render_report`:
|
||||
- `end_tag_rendered: true | false`
|
||||
- `end_tag_path: "projects/<name>/renders/end_tag.mp4"`
|
||||
- `end_tag_mode: "overlay" | "concat"`
|
||||
- `end_tag_path: "projects/<name>/renders/end_tag_overlay.mov"` (or `.mp4` for concat)
|
||||
- `end_tag_offset_seconds: <number>` (overlay mode only)
|
||||
- `end_tag_text: "..."` (for audit trail)
|
||||
|
||||
If the brief says "no music" and the edit correctly has no music
|
||||
|
||||
@@ -225,6 +225,32 @@ by layering the outgoing clip's audio as an SFX entry in
|
||||
Documentary montages with L-cuts feel 50% more coherent than ones
|
||||
without. Use them on the 3-4 hardest transitions in the piece.
|
||||
|
||||
### 8b. Place The End-Tag Overlay
|
||||
|
||||
If `brief.metadata.end_tag_plan.mode == "overlay"` (the default), the
|
||||
end-tag will be composited on top of the final body footage at compose
|
||||
time. The edit director's job is to decide **when** the tag appears.
|
||||
|
||||
Compute the offset: `offset_seconds = body_duration - tag_duration`.
|
||||
This makes the tag's fade-out align with the body's closing fade-out
|
||||
(the last cut's `transition_out: fade_out`). If the final cut's hold
|
||||
is shorter than the tag duration, start the tag earlier so it overlaps
|
||||
the second-to-last cut as well — this is fine and often looks better.
|
||||
|
||||
Record in `edit_decisions.end_tag`:
|
||||
|
||||
```json
|
||||
{
|
||||
"end_tag": {
|
||||
"offset_seconds": 84.5,
|
||||
"notes": "Tag starts at body_duration - tag_duration. Aligns tag fade-out with final cut fade-out."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If `mode == "concat"`, omit this section — the compose-director will
|
||||
append the tag after the body without needing a timing offset.
|
||||
|
||||
### 9. Emit The Edit Decisions
|
||||
|
||||
Canonical shape for this pipeline:
|
||||
@@ -266,6 +292,10 @@ Canonical shape for this pipeline:
|
||||
"ducking": false
|
||||
}
|
||||
},
|
||||
"end_tag": {
|
||||
"offset_seconds": 84.5,
|
||||
"notes": "Tag starts at body_duration - tag_duration. Aligns tag fade-out with final cut fade-out."
|
||||
},
|
||||
"metadata": {
|
||||
"pipeline": "documentary-montage",
|
||||
"tone": "elegiac",
|
||||
|
||||
@@ -95,7 +95,14 @@ defer this — it becomes an expensive surprise at the asset stage.
|
||||
Every documentary-montage film closes on a philosophical end-tag — one
|
||||
short, abstract line that gives the whole thing meaning. It is rendered
|
||||
as a Remotion end-card ("shining underlined tag" register — bold weight,
|
||||
letter-spaced, animated underline) and concatenated after the last clip.
|
||||
letter-spaced, animated underline).
|
||||
|
||||
**Default mode is `"overlay"`** — the tag fades in over the final scenes
|
||||
of the body footage, so it feels like part of the film rather than a
|
||||
separate card tacked on at the end. The alternative is `"concat"` which
|
||||
appends a standalone black-card after the body. Use concat only when the
|
||||
user explicitly asks for a separated title card, or when the final
|
||||
footage is too visually busy for legible text overlay.
|
||||
|
||||
**End-tag is MANDATORY.** The ONLY way out is an explicit user opt-out
|
||||
recorded as `end_tag_plan: null` with an `end_tag_opt_out_reason` field.
|
||||
@@ -110,12 +117,26 @@ Expected shape:
|
||||
"palette": "warm_ivory_on_black",
|
||||
"duration_seconds": 5.5,
|
||||
"render_engine": "remotion",
|
||||
"component": "EndTag"
|
||||
"component": "EndTag",
|
||||
"mode": "overlay"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Keep the copy to 3-9 words. It must be a thesis, not a summary.
|
||||
Fields:
|
||||
- `text` — 3-9 words. A thesis, not a summary.
|
||||
- `palette` — `"cool_offwhite_on_black"` or `"warm_ivory_on_black"`.
|
||||
- `duration_seconds` — total tag screen time (fade-in + hold + fade-out).
|
||||
5-8s is the sweet spot.
|
||||
- `render_engine` — always `"remotion"`.
|
||||
- `component` — always `"EndTag"`.
|
||||
- `mode` — `"overlay"` (default) or `"concat"`.
|
||||
- **overlay**: tag rendered as ProRes 4444 with alpha → composited on
|
||||
final body footage via FFmpeg overlay filter. Tag fades appear over
|
||||
the last N seconds of live footage. The body's own fade-out and the
|
||||
tag's fade-out should align.
|
||||
- **concat**: tag rendered as opaque MP4 → appended after body via
|
||||
FFmpeg concat. Total output duration = body + tag.
|
||||
|
||||
### 6. Note Narration Intent (OPTIONAL)
|
||||
|
||||
|
||||
@@ -77,8 +77,8 @@ class GoogleTTS(BaseTool):
|
||||
"text": {"type": "string", "description": "Text to convert to speech"},
|
||||
"voice": {
|
||||
"type": "string",
|
||||
"default": "en-US-Neural2-D",
|
||||
"description": "Voice name (e.g. en-US-Neural2-D, en-US-Studio-O, en-GB-WaveNet-A)",
|
||||
"default": "en-US-Chirp3-HD-Orus",
|
||||
"description": "Voice name. Default tier is Chirp 3 HD (2024, most natural). Examples: en-US-Chirp3-HD-Orus (male, rich/cinematic), en-US-Chirp3-HD-Aoede (female, warm). Legacy tiers: en-US-Studio-O, en-US-Neural2-D, en-US-Journey-D.",
|
||||
},
|
||||
"language_code": {
|
||||
"type": "string",
|
||||
@@ -144,7 +144,7 @@ class GoogleTTS(BaseTool):
|
||||
def estimate_cost(self, inputs: dict[str, Any]) -> float:
|
||||
text = inputs.get("text", "")
|
||||
char_count = len(text)
|
||||
voice = inputs.get("voice", "en-US-Neural2-D")
|
||||
voice = inputs.get("voice", "en-US-Chirp3-HD-Orus")
|
||||
# Pricing per million characters (approximate)
|
||||
if "Chirp3-HD" in voice:
|
||||
rate_per_char = 0.000030 # $30/1M chars
|
||||
@@ -180,7 +180,7 @@ class GoogleTTS(BaseTool):
|
||||
import requests
|
||||
|
||||
text = inputs["text"]
|
||||
voice_name = inputs.get("voice", "en-US-Neural2-D")
|
||||
voice_name = inputs.get("voice", "en-US-Chirp3-HD-Orus")
|
||||
language_code = inputs.get("language_code", "en-US")
|
||||
speaking_rate = inputs.get("speaking_rate", 1.0)
|
||||
pitch = inputs.get("pitch", 0.0)
|
||||
|
||||
Reference in New Issue
Block a user