Upgrade Remotion composition engine with cinematic enhancements
- Upgrade Remotion packages to 4.0.441, add transitions/captions/fonts/media - Add spring-animated image scenes, stat reveals, section titles, hero cards - Add TikTok-style word-by-word captions with highlight effect - Add Google Fonts (Space Grotesk) and dynamic duration via calculateMetadata - Fix Remotion false-positive: check node_modules/ in _remotion_available() - Add project directory convention, music library, tool naming docs - Add music transparency to proposal stage, subtitle pipeline to compose stage - Add README showcase montage and Windows npm troubleshooting note
This commit is contained in:
calesthio
@@ -99,11 +99,16 @@ Process asset tasks grouped by tool for efficiency:
|
||||
### Step 5: Generate Music
|
||||
|
||||
1. Read playbook's `audio.music_mood` and `audio.music_volume`
|
||||
2. Generate or select a background track:
|
||||
- **Primary**: `music_gen` (ElevenLabs Music) — custom, costs per generation
|
||||
- **Fallback**: Stock music library (if available)
|
||||
3. Duration should match total video duration (or be loopable)
|
||||
4. Verify the audio file exists
|
||||
2. Check the music decision from `proposal_packet.production_plan.music_source` (set by the Proposal Director)
|
||||
3. Source the background track in this priority order:
|
||||
- **User-selected library track**: If the proposal specified a track from `music_library/`, copy it to `projects/<project>/assets/music/background_music.mp3`
|
||||
- **User music library (`music_library/`)**: If the folder exists and has tracks, pick the best match for the playbook's `audio.music_mood`. List candidates by filename and let the EP decide.
|
||||
- **Music generation API**: Use `music_gen` (ElevenLabs) or `suno_music` if available. Check status via registry first — if the tool is unavailable or quota-exhausted, skip immediately (do NOT attempt and fail silently).
|
||||
- **No music available**: Log this clearly in the asset manifest as `"music_status": "unavailable"` with the reason. Do NOT silently produce a video without music — the EP and user should know.
|
||||
4. Duration should be at least as long as total video duration. If shorter, it can be looped by the compose stage.
|
||||
5. Verify the audio file exists at `projects/<project>/assets/music/background_music.mp3`
|
||||
|
||||
**Critical:** If music generation fails or is unavailable, report it immediately in the asset manifest — do not defer the problem to the compose stage.
|
||||
|
||||
### Step 6: Build Asset Manifest
|
||||
|
||||
|
||||
@@ -95,6 +95,56 @@ Call the `audio_mixer` tool to:
|
||||
|
||||
The video_compose tool will mux this with the video.
|
||||
|
||||
### Step 5b: Generate Subtitles (Mandatory)
|
||||
|
||||
Subtitles are mandatory for all explainer content. Generate them from the narration audio — do NOT skip this step.
|
||||
|
||||
1. **Transcribe** the full narration using the `transcriber` tool (whisperx):
|
||||
```python
|
||||
from tools.analysis.transcriber import Transcriber
|
||||
result = Transcriber().execute({
|
||||
'input_path': 'projects/<project>/assets/audio/narration_full.mp3',
|
||||
'model_size': 'base',
|
||||
'language': 'en',
|
||||
'output_dir': 'projects/<project>/assets/audio'
|
||||
})
|
||||
# result.data contains segments with word-level timestamps
|
||||
```
|
||||
|
||||
2. **Generate SRT** from the transcription using `subtitle_gen`:
|
||||
```python
|
||||
from tools.subtitle.subtitle_gen import SubtitleGen
|
||||
result = SubtitleGen().execute({
|
||||
'segments': transcription_data['segments'],
|
||||
'format': 'srt',
|
||||
'output_path': 'projects/<project>/assets/subtitles.srt',
|
||||
'max_words_per_cue': 8,
|
||||
'max_chars_per_line': 42
|
||||
})
|
||||
```
|
||||
|
||||
3. **Burn subtitles** into the video using `video_compose`:
|
||||
```python
|
||||
from tools.video.video_compose import VideoCompose
|
||||
result = VideoCompose().execute({
|
||||
'operation': 'burn_subtitles',
|
||||
'input_path': 'projects/<project>/renders/output.mp4',
|
||||
'output_path': 'projects/<project>/renders/final.mp4',
|
||||
'subtitle_path': 'projects/<project>/assets/subtitles.srt',
|
||||
'subtitle_style': {
|
||||
'font': '<from playbook typography.headings.font or Arial>',
|
||||
'font_size': 22,
|
||||
'primary_color': '&HFFFFFF',
|
||||
'outline_color': '&H000000',
|
||||
'outline_width': 2,
|
||||
'margin_v': 50,
|
||||
'alignment': 2
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
**The final deliverable is the subtitled version**, not the pre-subtitle render.
|
||||
|
||||
### Step 6: Verify Output
|
||||
|
||||
**File verification:**
|
||||
|
||||
@@ -216,6 +216,43 @@ Also present **alternative production paths** — complete packages at different
|
||||
| Budget | Local TTS + images | ~$0.05-0.15 | Robotic voice, image-only |
|
||||
| Free | Local TTS + diagrams | $0.00 | Functional but minimal |
|
||||
|
||||
### Step 5b: Music Plan (Mandatory)
|
||||
|
||||
Music is a critical part of the video's feel. **Surface the music situation to the user at proposal time** — do not silently defer it to the asset stage where a failure becomes expensive.
|
||||
|
||||
**Check music availability in this order:**
|
||||
|
||||
1. **User music library (`music_library/`):** Check if this folder exists and contains tracks. If so, list available tracks with durations and let the user pick one.
|
||||
2. **Music generation APIs:** Check which music tools are available via the registry (`registry.get_by_capability("music_generation")`). Report their status honestly.
|
||||
3. **Stock music sources:** Note if stock music is available via any provider.
|
||||
|
||||
**Present to the user:**
|
||||
|
||||
```
|
||||
MUSIC PLAN
|
||||
├── Your music library: 3 tracks available
|
||||
│ ├── cosmic_interstellar_space.mp3 (3:13) — ambient, cosmic
|
||||
│ ├── cinematic_epic.mp3 (2:45) — dramatic, building
|
||||
│ └── lofi_beat.mp3 (4:00) — chill, electronic
|
||||
├── AI generation: music_gen (ElevenLabs) — UNAVAILABLE (plan limit)
|
||||
└── Recommendation: Use "cosmic_interstellar_space.mp3" from your library
|
||||
OR provide a different track before asset generation
|
||||
|
||||
Would you like to:
|
||||
(a) Use a track from your library (which one?)
|
||||
(b) Provide a different track (drop it in music_library/)
|
||||
(c) Generate one via API (if available)
|
||||
(d) Proceed without music
|
||||
```
|
||||
|
||||
**If no music source is available:** Tell the user explicitly. Do NOT let this surface as a surprise at the asset stage. Offer the `music_library/` path so they can add a track before production starts.
|
||||
|
||||
**Rules:**
|
||||
- Always check `music_library/` first — user-provided music is free and intentional
|
||||
- Always report music API status (available, unavailable, quota remaining if checkable)
|
||||
- Record the music decision in `proposal_packet.production_plan.music_source`
|
||||
- If the user picks a library track, record its path for the asset director
|
||||
|
||||
### Step 6: Build the Cost Estimate
|
||||
|
||||
Itemize every paid operation:
|
||||
|
||||
Reference in New Issue
Block a user