Animation pipeline: AnimeScene engine, Ghibli-style compositions, audio energy tool, and README showcase
Add anime_scene rendering engine (AnimeScene + ParticleOverlay components) with multi-image crossfade, 9 camera motion types, 5 particle systems, and cinematic lighting overlays. Fix critical Remotion durationInFrames footgun by passing sceneDurationSeconds from parent. Add audio offset/loop support in Explainer for skipping quiet music intros. New tools: audio_energy.py analyzes per-second loudness via ebur128 to find optimal music offset and detect when looping is needed. Update all 6 animation pipeline skills (proposal, scene, asset, compose, executive-producer, remotion.md) with battle-tested image_animation workflow including tool availability scan, FLUX multi-image generation, composition JSON format, pre-render validation, and post-render self-review. Add 3 demo compositions (Candyland, Mori no Seishin, Deep Ocean) and anime-ghibli style playbook. Update README with 3 anime video showcases and animation prompts. Add Animation Pipeline section to PROMPT_GALLERY.md.
This commit is contained in:
calesthio
@@ -88,6 +88,44 @@ Adding `FAL_KEY` to your `.env` unlocks FLUX image generation. These prompts com
|
||||
|
||||
---
|
||||
|
||||
## Animation Pipeline — Anime/Ghibli Style (FAL_KEY, ~$0.15)
|
||||
|
||||
These use the **Animation pipeline** with `image_animation` approach — FLUX-generated still images brought to life through multi-image crossfade, cinematic camera motion, particle overlays, and ambient music. No video generation APIs needed. Each 30-second video costs ~$0.15.
|
||||
|
||||
### Ghibli Fantasy World
|
||||
|
||||
> "Create a 30-second Ghibli-style animated video of a magical floating library in the clouds at golden hour. Books drift between shelves, warm light streams through stained glass windows, and a small cat naps on a reading desk."
|
||||
|
||||
**What you get:** 6 anime scenes with 12 FLUX-generated images, camera motion (zoom, pan, Ken Burns, drift), sparkle and light-ray particles, cinematic vignette, hero title overlay, and auto-sourced ambient music with energy-optimized offset.
|
||||
|
||||
**Estimated time:** 10-15 minutes | **Cost:** ~$0.15
|
||||
|
||||
### Underwater Exploration
|
||||
|
||||
> "Make a 30-second anime-style animation of an underwater temple with bioluminescent coral, ancient ruins covered in sea moss, luminous jellyfish drifting past stone pillars, and shafts of sunlight piercing the deep blue."
|
||||
|
||||
**What you get:** Deep ocean atmosphere with mist and sparkle particles, pan and drift camera motion, blue-green lighting overlays, section title overlays, and oceanic ambient soundtrack.
|
||||
|
||||
**Estimated time:** 10-15 minutes | **Cost:** ~$0.15
|
||||
|
||||
### Seasonal Journey
|
||||
|
||||
> "Create a 30-second Ghibli-style animated video showing the four seasons in a Japanese countryside village — cherry blossoms in spring, fireflies in summer, red maple leaves in autumn, and snow-covered thatched roofs in winter."
|
||||
|
||||
**What you get:** 6 scenes transitioning through seasons with petal, firefly, sparkle, and mist particles matching each season. Warm-to-cool lighting transitions and ambient seasonal soundtrack.
|
||||
|
||||
**Estimated time:** 10-15 minutes | **Cost:** ~$0.15
|
||||
|
||||
### Steampunk Cityscape
|
||||
|
||||
> "Make a 30-second anime-style animation of a steampunk city at dusk — airships floating between brass towers, steam rising from street vents, clockwork birds perching on copper lampposts, and a lone inventor walking home through cobblestone streets."
|
||||
|
||||
**What you get:** Industrial-fantasy atmosphere with mist and sparkle particles, parallax and zoom camera motion, warm amber lighting overlays, and steampunk-ambient soundtrack.
|
||||
|
||||
**Estimated time:** 10-15 minutes | **Cost:** ~$0.15
|
||||
|
||||
---
|
||||
|
||||
## Full Setup Prompts (~$1-$3)
|
||||
|
||||
With video generation (Veo, Kling, Runway) + premium TTS (ElevenLabs) + music (Suno). These produce broadcast-quality content.
|
||||
@@ -150,6 +188,8 @@ With video generation (Veo, Kling, Runway) + premium TTS (ElevenLabs) + music (S
|
||||
|
||||
**Ask for the zero-key path.** If you want free results, say "use only free tools" or "no paid APIs." The agent will route to Piper TTS, stock media, and Remotion-only compositions.
|
||||
|
||||
**For anime/Ghibli-style videos,** mention the style explicitly: "Ghibli-style" or "anime-style." Describe the atmosphere, lighting, and mood. The agent uses the Animation pipeline with FLUX image generation and Remotion's anime scene engine — multi-image crossfade, camera motion, and particle overlays create the illusion of animation from still images. Cost is minimal (~$0.15 for 30 seconds).
|
||||
|
||||
---
|
||||
|
||||
## Contributing Prompts
|
||||
|
||||
@@ -36,10 +36,22 @@ Turn your AI coding assistant into a full video production studio. Describe what
|
||||
> **"VOID — Neural Interface"** — a product ad produced with just one API key (OpenAI). 4 AI-generated images (gpt-image-1), TTS narration, auto-sourced royalty-free music, word-level subtitles via WhisperX, and Remotion data visualizations. Total cost: **$0.69**. Zero manual asset work.
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/08b108e6-c120-4fed-ab21-8549a71f4bd1" width="100%" controls></video>
|
||||
<video src="https://github.com/user-attachments/assets/3c5d7122-7198-43e2-a97d-ed27558dd324" width="100%" controls></video>
|
||||
</div>
|
||||
|
||||
> **"LUMINA — Bioluminescent Terrarium"** — a product ad with zero narration, driven entirely by music, typography, and AI-generated imagery. 5 FLUX-generated images, cinematic ambient soundtrack from Pixabay, animated data visualizations (KPI grids, bar charts, comparison cards), and layered stat reveal overlays. Total cost: **$0.15**. No voiceover, no manual editing.
|
||||
> **"Afternoon in Candyland"** — a Ghibli-style anime animation. A little girl's whimsical afternoon adventure through candy gates, gumdrop rivers, and lollipop gardens. 12 FLUX-generated images with multi-image crossfade, cinematic camera motion (zoom, pan, Ken Burns), sparkle/petal/firefly particle overlays, and ambient music with auto-detected energy offset. Total cost: **$0.15**. No video generation, no manual editing.
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/e8dc5e32-5c70-46de-bd52-eef887719d13" width="100%" controls></video>
|
||||
</div>
|
||||
|
||||
> **"Mori no Seishin"** — a Ghibli-style anime animation of a forest spirit's journey through ancient woods. 12 FLUX-generated images with parallax crossfade, drift and pan camera motion, firefly and petal particles, cinematic vignette lighting, and ambient forest soundtrack. Total cost: **$0.15**. Still images brought to life through Remotion's animation engine.
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/9cf633d9-c264-4961-bfd0-b1db188654aa" width="100%" controls></video>
|
||||
</div>
|
||||
|
||||
> **"Into the Abyss"** — a deep ocean exploration rendered in anime style. Bioluminescent gardens, coral cathedrals, and creatures of light — 12 FLUX-generated images with sparkle and mist particle overlays, light-ray effects, smooth camera motion, and ambient oceanic soundtrack. Total cost: **$0.15**. Zero video generation APIs needed.
|
||||
|
||||
Works with **Claude Code, Cursor, Copilot, Windsurf, Codex** — any AI coding assistant that can read files and run code.
|
||||
|
||||
@@ -142,7 +154,11 @@ Copy any of these into your AI coding assistant after setup. Each one runs a ful
|
||||
|
||||
> "Make a data-driven explainer about coffee consumption around the world"
|
||||
|
||||
### With FAL_KEY (~$0.50–$1.50)
|
||||
### With FAL_KEY (~$0.15–$1.50)
|
||||
|
||||
> "Create a 30-second Ghibli-style animated video of a magical floating library in the clouds at golden hour"
|
||||
|
||||
> "Make a 30-second anime-style animation of an underwater temple with bioluminescent coral and ancient ruins"
|
||||
|
||||
> "Create an animated explainer about how CRISPR gene editing works, using AI-generated visuals"
|
||||
|
||||
|
||||
@@ -35,6 +35,7 @@ compatible_playbooks:
|
||||
- clean-professional
|
||||
- flat-motion-graphics
|
||||
- minimalist-diagram
|
||||
- anime-ghibli
|
||||
|
||||
stages:
|
||||
# ── Pre-Production ──────────────────────────────────────────────
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
{
|
||||
"cuts": [
|
||||
{
|
||||
"id": "scene-1-candy-gate",
|
||||
"source": "",
|
||||
"in_seconds": 0,
|
||||
"out_seconds": 5,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"candyland/scene1-candy-gate-a.png",
|
||||
"candyland/scene1-candy-gate-b.png"
|
||||
],
|
||||
"animation": "zoom-in",
|
||||
"particles": "sparkles",
|
||||
"particleColor": "#FFE082",
|
||||
"particleCount": 22,
|
||||
"particleIntensity": 0.6,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(255,200,100,0.15)",
|
||||
"lightingTo": "transparent"
|
||||
},
|
||||
{
|
||||
"id": "scene-2-candy-forest",
|
||||
"source": "",
|
||||
"in_seconds": 5,
|
||||
"out_seconds": 10,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"candyland/scene2-candy-forest-a.png",
|
||||
"candyland/scene2-candy-forest-b.png"
|
||||
],
|
||||
"animation": "pan-right",
|
||||
"particles": "petals",
|
||||
"particleColor": "#FF6B9D",
|
||||
"particleCount": 18,
|
||||
"particleIntensity": 0.5,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "transparent",
|
||||
"lightingTo": "rgba(255,107,157,0.1)"
|
||||
},
|
||||
{
|
||||
"id": "scene-3-gumdrop-river",
|
||||
"source": "",
|
||||
"in_seconds": 10,
|
||||
"out_seconds": 15,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"candyland/scene3-gumdrop-river-a.png",
|
||||
"candyland/scene3-gumdrop-river-b.png"
|
||||
],
|
||||
"animation": "pan-left",
|
||||
"particles": "sparkles",
|
||||
"particleColor": "#87CEEB",
|
||||
"particleCount": 15,
|
||||
"particleIntensity": 0.4,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(100,200,255,0.1)",
|
||||
"lightingTo": "transparent"
|
||||
},
|
||||
{
|
||||
"id": "scene-4-lollipop-garden",
|
||||
"source": "",
|
||||
"in_seconds": 15,
|
||||
"out_seconds": 20,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"candyland/scene4-lollipop-garden-a.png",
|
||||
"candyland/scene4-lollipop-garden-b.png"
|
||||
],
|
||||
"animation": "drift-up",
|
||||
"particles": "fireflies",
|
||||
"particleColor": "#A8E6CF",
|
||||
"particleCount": 20,
|
||||
"particleIntensity": 0.55,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "transparent",
|
||||
"lightingTo": "rgba(168,230,207,0.12)"
|
||||
},
|
||||
{
|
||||
"id": "scene-5-chocolate-mountain",
|
||||
"source": "",
|
||||
"in_seconds": 20,
|
||||
"out_seconds": 25,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"candyland/scene5-chocolate-mountain-a.png",
|
||||
"candyland/scene5-chocolate-mountain-b.png"
|
||||
],
|
||||
"animation": "zoom-out",
|
||||
"particles": "mist",
|
||||
"particleColor": "#DDA0DD",
|
||||
"particleCount": 5,
|
||||
"particleIntensity": 0.35,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(160,100,200,0.08)",
|
||||
"lightingTo": "transparent"
|
||||
},
|
||||
{
|
||||
"id": "scene-6-sunset-farewell",
|
||||
"source": "",
|
||||
"in_seconds": 25,
|
||||
"out_seconds": 30,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"candyland/scene6-sunset-farewell-a.png",
|
||||
"candyland/scene6-sunset-farewell-b.png"
|
||||
],
|
||||
"animation": "ken-burns",
|
||||
"particles": "sparkles",
|
||||
"particleColor": "#FFB347",
|
||||
"particleCount": 28,
|
||||
"particleIntensity": 0.7,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(255,180,71,0.12)",
|
||||
"lightingTo": "rgba(255,107,157,0.1)"
|
||||
}
|
||||
],
|
||||
"overlays": [
|
||||
{
|
||||
"type": "hero_title",
|
||||
"in_seconds": 0.5,
|
||||
"out_seconds": 4.5,
|
||||
"text": "Afternoon in Candyland",
|
||||
"subtitle": "A whimsical journey"
|
||||
},
|
||||
{
|
||||
"type": "section_title",
|
||||
"in_seconds": 25.5,
|
||||
"out_seconds": 29.5,
|
||||
"text": "Every sweet dream begins here...",
|
||||
"accentColor": "#FFB347",
|
||||
"position": "bottom-center"
|
||||
}
|
||||
],
|
||||
"audio": {
|
||||
"music": {
|
||||
"src": "candyland/ambient-music.mp3",
|
||||
"volume": 0.15,
|
||||
"fadeInSeconds": 2,
|
||||
"fadeOutSeconds": 3,
|
||||
"offsetSeconds": 43,
|
||||
"loop": false
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
{
|
||||
"cuts": [
|
||||
{
|
||||
"id": "scene-1-abyss-awakens",
|
||||
"source": "",
|
||||
"in_seconds": 0,
|
||||
"out_seconds": 5,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"deep-ocean/test-scene1-abyss.png",
|
||||
"deep-ocean/scene1-abyss-b.png"
|
||||
],
|
||||
"animation": "zoom-in",
|
||||
"particles": "sparkles",
|
||||
"particleColor": "#00E5FF",
|
||||
"particleCount": 20,
|
||||
"particleIntensity": 0.5,
|
||||
"backgroundColor": "#020810",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(0,229,255,0.08)",
|
||||
"lightingTo": "transparent"
|
||||
},
|
||||
{
|
||||
"id": "scene-2-bioluminescent-garden",
|
||||
"source": "",
|
||||
"in_seconds": 5,
|
||||
"out_seconds": 10,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"deep-ocean/scene2-garden-a.png",
|
||||
"deep-ocean/scene2-garden-b.png"
|
||||
],
|
||||
"animation": "pan-right",
|
||||
"particles": "fireflies",
|
||||
"particleColor": "#A8E6CF",
|
||||
"particleCount": 25,
|
||||
"particleIntensity": 0.6,
|
||||
"backgroundColor": "#020810",
|
||||
"vignette": true
|
||||
},
|
||||
{
|
||||
"id": "scene-3-coral-cathedral",
|
||||
"source": "",
|
||||
"in_seconds": 10,
|
||||
"out_seconds": 15,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"deep-ocean/scene3-coral-a.png",
|
||||
"deep-ocean/scene3-coral-b.png"
|
||||
],
|
||||
"animation": "drift-up",
|
||||
"particles": "mist",
|
||||
"particleColor": "#FFFFFF",
|
||||
"particleCount": 5,
|
||||
"particleIntensity": 0.4,
|
||||
"backgroundColor": "#020810",
|
||||
"vignette": true,
|
||||
"lightingFrom": "transparent",
|
||||
"lightingTo": "rgba(168,230,207,0.10)"
|
||||
},
|
||||
{
|
||||
"id": "scene-4-creatures-of-light",
|
||||
"source": "",
|
||||
"in_seconds": 15,
|
||||
"out_seconds": 20,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"deep-ocean/scene4-creatures-a.png",
|
||||
"deep-ocean/scene4-creatures-b.png"
|
||||
],
|
||||
"animation": "pan-left",
|
||||
"particles": "sparkles",
|
||||
"particleColor": "#FFB347",
|
||||
"particleCount": 18,
|
||||
"particleIntensity": 0.5,
|
||||
"backgroundColor": "#020810",
|
||||
"vignette": true
|
||||
},
|
||||
{
|
||||
"id": "scene-5-the-deep-trench",
|
||||
"source": "",
|
||||
"in_seconds": 20,
|
||||
"out_seconds": 25,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"deep-ocean/scene5-trench-a.png",
|
||||
"deep-ocean/scene5-trench-b.png"
|
||||
],
|
||||
"animation": "zoom-out",
|
||||
"particles": "light-rays",
|
||||
"particleColor": "#4FC3F7",
|
||||
"particleCount": 3,
|
||||
"particleIntensity": 0.35,
|
||||
"backgroundColor": "#020810",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(79,195,247,0.08)",
|
||||
"lightingTo": "rgba(0,0,0,0.2)"
|
||||
},
|
||||
{
|
||||
"id": "scene-6-return-to-light",
|
||||
"source": "",
|
||||
"in_seconds": 25,
|
||||
"out_seconds": 30,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"deep-ocean/scene6-ascent-a.png",
|
||||
"deep-ocean/scene6-ascent-b.png"
|
||||
],
|
||||
"animation": "ken-burns",
|
||||
"particles": "fireflies",
|
||||
"particleColor": "#FFE082",
|
||||
"particleCount": 22,
|
||||
"particleIntensity": 0.65,
|
||||
"backgroundColor": "#020810",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(0,100,200,0.08)",
|
||||
"lightingTo": "rgba(255,224,130,0.12)"
|
||||
}
|
||||
],
|
||||
"overlays": [
|
||||
{
|
||||
"type": "hero_title",
|
||||
"in_seconds": 0.5,
|
||||
"out_seconds": 4.5,
|
||||
"text": "Into the Abyss",
|
||||
"subtitle": "Where Light Is Born from Darkness"
|
||||
},
|
||||
{
|
||||
"type": "section_title",
|
||||
"in_seconds": 10.5,
|
||||
"out_seconds": 14.0,
|
||||
"text": "Ancient Beyond Measure",
|
||||
"accentColor": "#A8E6CF",
|
||||
"position": "bottom-center"
|
||||
},
|
||||
{
|
||||
"type": "section_title",
|
||||
"in_seconds": 25.5,
|
||||
"out_seconds": 29.5,
|
||||
"text": "The deep remembers.",
|
||||
"accentColor": "#FFE082",
|
||||
"position": "bottom-center"
|
||||
}
|
||||
],
|
||||
"audio": {
|
||||
"music": {
|
||||
"src": "deep-ocean/ambient-ocean.mp3",
|
||||
"volume": 0.15,
|
||||
"fadeInSeconds": 2,
|
||||
"fadeOutSeconds": 3,
|
||||
"offsetSeconds": 55
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
{
|
||||
"cuts": [
|
||||
{
|
||||
"id": "scene-1-awakening",
|
||||
"source": "",
|
||||
"in_seconds": 0,
|
||||
"out_seconds": 5,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"mori-no-seishin/test-scene1-awakening.png",
|
||||
"mori-no-seishin/scene1-awakening-b.png"
|
||||
],
|
||||
"animation": "zoom-in",
|
||||
"particles": "fireflies",
|
||||
"particleColor": "#FFE082",
|
||||
"particleCount": 18,
|
||||
"particleIntensity": 0.5,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true
|
||||
},
|
||||
{
|
||||
"id": "scene-2-gate",
|
||||
"source": "",
|
||||
"in_seconds": 5,
|
||||
"out_seconds": 10,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"mori-no-seishin/scene2-gate-a.png",
|
||||
"mori-no-seishin/scene2-gate-b.png"
|
||||
],
|
||||
"animation": "pan-right",
|
||||
"particles": "mist",
|
||||
"particleColor": "#FFFFFF",
|
||||
"particleCount": 4,
|
||||
"particleIntensity": 0.5,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(255,200,100,0.15)",
|
||||
"lightingTo": "transparent"
|
||||
},
|
||||
{
|
||||
"id": "scene-3-stream",
|
||||
"source": "",
|
||||
"in_seconds": 10,
|
||||
"out_seconds": 15,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"mori-no-seishin/scene3-stream-a.png",
|
||||
"mori-no-seishin/scene3-stream-b.png"
|
||||
],
|
||||
"animation": "pan-left",
|
||||
"particles": "sparkles",
|
||||
"particleColor": "#87CEEB",
|
||||
"particleCount": 15,
|
||||
"particleIntensity": 0.4,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true
|
||||
},
|
||||
{
|
||||
"id": "scene-4-canopy",
|
||||
"source": "",
|
||||
"in_seconds": 15,
|
||||
"out_seconds": 20,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"mori-no-seishin/scene4-canopy-a.png",
|
||||
"mori-no-seishin/scene4-canopy-b.png"
|
||||
],
|
||||
"animation": "drift-up",
|
||||
"particles": "fireflies",
|
||||
"particleColor": "#FFF3B0",
|
||||
"particleCount": 25,
|
||||
"particleIntensity": 0.6,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "transparent",
|
||||
"lightingTo": "rgba(255,243,176,0.12)"
|
||||
},
|
||||
{
|
||||
"id": "scene-5-overlook",
|
||||
"source": "",
|
||||
"in_seconds": 20,
|
||||
"out_seconds": 25,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"mori-no-seishin/scene5-overlook-a.png",
|
||||
"mori-no-seishin/scene5-overlook-b.png"
|
||||
],
|
||||
"animation": "zoom-out",
|
||||
"particles": "light-rays",
|
||||
"particleColor": "#FFE082",
|
||||
"particleCount": 3,
|
||||
"particleIntensity": 0.35,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(255,180,71,0.1)",
|
||||
"lightingTo": "rgba(255,107,157,0.08)"
|
||||
},
|
||||
{
|
||||
"id": "scene-6-dance",
|
||||
"source": "",
|
||||
"in_seconds": 25,
|
||||
"out_seconds": 30,
|
||||
"type": "anime_scene",
|
||||
"images": [
|
||||
"mori-no-seishin/scene6-dance-a.png",
|
||||
"mori-no-seishin/scene6-dance-b.png"
|
||||
],
|
||||
"animation": "ken-burns",
|
||||
"particles": "sparkles",
|
||||
"particleColor": "#FFB347",
|
||||
"particleCount": 30,
|
||||
"particleIntensity": 0.7,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(255,180,71,0.08)",
|
||||
"lightingTo": "rgba(100,80,200,0.1)"
|
||||
}
|
||||
],
|
||||
"overlays": [
|
||||
{
|
||||
"type": "hero_title",
|
||||
"in_seconds": 0.5,
|
||||
"out_seconds": 4.5,
|
||||
"text": "Mori no Seishin",
|
||||
"subtitle": "Spirit of the Forest"
|
||||
},
|
||||
{
|
||||
"type": "section_title",
|
||||
"in_seconds": 25.5,
|
||||
"out_seconds": 29.5,
|
||||
"text": "They were never alone.",
|
||||
"accentColor": "#FFB347",
|
||||
"position": "bottom-center"
|
||||
}
|
||||
],
|
||||
"audio": {
|
||||
"music": {
|
||||
"src": "mori-no-seishin/ambient-piano.mp3",
|
||||
"volume": 0.18,
|
||||
"fadeInSeconds": 2,
|
||||
"fadeOutSeconds": 3,
|
||||
"offsetSeconds": 56
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -34,6 +34,9 @@ import { CaptionOverlay, WordCaption } from "./components/CaptionOverlay";
|
||||
import { SectionTitle } from "./components/SectionTitle";
|
||||
import { StatReveal } from "./components/StatReveal";
|
||||
import { HeroTitle } from "./components/HeroTitle";
|
||||
import { AnimeScene } from "./components/AnimeScene";
|
||||
import type { CameraMotion } from "./components/AnimeScene";
|
||||
import type { ParticleType } from "./components/ParticleOverlay";
|
||||
|
||||
// Load Space Grotesk font for cinematic typography
|
||||
const { fontFamily } = loadFont("normal", {
|
||||
@@ -102,6 +105,15 @@ interface Cut {
|
||||
scale?: number;
|
||||
position?: string | { x: number; y: number };
|
||||
};
|
||||
// Anime scene props (type: "anime_scene")
|
||||
images?: string[];
|
||||
particles?: ParticleType;
|
||||
particleColor?: string;
|
||||
particleCount?: number;
|
||||
particleIntensity?: number;
|
||||
vignette?: boolean;
|
||||
lightingFrom?: string;
|
||||
lightingTo?: string;
|
||||
}
|
||||
|
||||
interface Overlay {
|
||||
@@ -124,6 +136,11 @@ interface AudioConfig {
|
||||
music?: AudioLayer & {
|
||||
fadeInSeconds?: number;
|
||||
fadeOutSeconds?: number;
|
||||
/** Start playback from this offset in seconds (skip quiet intros).
|
||||
* Use the audio_energy tool to find the optimal offset. */
|
||||
offsetSeconds?: number;
|
||||
/** Loop the music if it's shorter than the video duration. */
|
||||
loop?: boolean;
|
||||
};
|
||||
}
|
||||
|
||||
@@ -439,6 +456,25 @@ const SceneRenderer: React.FC<{ cut: Cut }> = ({ cut }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// --- Anime scene (multi-image crossfade + particles) ---
|
||||
if (cut.type === "anime_scene" && cut.images && cut.images.length > 0) {
|
||||
return (
|
||||
<AnimeScene
|
||||
images={cut.images}
|
||||
animation={(cut.animation as CameraMotion) || "ken-burns"}
|
||||
particles={cut.particles}
|
||||
particleColor={cut.particleColor}
|
||||
particleCount={cut.particleCount}
|
||||
particleIntensity={cut.particleIntensity}
|
||||
backgroundColor={cut.backgroundColor}
|
||||
vignette={cut.vignette ?? true}
|
||||
lightingFrom={cut.lightingFrom}
|
||||
lightingTo={cut.lightingTo}
|
||||
sceneDurationSeconds={cut.out_seconds - cut.in_seconds}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
// --- Media types (image / video fallback) ---
|
||||
const animation = cut.animation || cut.transform?.animation;
|
||||
|
||||
@@ -546,10 +582,13 @@ export const Explainer: React.FC<ExplainerProps> = ({
|
||||
<Audio src={resolveAsset(audio.narration.src)} volume={audio.narration.volume ?? 1} />
|
||||
)}
|
||||
|
||||
{/* Layer 4: Audio — music with fade in/out */}
|
||||
{/* Layer 4: Audio — music with offset, fade in/out, and optional loop */}
|
||||
{audio?.music?.src && (
|
||||
<Audio
|
||||
src={resolveAsset(audio.music.src)}
|
||||
startFrom={Math.round((audio.music.offsetSeconds ?? 0) * fps)}
|
||||
loop={audio.music.loop ?? false}
|
||||
loopVolumeCurveBehavior="repeat"
|
||||
volume={(f) => {
|
||||
const baseVol = audio.music!.volume ?? 0.1;
|
||||
const fadeInDur = (audio.music!.fadeInSeconds ?? 2) * fps;
|
||||
|
||||
@@ -0,0 +1,303 @@
|
||||
import {
|
||||
AbsoluteFill,
|
||||
Img,
|
||||
interpolate,
|
||||
spring,
|
||||
staticFile,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
} from "remotion";
|
||||
import { ParticleOverlay, type ParticleType } from "./ParticleOverlay";
|
||||
|
||||
/**
|
||||
* Resolve asset path — use staticFile() for local paths, passthrough URLs.
|
||||
* Duplicated from Explainer.tsx to keep the component self-contained.
|
||||
*/
|
||||
function resolveAsset(src: string): string {
|
||||
if (
|
||||
src.startsWith("http://") ||
|
||||
src.startsWith("https://") ||
|
||||
src.startsWith("data:")
|
||||
) {
|
||||
return src;
|
||||
}
|
||||
const clean = src.replace(/^file:\/\/\/?/, "");
|
||||
return staticFile(clean);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type CameraMotion =
|
||||
| "zoom-in"
|
||||
| "zoom-out"
|
||||
| "pan-left"
|
||||
| "pan-right"
|
||||
| "ken-burns"
|
||||
| "drift-up"
|
||||
| "drift-down"
|
||||
| "parallax"
|
||||
| "static";
|
||||
|
||||
export interface AnimeSceneProps {
|
||||
/** Array of 1-4 image paths — crossfaded sequentially within the scene */
|
||||
images: string[];
|
||||
/** Camera motion applied to all image layers */
|
||||
animation?: CameraMotion;
|
||||
/** Particle effect overlay */
|
||||
particles?: ParticleType;
|
||||
/** Particle color (default: warm yellow) */
|
||||
particleColor?: string;
|
||||
/** Number of particles (default: 20) */
|
||||
particleCount?: number;
|
||||
/** Particle opacity multiplier 0-1 (default: 0.6) */
|
||||
particleIntensity?: number;
|
||||
/** Scene background color behind images (default: dark navy) */
|
||||
backgroundColor?: string;
|
||||
/** Show cinematic vignette (default: true) */
|
||||
vignette?: boolean;
|
||||
/** Starting gradient color for animated lighting shift */
|
||||
lightingFrom?: string;
|
||||
/** Ending gradient color for animated lighting shift */
|
||||
lightingTo?: string;
|
||||
/**
|
||||
* Actual scene duration in seconds.
|
||||
* CRITICAL: useVideoConfig().durationInFrames returns the FULL composition
|
||||
* duration, not the Sequence duration. This prop provides the real scene
|
||||
* length so crossfade/camera/lighting calculations use the correct range.
|
||||
*/
|
||||
sceneDurationSeconds?: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cinematic vignette — slightly stronger than the Explainer default
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const AnimeVignette: React.FC = () => (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
background:
|
||||
"radial-gradient(ellipse at center, transparent 35%, rgba(0,0,0,0.6) 100%)",
|
||||
pointerEvents: "none",
|
||||
}}
|
||||
/>
|
||||
);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Camera motion calculator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function useCameraMotion(animation: CameraMotion, effectiveDuration: number) {
|
||||
const frame = useCurrentFrame();
|
||||
|
||||
const progress = interpolate(frame, [0, effectiveDuration], [0, 1], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
|
||||
let scale = 1;
|
||||
let translateX = 0;
|
||||
let translateY = 0;
|
||||
|
||||
switch (animation) {
|
||||
case "zoom-in":
|
||||
scale = 1 + progress * 0.15;
|
||||
break;
|
||||
case "zoom-out":
|
||||
scale = 1.15 - progress * 0.15;
|
||||
break;
|
||||
case "pan-left":
|
||||
translateX = interpolate(progress, [0, 1], [35, -35]);
|
||||
scale = 1.12;
|
||||
break;
|
||||
case "pan-right":
|
||||
translateX = interpolate(progress, [0, 1], [-35, 35]);
|
||||
scale = 1.12;
|
||||
break;
|
||||
case "ken-burns":
|
||||
scale = 1 + progress * 0.18;
|
||||
translateX = interpolate(progress, [0, 1], [0, -22]);
|
||||
translateY = interpolate(progress, [0, 1], [0, -14]);
|
||||
break;
|
||||
case "drift-up":
|
||||
translateY = interpolate(progress, [0, 1], [22, -22]);
|
||||
scale = 1.1;
|
||||
break;
|
||||
case "drift-down":
|
||||
translateY = interpolate(progress, [0, 1], [-22, 22]);
|
||||
scale = 1.1;
|
||||
break;
|
||||
case "parallax":
|
||||
translateY = interpolate(progress, [0, 1], [14, -14]);
|
||||
translateX = interpolate(progress, [0, 1], [6, -6]);
|
||||
scale = 1.12;
|
||||
break;
|
||||
case "static":
|
||||
default:
|
||||
scale = 1.02; // tiny scale to avoid edge artifacts
|
||||
break;
|
||||
}
|
||||
|
||||
return { scale, translateX, translateY };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Main component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const AnimeScene: React.FC<AnimeSceneProps> = ({
|
||||
images,
|
||||
animation = "ken-burns",
|
||||
particles,
|
||||
particleColor = "#FFE082",
|
||||
particleCount = 20,
|
||||
particleIntensity = 0.6,
|
||||
backgroundColor = "#0A0A1A",
|
||||
vignette = true,
|
||||
lightingFrom,
|
||||
lightingTo,
|
||||
sceneDurationSeconds,
|
||||
}) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
// CRITICAL FIX: useVideoConfig().durationInFrames returns the FULL
|
||||
// composition duration (e.g. 930 for a 31s video), NOT the Sequence
|
||||
// duration (e.g. 150 for a 5s scene). This caused multi-image crossfade
|
||||
// segments to span the wrong range, making images invisible.
|
||||
const effectiveDuration = sceneDurationSeconds
|
||||
? Math.round(sceneDurationSeconds * fps)
|
||||
: durationInFrames;
|
||||
|
||||
const { scale, translateX, translateY } = useCameraMotion(
|
||||
animation,
|
||||
effectiveDuration
|
||||
);
|
||||
|
||||
const imageCount = images.length;
|
||||
|
||||
// Cross-fade duration in frames (~1.2 seconds)
|
||||
const crossfadeDur = Math.round(fps * 1.2);
|
||||
|
||||
/**
|
||||
* Compute opacity for image at index `idx`.
|
||||
*
|
||||
* Single image → simple spring fade-in, gentle fade-out at end.
|
||||
* Multi-image → each image fades in at its segment start and fades out
|
||||
* as the next image fades in. Creates a continuous morph
|
||||
* that simulates subtle motion within the scene.
|
||||
*/
|
||||
const getOpacity = (idx: number): number => {
|
||||
// Scene-level fade-in (first 0.5s) and fade-out (last 0.3s)
|
||||
const sceneIn = spring({
|
||||
frame,
|
||||
fps,
|
||||
config: { damping: 18, stiffness: 80 },
|
||||
});
|
||||
const sceneOut = interpolate(
|
||||
frame,
|
||||
[effectiveDuration - 10, effectiveDuration],
|
||||
[1, 0.25],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
|
||||
if (imageCount <= 1) {
|
||||
return sceneIn * sceneOut;
|
||||
}
|
||||
|
||||
// Each image owns a time segment; crossfade regions OVERLAP so there's
|
||||
// never a gap where both images are at zero opacity.
|
||||
//
|
||||
// Segment boundaries: [0, segDur, 2*segDur, ...]
|
||||
// Image N fades OUT over [segEnd - xfade, segEnd]
|
||||
// Image N+1 fades IN over [segEnd - xfade, segEnd] (same window!)
|
||||
//
|
||||
// This ensures a smooth blend at every boundary.
|
||||
const segmentDur = effectiveDuration / imageCount;
|
||||
const segStart = idx * segmentDur;
|
||||
const segEnd = segStart + segmentDur;
|
||||
|
||||
// Fade in — first image uses spring, others overlap with prev image's fade-out
|
||||
const fadeIn =
|
||||
idx === 0
|
||||
? sceneIn
|
||||
: interpolate(
|
||||
frame,
|
||||
[segStart - crossfadeDur, segStart],
|
||||
[0, 1],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
|
||||
// Fade out — last image uses scene-level fade; others fade as next blends in
|
||||
const fadeOut =
|
||||
idx === imageCount - 1
|
||||
? sceneOut
|
||||
: interpolate(
|
||||
frame,
|
||||
[segEnd - crossfadeDur, segEnd],
|
||||
[1, 0],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
|
||||
return Math.max(0, Math.min(1, fadeIn * fadeOut));
|
||||
};
|
||||
|
||||
// Lighting shift progress
|
||||
const lightProgress = interpolate(frame, [0, effectiveDuration], [0, 1], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
const lightOpacity =
|
||||
lightingFrom && lightingTo
|
||||
? interpolate(lightProgress, [0, 0.3, 0.7, 1], [0, 0.25, 0.25, 0.1], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
})
|
||||
: 0;
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ overflow: "hidden", background: backgroundColor }}>
|
||||
{/* Layer 1: Image stack with crossfade + camera motion */}
|
||||
{images.map((src, i) => (
|
||||
<AbsoluteFill key={i}>
|
||||
<Img
|
||||
src={resolveAsset(src)}
|
||||
style={{
|
||||
width: "100%",
|
||||
height: "100%",
|
||||
objectFit: "cover",
|
||||
opacity: getOpacity(i),
|
||||
transform: `scale(${scale}) translate(${translateX}px, ${translateY}px)`,
|
||||
willChange: "transform, opacity",
|
||||
}}
|
||||
/>
|
||||
</AbsoluteFill>
|
||||
))}
|
||||
|
||||
{/* Layer 2: Animated lighting gradient */}
|
||||
{lightingFrom && lightingTo && (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
background: `linear-gradient(135deg, ${lightingFrom}, ${lightingTo})`,
|
||||
opacity: lightOpacity,
|
||||
pointerEvents: "none",
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Layer 3: Cinematic vignette */}
|
||||
{vignette && <AnimeVignette />}
|
||||
|
||||
{/* Layer 4: Particle effects */}
|
||||
{particles && (
|
||||
<ParticleOverlay
|
||||
type={particles}
|
||||
count={particleCount}
|
||||
color={particleColor}
|
||||
intensity={particleIntensity}
|
||||
/>
|
||||
)}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,349 @@
|
||||
import {
|
||||
AbsoluteFill,
|
||||
interpolate,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
} from "remotion";
|
||||
|
||||
/**
|
||||
* Deterministic pseudo-random based on seed index.
|
||||
* Produces the same value every frame for the same seed — required for Remotion.
|
||||
*/
|
||||
function seededRandom(seed: number): number {
|
||||
const x = Math.sin(seed * 12.9898 + seed * 78.233) * 43758.5453;
|
||||
return x - Math.floor(x);
|
||||
}
|
||||
|
||||
export type ParticleType =
|
||||
| "fireflies"
|
||||
| "petals"
|
||||
| "sparkles"
|
||||
| "mist"
|
||||
| "light-rays";
|
||||
|
||||
interface ParticleOverlayProps {
|
||||
type: ParticleType;
|
||||
count?: number;
|
||||
color?: string;
|
||||
intensity?: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fireflies — glowing dots on sine-wave paths with pulsing opacity
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const Fireflies: React.FC<{
|
||||
count: number;
|
||||
color: string;
|
||||
intensity: number;
|
||||
}> = ({ count, color, intensity }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
const globalFadeIn = interpolate(frame, [0, fps * 0.8], [0, 1], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
const globalFadeOut = interpolate(
|
||||
frame,
|
||||
[durationInFrames - fps * 0.5, durationInFrames],
|
||||
[1, 0],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ pointerEvents: "none" }}>
|
||||
{Array.from({ length: count }, (_, i) => {
|
||||
const baseX = seededRandom(i * 7 + 1) * 90 + 5;
|
||||
const baseY = seededRandom(i * 13 + 2) * 80 + 10;
|
||||
const speed = 0.4 + seededRandom(i * 3 + 5) * 1.2;
|
||||
const phase = seededRandom(i * 11 + 3) * Math.PI * 2;
|
||||
const size = 3 + seededRandom(i * 17 + 4) * 7;
|
||||
|
||||
const t = (frame / fps) * speed;
|
||||
const xOffset = Math.sin(t + phase) * 25;
|
||||
const yOffset = Math.cos(t * 0.7 + phase) * 18;
|
||||
const glowPulse = 0.3 + (Math.sin(t * 2.5 + phase) * 0.35 + 0.35);
|
||||
|
||||
return (
|
||||
<div
|
||||
key={i}
|
||||
style={{
|
||||
position: "absolute",
|
||||
left: `calc(${baseX}% + ${xOffset}px)`,
|
||||
top: `calc(${baseY}% + ${yOffset}px)`,
|
||||
width: size,
|
||||
height: size,
|
||||
borderRadius: "50%",
|
||||
backgroundColor: color,
|
||||
opacity: glowPulse * intensity * globalFadeIn * globalFadeOut,
|
||||
boxShadow: `0 0 ${size * 3}px ${size * 1.5}px ${color}`,
|
||||
}}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Petals — elliptical shapes drifting diagonally with rotation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const Petals: React.FC<{
|
||||
count: number;
|
||||
color: string;
|
||||
intensity: number;
|
||||
}> = ({ count, color, intensity }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
const globalFadeOut = interpolate(
|
||||
frame,
|
||||
[durationInFrames - fps * 0.5, durationInFrames],
|
||||
[1, 0],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ pointerEvents: "none", overflow: "hidden" }}>
|
||||
{Array.from({ length: count }, (_, i) => {
|
||||
const startX = seededRandom(i * 7 + 1) * 110 - 5;
|
||||
const speed = 0.3 + seededRandom(i * 3 + 5) * 0.5;
|
||||
const phase = seededRandom(i * 11 + 3) * Math.PI * 2;
|
||||
const size = 5 + seededRandom(i * 17 + 4) * 9;
|
||||
const delay = seededRandom(i * 19 + 6) * durationInFrames * 0.6;
|
||||
|
||||
const elapsed = Math.max(0, frame - delay);
|
||||
const t = (elapsed / fps) * speed;
|
||||
|
||||
const x = startX + Math.sin(t * 1.3 + phase) * 12 + t * 8;
|
||||
const y = -5 + t * 35;
|
||||
const rotation = t * 50 + phase * 57.3;
|
||||
|
||||
const fadeIn = interpolate(
|
||||
frame,
|
||||
[delay, delay + fps * 0.4],
|
||||
[0, 1],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
|
||||
if (y > 110) return null;
|
||||
|
||||
return (
|
||||
<div
|
||||
key={i}
|
||||
style={{
|
||||
position: "absolute",
|
||||
left: `${x}%`,
|
||||
top: `${y}%`,
|
||||
width: size,
|
||||
height: size * 0.55,
|
||||
borderRadius: "50%",
|
||||
backgroundColor: color,
|
||||
opacity: fadeIn * globalFadeOut * intensity * 0.75,
|
||||
transform: `rotate(${rotation}deg)`,
|
||||
}}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Sparkles — brief cross-shaped flashes at staggered timings
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const Sparkles: React.FC<{
|
||||
count: number;
|
||||
color: string;
|
||||
intensity: number;
|
||||
}> = ({ count, color, intensity }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ pointerEvents: "none" }}>
|
||||
{Array.from({ length: count }, (_, i) => {
|
||||
const x = seededRandom(i * 7 + 1) * 90 + 5;
|
||||
const y = seededRandom(i * 13 + 2) * 85 + 5;
|
||||
const size = 6 + seededRandom(i * 17 + 4) * 10;
|
||||
|
||||
const cycleLen = Math.round(fps * (1.2 + seededRandom(i * 23 + 8) * 2));
|
||||
const offset = Math.round(seededRandom(i * 29 + 9) * durationInFrames);
|
||||
const cycleFrame =
|
||||
((frame - offset) % cycleLen + cycleLen) % cycleLen;
|
||||
|
||||
const sparkleAlpha = interpolate(
|
||||
cycleFrame,
|
||||
[0, cycleLen * 0.15, cycleLen * 0.4, cycleLen],
|
||||
[0, 1, 0.2, 0],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
const sparkleScale = interpolate(
|
||||
cycleFrame,
|
||||
[0, cycleLen * 0.25, cycleLen],
|
||||
[0.3, 1, 0.6],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
|
||||
return (
|
||||
<div
|
||||
key={i}
|
||||
style={{
|
||||
position: "absolute",
|
||||
left: `${x}%`,
|
||||
top: `${y}%`,
|
||||
width: size,
|
||||
height: size,
|
||||
opacity: sparkleAlpha * intensity,
|
||||
transform: `scale(${sparkleScale}) rotate(45deg)`,
|
||||
}}
|
||||
>
|
||||
<div
|
||||
style={{
|
||||
position: "absolute",
|
||||
width: "100%",
|
||||
height: 2,
|
||||
top: "50%",
|
||||
marginTop: -1,
|
||||
backgroundColor: color,
|
||||
borderRadius: 1,
|
||||
boxShadow: `0 0 ${size * 0.8}px ${color}`,
|
||||
}}
|
||||
/>
|
||||
<div
|
||||
style={{
|
||||
position: "absolute",
|
||||
width: 2,
|
||||
height: "100%",
|
||||
left: "50%",
|
||||
marginLeft: -1,
|
||||
backgroundColor: color,
|
||||
borderRadius: 1,
|
||||
boxShadow: `0 0 ${size * 0.8}px ${color}`,
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mist — translucent gradient layers drifting horizontally
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const Mist: React.FC<{
|
||||
count: number;
|
||||
color: string;
|
||||
intensity: number;
|
||||
}> = ({ count, color, intensity }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps } = useVideoConfig();
|
||||
|
||||
const layers = Math.min(count, 5);
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ pointerEvents: "none", overflow: "hidden" }}>
|
||||
{Array.from({ length: layers }, (_, i) => {
|
||||
const baseY = 55 + seededRandom(i * 7 + 1) * 35;
|
||||
const speed = 0.8 + seededRandom(i * 13 + 2) * 1.2;
|
||||
const xDrift = ((frame / fps) * speed * 3) % 200 - 50;
|
||||
const pulse = 0.08 + Math.sin(frame / fps * 0.4 + i * 1.8) * 0.05;
|
||||
|
||||
return (
|
||||
<div
|
||||
key={i}
|
||||
style={{
|
||||
position: "absolute",
|
||||
left: `${xDrift}%`,
|
||||
top: `${baseY}%`,
|
||||
width: "160%",
|
||||
height: "25%",
|
||||
background: `radial-gradient(ellipse at center, rgba(255,255,255,${pulse}) 0%, transparent 70%)`,
|
||||
opacity: intensity,
|
||||
}}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Light Rays — angled gradient beams with gentle pulsing
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const LightRays: React.FC<{
|
||||
count: number;
|
||||
color: string;
|
||||
intensity: number;
|
||||
}> = ({ count, color, intensity }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
const rays = Math.min(count, 5);
|
||||
|
||||
const globalFadeIn = interpolate(frame, [0, fps * 1.2], [0, 1], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ pointerEvents: "none", overflow: "hidden" }}>
|
||||
{Array.from({ length: rays }, (_, i) => {
|
||||
const angle = -35 + seededRandom(i * 7 + 1) * 25;
|
||||
const xPos = 15 + seededRandom(i * 13 + 2) * 65;
|
||||
const beamWidth = 4 + seededRandom(i * 17 + 3) * 8;
|
||||
const pulse =
|
||||
0.06 + Math.sin(frame / fps * 0.6 + i * 2.2) * 0.04;
|
||||
|
||||
return (
|
||||
<div
|
||||
key={i}
|
||||
style={{
|
||||
position: "absolute",
|
||||
left: `${xPos}%`,
|
||||
top: "-10%",
|
||||
width: `${beamWidth}%`,
|
||||
height: "120%",
|
||||
background: `linear-gradient(180deg, rgba(255,255,240,${pulse}) 0%, transparent 80%)`,
|
||||
transform: `rotate(${angle}deg)`,
|
||||
transformOrigin: "top center",
|
||||
opacity: intensity * globalFadeIn,
|
||||
}}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Main export — dispatches to the right particle renderer
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const ParticleOverlay: React.FC<ParticleOverlayProps> = ({
|
||||
type,
|
||||
count = 20,
|
||||
color = "#FFE082",
|
||||
intensity = 0.6,
|
||||
}) => {
|
||||
switch (type) {
|
||||
case "fireflies":
|
||||
return <Fireflies count={count} color={color} intensity={intensity} />;
|
||||
case "petals":
|
||||
return <Petals count={count} color={color} intensity={intensity} />;
|
||||
case "sparkles":
|
||||
return <Sparkles count={count} color={color} intensity={intensity} />;
|
||||
case "mist":
|
||||
return <Mist count={count} color={color} intensity={intensity} />;
|
||||
case "light-rays":
|
||||
return <LightRays count={count} color={color} intensity={intensity} />;
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
};
|
||||
@@ -8,3 +8,7 @@ export { CaptionOverlay } from "./CaptionOverlay";
|
||||
export { SectionTitle } from "./SectionTitle";
|
||||
export { StatReveal } from "./StatReveal";
|
||||
export { HeroTitle } from "./HeroTitle";
|
||||
export { ParticleOverlay } from "./ParticleOverlay";
|
||||
export { AnimeScene } from "./AnimeScene";
|
||||
export type { ParticleType } from "./ParticleOverlay";
|
||||
export type { CameraMotion, AnimeSceneProps } from "./AnimeScene";
|
||||
|
||||
+47
-9
@@ -46,9 +46,26 @@ The Explainer composition supports the following cut types:
|
||||
| `pie_chart` | `chartData` [{label, value}], optional `donut`, `centerLabel` | Proportions, breakdowns |
|
||||
| `kpi_grid` | `chartData` [{label, value, prefix, suffix, change, icon}] | Dashboards, traction metrics |
|
||||
| `progress_bar` | `progress` (0-100), optional `progressSegments` | Journey viz, completion, stacked metrics |
|
||||
| `anime_scene` | `images` (1-4 paths), optional `animation`, `particles`, `particleColor`, `particleCount`, `particleIntensity`, `vignette`, `lightingFrom`, `lightingTo` | Anime/Ghibli-style scenes with multi-image crossfade, camera motion, particle overlays |
|
||||
|
||||
**Chart animations:** `grow-up`, `slide-in`, `pop` (bar), `draw`, `fade-in` (line), `spin`, `expand`, `sequential` (pie), `count-up`, `pop`, `cascade` (kpi)
|
||||
|
||||
### Anime Scene — Multi-Image Crossfade + Particles
|
||||
|
||||
The `anime_scene` type renders 1-4 images with smooth crossfade transitions, cinematic camera motion, and animated particle overlays. This creates the illusion of animation from still images.
|
||||
|
||||
**Camera motion types:** `zoom-in`, `zoom-out`, `pan-left`, `pan-right`, `ken-burns`, `drift-up`, `drift-down`, `parallax`, `static`
|
||||
|
||||
**Particle types:** `fireflies` (floating golden orbs), `petals` (falling cherry blossoms), `sparkles` (twinkling stars), `mist` (drifting fog layers), `light-rays` (crepuscular rays)
|
||||
|
||||
**Key prop:** `sceneDurationSeconds` is automatically passed by `SceneRenderer` — this fixes a critical Remotion pitfall where `useVideoConfig().durationInFrames` returns the full composition duration, not the scene's Sequence duration.
|
||||
|
||||
**Multi-image crossfade math:** Each image owns an equal time segment. Fade-out of image N and fade-in of image N+1 OVERLAP by `crossfadeDur` (~1.2s) so there's never a dead frame. Generate 2-3 images per scene with same style prefix + different seeds for subtle motion effect.
|
||||
|
||||
**Reference composition:** `remotion-composer/public/demo-props/mori-no-seishin.json` — 6 anime scenes, 30 seconds, with particles, lighting, overlays, and ambient music.
|
||||
|
||||
**Style playbook:** `styles/anime-ghibli.yaml` — Ghibli-inspired aesthetic with color palette, typography, motion parameters, and FLUX prompt prefix.
|
||||
|
||||
**Zero-key video strategy:** When no image or video generation is available, build
|
||||
entire videos from these component types. A well-composed sequence of hero_title →
|
||||
kpi_grid → bar_chart → comparison → stat_card → text_card produces a polished,
|
||||
@@ -153,19 +170,21 @@ remotion-composer/
|
||||
The orchestrator calls Remotion renders via CLI:
|
||||
|
||||
```bash
|
||||
# Standard render
|
||||
npx remotion render src/index.ts ExplainerVideo \
|
||||
--props='{"scenes": [...], "theme": "clean_professional"}' \
|
||||
--output=pipeline/<project>/output/final_output.mp4 \
|
||||
--codec=h264
|
||||
# Standard render (composition name is "Explainer", no entry point needed)
|
||||
npx remotion render Explainer \
|
||||
--props="public/demo-props/my-video.json" \
|
||||
--output=output/final.mp4 \
|
||||
--codec=h264 --crf=18
|
||||
|
||||
# With specific media profile
|
||||
npx remotion render src/index.ts ExplainerVideo \
|
||||
npx remotion render Explainer \
|
||||
--width=1080 --height=1920 --fps=30 \
|
||||
--props=props.json \
|
||||
--props="public/demo-props/my-video.json" \
|
||||
--output=output.mp4
|
||||
```
|
||||
|
||||
**Note:** Do NOT specify `src/index.ts` as entry point — Remotion auto-discovers compositions. The composition name is `Explainer` (not `ExplainerVideo`).
|
||||
|
||||
In Python, invoke via `subprocess` from `video_compose.py` when `backend="remotion"`.
|
||||
|
||||
### Media Profile Mapping
|
||||
@@ -249,12 +268,30 @@ const cleanProfessional = {
|
||||
|
||||
### Audio Layering
|
||||
|
||||
Narration + background music + SFX as parallel `<Audio>` components:
|
||||
Narration + background music + SFX as parallel `<Audio>` components.
|
||||
|
||||
**Music offset and looping:** The `audio.music` config supports:
|
||||
- `offsetSeconds` — skip quiet intros, start from the energetic part of the track. Use `tools/analysis/audio_energy.py` to find the optimal offset automatically.
|
||||
- `loop` — loop the music if it's shorter than the video. Remotion handles this natively.
|
||||
- `fadeInSeconds` / `fadeOutSeconds` — smooth volume ramps at start/end.
|
||||
|
||||
```json
|
||||
"audio": {
|
||||
"music": {
|
||||
"src": "project/music.mp3",
|
||||
"volume": 0.15,
|
||||
"offsetSeconds": 55,
|
||||
"loop": false,
|
||||
"fadeInSeconds": 2,
|
||||
"fadeOutSeconds": 3
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```tsx
|
||||
<AbsoluteFill>
|
||||
<Audio src={narrationUrl} />
|
||||
<Audio src={musicUrl} volume={0.06} />
|
||||
<Audio src={musicUrl} volume={0.06} startFrom={offsetFrames} loop />
|
||||
{sfxCues.map(cue => (
|
||||
<Sequence key={cue.id} from={secondsToFrames(cue.time)}>
|
||||
<Audio src={cue.url} volume={cue.volume} />
|
||||
@@ -276,6 +313,7 @@ Remotion renders are CPU-intensive but $0 API cost. Track via cost_tracker:
|
||||
- **No CSS animations or transitions** — they don't render correctly. Use `useCurrentFrame()` + `interpolate()` for all motion.
|
||||
- **No Tailwind animation classes** — `animate-*` classes break frame-based rendering. Static Tailwind utilities are fine.
|
||||
- **Always clamp interpolate()** — use `extrapolateLeft: 'clamp', extrapolateRight: 'clamp'` to prevent values shooting past endpoints.
|
||||
- **`useVideoConfig().durationInFrames` returns COMPOSITION duration, not Sequence duration** — This is the #1 Remotion footgun. If your composition is 31s (930 frames) and a scene's `<Sequence>` is 5s (150 frames), `durationInFrames` still returns 930 inside that scene. Any crossfade, camera motion, or timing logic that uses `durationInFrames` directly will be wildly wrong. **Fix:** Pass `sceneDurationSeconds` as a prop from the parent and compute `effectiveDuration = Math.round(sceneDurationSeconds * fps)` inside the component. The `AnimeScene` component implements this pattern.
|
||||
- **Node.js 18+ required** — listed as optional in minimum system, required in recommended.
|
||||
- **Render in series, not parallel** — unless the machine has enough RAM. Each render spawns a Chromium instance.
|
||||
|
||||
|
||||
@@ -33,6 +33,38 @@ Before batch-generating assets, produce one sample of each expensive type and sh
|
||||
|
||||
If rejected, adjust parameters and retry (max 3 iterations). Do not batch until approved.
|
||||
|
||||
### 1c. Multi-Image Generation for Image-Based Animation (Approach A)
|
||||
|
||||
When `animation_mode == "image_animation"`, each scene needs **2-3 images** for crossfade animation. This is what makes stills look like movement.
|
||||
|
||||
**Image generation workflow:**
|
||||
|
||||
1. **Define a STYLE_PREFIX** — a consistent prompt prefix used across ALL images in the project. This ensures visual coherence. Store it as a reusable asset.
|
||||
```
|
||||
Example: "Studio Ghibli anime style, hand-painted watercolor aesthetic,
|
||||
soft diffused lighting, lush natural environment, warm color palette,
|
||||
painterly brushstrokes visible, high detail..."
|
||||
```
|
||||
|
||||
2. **Use seed management** — for each scene, use nearby seed values (e.g., seed 100 and 101) for the A/B variants. Same prompt + different seed = same composition with subtle differences = natural crossfade motion.
|
||||
|
||||
3. **Generate one test image first** — render a single scene to verify the style prefix produces good results at 1920×1080 before batch generating all images.
|
||||
|
||||
4. **Batch generation** — generate all scene images. Skip any that already exist on disk (idempotent).
|
||||
|
||||
5. **Composition JSON** — each scene gets `type: "anime_scene"` with `images: ["path/a.png", "path/b.png"]` plus camera motion, particle type, and lighting config.
|
||||
|
||||
**Cost estimation:** 2-3 images per scene × $0.03-0.13/image depending on provider.
|
||||
|
||||
**Reference:** See `projects/mori-no-seishin/generate_images.py` for the proven batch generation pattern.
|
||||
|
||||
6. **Copy to Remotion public directory** — After generating all images, copy them to `remotion-composer/public/<project-name>/` so Remotion can access them via `staticFile()`. Image paths in the composition JSON are relative to this directory:
|
||||
```
|
||||
remotion-composer/public/<project-name>/scene1-a.png ← Remotion reads from here
|
||||
remotion-composer/public/<project-name>/ambient-music.mp3 ← Music too
|
||||
```
|
||||
**If you skip this step, the render will fail with missing file errors.** This is the #1 cause of render failures for new projects.
|
||||
|
||||
### 2. Build Reusable Systems
|
||||
|
||||
Create once:
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## When To Use
|
||||
|
||||
Render the animation with an emphasis on text sharpness, timing integrity, and consistent output cadence.
|
||||
Render the animation with an emphasis on text sharpness, timing integrity, and consistent output cadence. For `image_animation` approach, this stage also includes building the composition JSON, sourcing music, running pre-render validation, and performing post-render self-review.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -11,27 +11,196 @@ Render the animation with an emphasis on text sharpness, timing integrity, and c
|
||||
| Schema | `schemas/artifacts/render_report.schema.json` | Artifact validation |
|
||||
| Prior artifacts | `state.artifacts["edit"]["edit_decisions"]`, `state.artifacts["assets"]["asset_manifest"]` | Timing plan and asset files |
|
||||
| Tools | `video_compose`, `audio_mixer`, `video_stitch` | Final assembly |
|
||||
| Tools | `composition_validator` | Pre-render validation (MANDATORY) |
|
||||
| Tools | `audio_probe` | Music duration check |
|
||||
| Playbook | Active style playbook | Render consistency |
|
||||
| Reference | `remotion-composer/public/demo-props/mori-no-seishin.json` | Composition JSON format reference |
|
||||
| Reference | `skills/core/remotion.md` | Remotion patterns, anime_scene type, critical constraints |
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Preserve Motion Timing
|
||||
### 1. Ensure Assets Are in Remotion's Public Directory
|
||||
|
||||
**CRITICAL:** Remotion can only access files via `staticFile()`, which resolves from `remotion-composer/public/`. Generated images and music files MUST be copied or symlinked into this directory before rendering.
|
||||
|
||||
```
|
||||
Project structure:
|
||||
projects/<name>/assets/images/*.png ← where images were generated
|
||||
remotion-composer/public/<name>/*.png ← where Remotion reads them
|
||||
|
||||
Required: Copy or symlink images AND music into public/<project-name>/
|
||||
```
|
||||
|
||||
Image paths in the composition JSON are relative to `remotion-composer/public/`:
|
||||
```json
|
||||
"images": ["deep-ocean/scene1-a.png", "deep-ocean/scene1-b.png"]
|
||||
"src": "deep-ocean/ambient-music.mp3"
|
||||
```
|
||||
|
||||
**If you skip this step, the render will fail with missing file errors or produce black frames.**
|
||||
|
||||
### 2. Build the Composition JSON (image_animation approach)
|
||||
|
||||
For `anime_scene` compositions, build a JSON file at `remotion-composer/public/demo-props/<name>.json`.
|
||||
|
||||
**Required structure:**
|
||||
|
||||
```json
|
||||
{
|
||||
"cuts": [
|
||||
{
|
||||
"id": "scene-1-name",
|
||||
"source": "",
|
||||
"in_seconds": 0,
|
||||
"out_seconds": 5,
|
||||
"type": "anime_scene",
|
||||
"images": ["<project>/<image-a>.png", "<project>/<image-b>.png"],
|
||||
"animation": "<camera-motion>",
|
||||
"particles": "<particle-type>",
|
||||
"particleColor": "#HEXCOLOR",
|
||||
"particleCount": 20,
|
||||
"particleIntensity": 0.5,
|
||||
"backgroundColor": "#0A0A1A",
|
||||
"vignette": true,
|
||||
"lightingFrom": "rgba(r,g,b,a)",
|
||||
"lightingTo": "transparent"
|
||||
}
|
||||
],
|
||||
"overlays": [...],
|
||||
"audio": { "music": { "src": "<project>/music.mp3", "volume": 0.15, "fadeInSeconds": 2, "fadeOutSeconds": 3 } }
|
||||
}
|
||||
```
|
||||
|
||||
**Prop name reference (JSON field → AnimeScene prop):**
|
||||
|
||||
| JSON Field | Type | Values | Required |
|
||||
|------------|------|--------|----------|
|
||||
| `type` | string | `"anime_scene"` | YES |
|
||||
| `images` | string[] | 1-4 image paths relative to `public/` | YES |
|
||||
| `animation` | string | `zoom-in`, `zoom-out`, `pan-left`, `pan-right`, `ken-burns`, `drift-up`, `drift-down`, `parallax`, `static` | No (default: `ken-burns`) |
|
||||
| `particles` | string | `fireflies`, `petals`, `sparkles`, `mist`, `light-rays` | No |
|
||||
| `particleColor` | string | Hex color | No (default: `#FFE082`) |
|
||||
| `particleCount` | number | 1-50 | No (default: 20) |
|
||||
| `particleIntensity` | number | 0-1 | No (default: 0.6) |
|
||||
| `backgroundColor` | string | Hex color for scene background | No (default: `#0A0A1A`) |
|
||||
| `vignette` | boolean | Cinematic vignette overlay | No (default: true) |
|
||||
| `lightingFrom` | string | Starting gradient color (`rgba(...)` or `transparent`) | No |
|
||||
| `lightingTo` | string | Ending gradient color | No |
|
||||
|
||||
**References:** See `mori-no-seishin.json` (Ghibli forest) and `deep-ocean.json` (underwater bioluminescence) for complete working examples.
|
||||
|
||||
### 3. Source Music and Find Optimal Offset
|
||||
|
||||
Use `tools/audio/pixabay_music.py` to find royalty-free ambient music matching the mood.
|
||||
|
||||
**After downloading, run audio energy analysis (MANDATORY):**
|
||||
|
||||
```python
|
||||
from tools.analysis.audio_energy import AudioEnergy
|
||||
result = AudioEnergy().execute({
|
||||
"input_path": "path/to/music.mp3",
|
||||
"video_duration_seconds": 30, # your video duration
|
||||
})
|
||||
data = result.data
|
||||
print(f"Recommended offset: {data['recommended_offset_seconds']}s")
|
||||
print(f"Reason: {data['offset_reason']}")
|
||||
print(f"Needs loop: {data['needs_loop']}")
|
||||
```
|
||||
|
||||
This tool:
|
||||
1. **Finds the best section** — analyzes per-second loudness and finds the N-second window with highest average energy. Ambient music tracks often have quiet intros (10-30s) before the main melody kicks in.
|
||||
2. **Recommends loop** — if the music from the offset is shorter than the video, it tells you to enable looping.
|
||||
|
||||
**Apply the offset in the composition JSON:**
|
||||
|
||||
```json
|
||||
"audio": {
|
||||
"music": {
|
||||
"src": "project/music.mp3",
|
||||
"volume": 0.15,
|
||||
"fadeInSeconds": 2,
|
||||
"fadeOutSeconds": 3,
|
||||
"offsetSeconds": 55,
|
||||
"loop": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `offsetSeconds` — start playback from this point in the track (skips quiet intro)
|
||||
- `loop` — set to `true` if the remaining music is shorter than the video
|
||||
|
||||
**If the tool says `needs_loop: true`:** set `"loop": true` in the composition JSON. Remotion will loop the audio seamlessly with the volume fade resetting per loop.
|
||||
|
||||
### 4. Pre-Render Validation (MANDATORY — NO EXCEPTIONS)
|
||||
|
||||
Run `composition_validator` before every render:
|
||||
|
||||
```python
|
||||
from tools.analysis.composition_validator import CompositionValidator
|
||||
result = CompositionValidator().execute({
|
||||
"composition_path": "remotion-composer/public/demo-props/<name>.json",
|
||||
"assets_root": "remotion-composer/public",
|
||||
})
|
||||
# result.data["valid"] MUST be True before proceeding
|
||||
```
|
||||
|
||||
This catches:
|
||||
- Missing image/audio files that would cause black frames or render errors
|
||||
- Invalid cut timings (out ≤ in)
|
||||
- Audio longer than video duration
|
||||
|
||||
**If validation fails, fix the issue BEFORE rendering. Do not render an invalid composition.**
|
||||
|
||||
### 5. Preserve Motion Timing
|
||||
|
||||
Do not let export settings or careless composition change the perceived timing of holds, stagger, or scene transitions.
|
||||
|
||||
### 2. Protect Text And Diagram Sharpness
|
||||
### 6. Protect Text And Diagram Sharpness
|
||||
|
||||
Animation often fails on export through soft text, muddy thin lines, or cramped mobile framing.
|
||||
|
||||
### 3. Verify The First And Last Frames
|
||||
### 7. Render
|
||||
|
||||
Ensure:
|
||||
```bash
|
||||
cd remotion-composer
|
||||
npx remotion render Explainer \
|
||||
--props="public/demo-props/<name>.json" \
|
||||
--output="<output-path>/final.mp4" \
|
||||
--codec=h264 --crf=18
|
||||
```
|
||||
|
||||
- the opening frame reads immediately,
|
||||
- the final frame lands cleanly,
|
||||
- nothing important is clipped by safe zones.
|
||||
**Note:** The composition name is `Explainer` (not `ExplainerVideo`). Do NOT specify `src/index.ts` as entry point — Remotion auto-discovers it.
|
||||
|
||||
### 4. Use Render Metadata
|
||||
### 8. Post-Render Self-Review (MANDATORY)
|
||||
|
||||
After rendering, extract mid-scene frames and visually inspect:
|
||||
|
||||
```bash
|
||||
# Extract one frame from the middle of each scene
|
||||
ffmpeg -y -i final.mp4 \
|
||||
-vf "select='eq(n\,75)+eq(n\,225)+eq(n\,375)+eq(n\,525)+eq(n\,675)+eq(n\,825)'" \
|
||||
-vsync vfr frames/scene_%02d.png
|
||||
```
|
||||
|
||||
**Check each frame for:**
|
||||
- [ ] Images are visible (not black/dark frames)
|
||||
- [ ] Particles are rendering (sparkles, fireflies, etc. visible)
|
||||
- [ ] Camera motion is evident (framing differs from static)
|
||||
- [ ] Overlays display at correct moments with clean text
|
||||
- [ ] Color palette is consistent across scenes
|
||||
- [ ] Vignette creates cinematic depth
|
||||
|
||||
**Also verify the output file:**
|
||||
```bash
|
||||
ffprobe -v quiet -print_format json -show_format -show_streams final.mp4
|
||||
```
|
||||
- Duration within ±5% of target?
|
||||
- Resolution matches 1920×1080?
|
||||
- Audio stream present?
|
||||
|
||||
**If issues are found:** identify the cause (missing images, wrong timing, rendering glitch) and fix before presenting to user.
|
||||
|
||||
### 9. Use Render Metadata
|
||||
|
||||
Recommended metadata keys:
|
||||
|
||||
@@ -42,6 +211,10 @@ Recommended metadata keys:
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- **Forgetting to copy assets to `remotion-composer/public/`** — the #1 cause of render failures. Images generate to `projects/<name>/assets/` but Remotion reads from `public/`.
|
||||
- Soft or aliased text after rendering.
|
||||
- Compression choices that damage diagrams.
|
||||
- Scene cadence changing between preview and final.
|
||||
- **Skipping `composition_validator`** — catches missing files, bad timings, audio mismatches before you waste render time.
|
||||
- **Not extracting frames for self-review** — a rendered video is not "done" until frames are visually inspected. Black frames, missing particles, or invisible images are not always obvious from file size alone.
|
||||
- **Using `durationInFrames` from `useVideoConfig()` for scene-level timing** — this returns the FULL composition duration, not the scene's Sequence duration. See `skills/core/remotion.md` Critical Constraints.
|
||||
|
||||
@@ -43,7 +43,14 @@ EP_STATE:
|
||||
budget_remaining_usd: <budget_total>
|
||||
|
||||
# Animation-specific state
|
||||
animation_mode: <manim | remotion | ai_video | diagram_stills | mixed>
|
||||
# Approaches:
|
||||
# image_animation — Multi-image crossfade via Remotion (anime/Ghibli/illustration style)
|
||||
# clip_video — AI-generated video clips composited as a story
|
||||
# manim — Programmatic math/physics animation via ManimCE
|
||||
# remotion_dataviz — Data visualization with Remotion components (zero-key capable)
|
||||
# diagram_stills — Diagram + image stills with Ken Burns
|
||||
# mixed — Combination of multiple approaches per-scene
|
||||
animation_mode: <image_animation | clip_video | manim | remotion_dataviz | diagram_stills | mixed>
|
||||
reuse_strategy:
|
||||
recurring_motifs: []
|
||||
layout_system: null
|
||||
@@ -218,12 +225,15 @@ CHECK: Approval gate (CRITICAL)
|
||||
- If "approved_with_changes": apply modifications before proceeding
|
||||
- Extract: animation_mode, reuse_strategy, target_duration, playbook, budget, tool selections
|
||||
|
||||
CHECK: Animation mode feasibility
|
||||
- Does the selected animation mode's required tools exist in the registry?
|
||||
- If Manim mode selected: is math_animate available?
|
||||
- If Remotion mode selected: is video_compose (Remotion) available?
|
||||
- If AI video mode selected: are video generation providers available?
|
||||
- If any required tool is unavailable: alert user, offer alternatives
|
||||
CHECK: Animation approach feasibility
|
||||
- Does the selected animation approach's required tools exist in the registry?
|
||||
- If image_animation selected: is image_selector available? Which providers? Is Remotion available?
|
||||
- If clip_video selected: is video_selector available? Which providers?
|
||||
- If manim selected: is math_animate (ManimCE) available?
|
||||
- If remotion_dataviz selected: is video_compose (Remotion) available?
|
||||
- If diagram_stills selected: is diagram_gen + image_selector available?
|
||||
- If any required tool is unavailable: alert user, offer alternatives with specific setup instructions
|
||||
- NEVER silently downgrade — if an approach needs a key the user doesn't have, STOP and tell them
|
||||
|
||||
CHECK: Reuse strategy validity
|
||||
- Does the reuse strategy define recurring motifs?
|
||||
|
||||
@@ -22,9 +22,13 @@ Animation proposals have a unique dimension: **animation mode selection**. Unlik
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Absorb the Research
|
||||
### Step 1: Absorb the Research (or Direct Brief)
|
||||
|
||||
Read the `research_brief` thoroughly. Extract:
|
||||
**If a `research_brief` artifact exists:** Read it thoroughly. Extract:
|
||||
|
||||
**If no research_brief exists (direct user brief):** The user has given you a creative brief directly. This is common for short videos (30-60s) where formal research is overkill. Use the user's brief as your input and proceed to Step 2. Note the missing research as a limitation — you won't have data_points, technique references, or audience_insights to draw from, so concept design relies on your knowledge and the user's direction.
|
||||
|
||||
**When a research_brief IS available,** extract:
|
||||
|
||||
- **`research_summary`** — read first. Contains both the key insight and the most promising animation approach.
|
||||
- **`angles_discovered`** — raw concept candidates, each with an `animation_fit` field.
|
||||
@@ -60,25 +64,105 @@ python -c "from tools.tool_registry import registry; import json; registry.disco
|
||||
|
||||
Record all findings. **Do not propose an animation mode that requires tools you don't have.**
|
||||
|
||||
### Step 3: Animation Mode Decision Matrix
|
||||
### Step 3: Animation Approach Selection
|
||||
|
||||
This is the key differentiator from the explainer proposal. For each viable animation mode, evaluate:
|
||||
This is the key differentiator from the explainer proposal. **Present the user with concrete animation approaches, explain what each looks like, what tools/keys they need, and what's already available.**
|
||||
|
||||
| Mode | Best For | Tool Required | Visual Quality | Cost | Iteration Speed |
|
||||
|------|----------|---------------|----------------|------|-----------------|
|
||||
| **Manim (ManimCE)** | Math, physics, geometry, algorithms | `math_animate` | Precise, programmatic | Free (local) | Fast (code-driven) |
|
||||
| **Remotion** | Data viz, charts, React components, kinetic type | `video_compose` (Remotion mode) | Smooth, web-native | Free (local) | Fast (code-driven) |
|
||||
| **AI Video Generation** | Abstract concepts, metaphors, transitions | `video_selector` providers | Variable, cinematic | $0.05-0.50/clip | Slow (generation time) |
|
||||
| **Diagram + Image Stills** | Process flows, architecture, comparisons | `diagram_gen` + `image_selector` | Clean, reliable | $0-0.05/image | Fast |
|
||||
| **Mixed Mode** | Complex topics needing multiple techniques | Multiple tools | Varied | Varies | Moderate |
|
||||
#### Step 3a: Tool Availability Scan
|
||||
|
||||
**Mode selection rules:**
|
||||
- If the topic involves math/formulas/geometry → prefer Manim
|
||||
- If the topic involves data/statistics/charts → prefer Remotion or diagram_gen
|
||||
- If the topic is abstract/conceptual → consider AI video for key moments
|
||||
- If the topic is process/workflow → prefer diagram builds
|
||||
- Always check tool availability before committing to a mode
|
||||
- Mixed mode is valid when different sections need different approaches
|
||||
Before designing concepts, scan what's available and present it honestly:
|
||||
|
||||
```
|
||||
TOOL AVAILABILITY SCAN
|
||||
──────────────────────
|
||||
Image generation:
|
||||
✅ FLUX (fal.ai) — FAL_KEY detected — $0.03-0.05/image
|
||||
❌ gpt-image-1 — OPENAI_API_KEY missing — $0.13/image
|
||||
❌ Stable Diffusion — Not installed locally — Free
|
||||
❌ FLUX (local) — Not installed locally — Free
|
||||
|
||||
Video generation:
|
||||
❌ Runway Gen-3 — No API key — $0.50/clip
|
||||
❌ Kling — No API key — $0.10-0.30/clip
|
||||
❌ CogVideoX (local) — Not installed — Free
|
||||
|
||||
Composition:
|
||||
✅ Remotion — Installed — Free (local CPU)
|
||||
✅ FFmpeg — Installed — Free
|
||||
|
||||
Audio:
|
||||
✅ Pixabay Music — No key needed — Free
|
||||
❌ OpenAI TTS — OPENAI_API_KEY missing — $0.015/min
|
||||
✅ Local TTS (piper) — Not checked — Free
|
||||
|
||||
Math/Diagram:
|
||||
❌ ManimCE — Not installed — Free
|
||||
✅ diagram_gen — Available — Free
|
||||
```
|
||||
|
||||
**Present this scan to the user.** Say: "Here's what I can see right now. Based on this, here are your animation approach options."
|
||||
|
||||
#### Step 3b: Animation Approach Decision Matrix
|
||||
|
||||
Present the approaches as clear options:
|
||||
|
||||
| Approach | What It Looks Like | Tools Required | Cost Range | Proven? |
|
||||
|----------|-------------------|----------------|------------|---------|
|
||||
| **A: Image-Based Animation (Remotion)** | AI-generated keyframes with crossfade, camera motion, particles. Looks like moving anime/illustration. | `image_selector` (any provider) + Remotion | $0.03-0.13/image × 2-3/scene | ✅ Proven (mori-no-seishin) |
|
||||
| **B: Clip-Based Video** | AI-generated video clips assembled as a story. Most cinematic but least consistent. | `video_selector` (Runway/Kling/etc.) | $0.10-0.50/clip × scenes | ❌ Not yet proven |
|
||||
| **C: Programmatic Animation (Manim)** | Code-driven math/geometry animation. Precise, clean, 3Blue1Brown style. | `math_animate` (ManimCE) | Free (local) | ❌ Not yet proven |
|
||||
| **D: Data Visualization (Remotion)** | Animated charts, KPIs, kinetic typography. Data-driven storytelling. | Remotion (built-in components) | Free (local) | ✅ Proven (zero-key formula) |
|
||||
| **E: Diagram + Image Stills** | Process flows and architecture diagrams with Ken Burns. | `diagram_gen` + `image_selector` | $0-0.05/image | ✅ Proven |
|
||||
| **F: Mixed Mode** | Combine any of the above per-scene. Most flexible. | Multiple tools | Varies | Partial |
|
||||
|
||||
**For each viable approach, present to the user:**
|
||||
|
||||
```
|
||||
APPROACH A: Image-Based Animation (Remotion)
|
||||
─────────────────────────────────────────────
|
||||
What it looks like: Multiple AI-generated images per scene, crossfaded with
|
||||
camera motion (zoom, pan, ken-burns) and particle overlays (fireflies, mist,
|
||||
sparkles). Creates the illusion of movement from still frames.
|
||||
|
||||
You need: An image generation API key.
|
||||
→ You already have: FAL_KEY (FLUX at $0.05/image)
|
||||
→ Alternative: Install Stable Diffusion locally (free, slower)
|
||||
→ Alternative: Add OPENAI_API_KEY for gpt-image-1 ($0.13/image)
|
||||
|
||||
Estimated cost for 30s video: ~$0.65 (13 images)
|
||||
Estimated cost for 5min video: ~$6.00 (120 images)
|
||||
|
||||
Style options: anime-ghibli, painterly, photorealistic, watercolor
|
||||
Reference: remotion-composer/public/demo-props/mori-no-seishin.json
|
||||
|
||||
APPROACH B: Clip-Based Video
|
||||
─────────────────────────────
|
||||
What it looks like: AI-generated 3-5 second video clips assembled as a story.
|
||||
Most cinematic output but hardest to maintain visual consistency across clips.
|
||||
|
||||
You need: A video generation API key.
|
||||
→ Currently available: None detected
|
||||
→ To enable: Add RUNWAY_API_KEY, KLING_API_KEY, or install CogVideoX locally
|
||||
|
||||
Estimated cost for 30s video: $3-15 depending on provider
|
||||
Estimated cost for 5min video: $30-150
|
||||
|
||||
Note: This approach is not yet proven in the OpenMontage pipeline.
|
||||
Consistency across clips is the #1 challenge.
|
||||
```
|
||||
|
||||
**Critical principle: Surface capabilities, don't hide limitations.** The user should know exactly what's possible right now vs. what needs setup.
|
||||
|
||||
#### Step 3c: Mode Selection Rules
|
||||
|
||||
- If the topic is visual/artistic (anime, illustration, fantasy) → **Approach A** (image-based)
|
||||
- If the topic involves data/statistics/business → **Approach D** (data viz) or **Approach A** with data overlays
|
||||
- If the topic involves math/physics → **Approach C** (Manim) if available, else **Approach E**
|
||||
- If the topic is abstract/conceptual and budget allows → **Approach B** (clip-based) for key moments
|
||||
- If no paid APIs available → **Approach D** (zero-key Remotion) or **Approach E** (diagrams)
|
||||
- If the user wants maximum quality and has video gen keys → **Approach F** (mixed: video clips for hero shots + Remotion for data)
|
||||
- **Always offer at least one free/local option** alongside paid approaches
|
||||
- **Never silently downgrade** — if the best approach needs a key the user doesn't have, say so explicitly
|
||||
|
||||
### Step 4: Design Concept Options
|
||||
|
||||
@@ -103,13 +187,15 @@ For each concept, specify:
|
||||
- Hook must promise a VISUAL experience, not just information
|
||||
- Hook must be grounded in a specific research finding
|
||||
|
||||
#### 4b: Animation Mode and Approach
|
||||
#### 4b: Animation Approach and Approach
|
||||
|
||||
For each concept, specify:
|
||||
- **Primary animation mode**: manim / remotion / ai_video / diagram_stills / mixed
|
||||
- **Why this mode**: grounded in technique research from the brief
|
||||
- **Animation approach**: `image_animation` / `clip_video` / `manim` / `remotion_dataviz` / `diagram_stills` / `mixed`
|
||||
- **Why this approach**: grounded in technique research AND tool availability from Step 3
|
||||
- **Image/video generation provider**: which specific provider from the preflight scan (e.g., "FLUX via fal.ai", "gpt-image-1 via OpenAI", "Stable Diffusion local")
|
||||
- **Reuse strategy**: What's the visual system? (recurring motifs, layout grid, color scheme, transition family)
|
||||
- **Complexity estimate**: How many unique scene types vs. reusable templates?
|
||||
- **Style playbook**: which playbook from `styles/*.yaml` (e.g., `anime-ghibli`, `clean-professional`)
|
||||
|
||||
#### 4c: Narrative Structure
|
||||
|
||||
@@ -130,11 +216,12 @@ Choose from: `myth_busting`, `problem_solution`, `data_narrative`, `comparison`,
|
||||
|
||||
#### 4e: Concept Diversity Check
|
||||
|
||||
- [ ] No two concepts use the same animation mode
|
||||
- [ ] No two concepts use the same animation approach
|
||||
- [ ] No two concepts use the same narrative structure
|
||||
- [ ] At least one concept is achievable with free/local tools only
|
||||
- [ ] At least one concept is achievable with free/local tools only (zero-key or local image gen)
|
||||
- [ ] At least one concept leverages the most surprising data point
|
||||
- [ ] Each concept's animation mode is grounded in technique research
|
||||
- [ ] Each concept's approach is grounded in tool availability AND technique research
|
||||
- [ ] Each concept states which API keys/tools it requires (and flags any the user doesn't have)
|
||||
|
||||
### Step 5: Present Concepts and Get Selection
|
||||
|
||||
@@ -255,9 +342,12 @@ Validate the `proposal_packet` artifact against `schemas/artifacts/proposal_pack
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- **Ignoring animation mode feasibility**: If Manim isn't installed, don't propose a Manim-based concept. Design around constraints.
|
||||
- **Three versions of the same concept with different titles**: Structural diversity means different animation modes, different narrative structures, different hooks.
|
||||
- **Not leveraging free tools**: Animation has a huge cost advantage — Manim, Remotion, and diagram_gen are free. If proposing expensive AI video, justify why free alternatives won't work.
|
||||
- **Not showing the Tool Availability Scan**: The user must know what's available BEFORE seeing concepts. Don't hide missing keys or tools.
|
||||
- **Ignoring animation approach feasibility**: If FLUX isn't available, don't propose image_animation without saying "you need to add FAL_KEY first." Design around constraints OR explicitly state what's needed.
|
||||
- **Three versions of the same concept with different titles**: Structural diversity means different animation approaches, different narrative structures, different hooks.
|
||||
- **Not leveraging free tools**: Animation has a huge cost advantage — Manim, Remotion data-viz, and diagram_gen are free. If proposing expensive AI video, justify why free alternatives won't work.
|
||||
- **Over-promising visual complexity**: 20 unique hand-crafted scenes is not realistic. Design reuse strategies that look varied but share underlying templates.
|
||||
- **Skipping the approval gate**: This is the whole point of pre-production. No shortcuts.
|
||||
- **Ignoring mathematical accuracy**: If the research brief flagged technical accuracy constraints, the concept MUST respect them. A beautiful but wrong animation is a failure.
|
||||
- **Not distinguishing image_animation from clip_video**: These are fundamentally different. Image-based animation (Approach A) generates still images and uses Remotion for motion/crossfade. Clip-based video (Approach B) generates actual video clips with an AI video model. The user should understand this distinction clearly.
|
||||
- **Silent downgrades**: If the user picked image_animation but image generation fails, STOP and tell them. Never silently fall back to text cards or diagram stills.
|
||||
|
||||
@@ -41,6 +41,40 @@ Use:
|
||||
- `text_card` for clean high-impact copy moments,
|
||||
- `generated` only where needed.
|
||||
|
||||
**For `image_animation` approach (anime/illustration style):**
|
||||
|
||||
Use `anime_scene` type for each scene. Plan:
|
||||
|
||||
- **Images per scene**: 2-3 images with consistent style prefix and nearby seeds for crossfade effect
|
||||
- **Camera motion**: choose from `zoom-in`, `zoom-out`, `pan-left`, `pan-right`, `ken-burns`, `drift-up`, `drift-down`, `parallax`, `static` — vary per scene to prevent monotony
|
||||
- **Particle type**: choose from `fireflies`, `petals`, `sparkles`, `mist`, `light-rays` — match to scene mood
|
||||
- **Lighting**: optional `lightingFrom`/`lightingTo` gradient for atmospheric shifts within the scene
|
||||
- **Vignette**: `true` for cinematic framing (default), `false` for bright/open scenes
|
||||
- **Scene duration**: 4-7 seconds per scene. Longer scenes need more images for crossfade variety.
|
||||
|
||||
**Scene variety rules for image_animation:**
|
||||
- Don't use the same camera motion for consecutive scenes
|
||||
- Alternate between warm and cool particle types
|
||||
- Mix close-up and wide establishing shots
|
||||
- Use overlays (`hero_title`, `section_title`) to add narrative structure
|
||||
|
||||
**JSON prop name mapping** (use these exact field names in the composition JSON):
|
||||
|
||||
| Concept | JSON Field | Example Values |
|
||||
|---------|-----------|----------------|
|
||||
| Camera motion | `animation` | `"zoom-in"`, `"pan-right"`, `"ken-burns"` |
|
||||
| Particle effect | `particles` | `"fireflies"`, `"sparkles"`, `"mist"` |
|
||||
| Particle color | `particleColor` | `"#FFE082"` |
|
||||
| Particle density | `particleCount` | `20` (range: 1-50) |
|
||||
| Particle brightness | `particleIntensity` | `0.5` (range: 0-1) |
|
||||
| Lighting start | `lightingFrom` | `"rgba(255,200,100,0.15)"` or `"transparent"` |
|
||||
| Lighting end | `lightingTo` | `"rgba(255,107,157,0.08)"` or `"transparent"` |
|
||||
| Cinematic edge darken | `vignette` | `true` / `false` |
|
||||
| Scene background | `backgroundColor` | `"#0A0A1A"` |
|
||||
|
||||
Reference: `remotion-composer/public/demo-props/mori-no-seishin.json` — 6 scenes using this pattern.
|
||||
Reference: `remotion-composer/public/demo-props/deep-ocean.json` — 6 underwater scenes with different palette.
|
||||
|
||||
### 4. Use Metadata For Timing Rules
|
||||
|
||||
Recommended metadata keys:
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
identity:
|
||||
name: "Anime Ghibli"
|
||||
category: anime-illustration
|
||||
mood: warm, whimsical, contemplative, magical
|
||||
pace: gentle
|
||||
best_for: "Narrative animations, nature-themed stories, emotional storytelling, fantasy visuals, educational content with wonder"
|
||||
|
||||
visual_language:
|
||||
color_palette:
|
||||
primary: ["#2D5016", "#1B4332"] # Deep forest greens
|
||||
accent: ["#FFB347", "#FF6B9D"] # Warm golden, soft cherry blossom pink
|
||||
background: "#0A0A1A" # Deep night sky
|
||||
text: "#F5F0E8" # Warm parchment white
|
||||
muted: "#8B9A7E" # Mossy sage
|
||||
spirit_glow: "#A8E6CF" # Soft teal glow for magical elements
|
||||
golden_hour: "#FFF3B0" # Warm golden hour light
|
||||
composition: centered subjects, rule of thirds for landscapes, generous negative space, layered depth
|
||||
texture: soft watercolor edges, painterly brushstrokes, organic shapes, no hard geometric lines
|
||||
|
||||
typography:
|
||||
headings:
|
||||
font: "Noto Serif JP"
|
||||
weight: 500
|
||||
tracking: "0.02em"
|
||||
body:
|
||||
font: "Noto Sans"
|
||||
weight: 300
|
||||
line_height: 1.7
|
||||
code:
|
||||
font: "Fira Code"
|
||||
weight: 400
|
||||
stat_card:
|
||||
font: "Noto Serif JP"
|
||||
weight: 600
|
||||
size_multiplier: 3.0
|
||||
scale_system: "major_third"
|
||||
weight_matrix:
|
||||
title: 600
|
||||
heading: 500
|
||||
body: 300
|
||||
caption: 300
|
||||
|
||||
motion:
|
||||
transitions: [crossfade, drift, dissolve]
|
||||
animation_style: "spring(1, 60, 20), gentle, no overshoot, organic easing"
|
||||
pacing_rules:
|
||||
min_scene_hold_seconds: 3.5
|
||||
max_scene_hold_seconds: 7
|
||||
text_card_hold_seconds: 4
|
||||
stat_card_hold_seconds: 3.5
|
||||
transition_duration_seconds: 1.0
|
||||
entrance: "gentle fade + slow drift-up (opacity 0->1, translateY 8->0 over 1s)"
|
||||
exit: "fade-dissolve (opacity 1->0 over 0.8s)"
|
||||
|
||||
audio:
|
||||
voice_style: "warm, gentle, storytelling cadence, unhurried, soft-spoken narrator"
|
||||
music_mood: "ambient piano, soft strings, nature sounds, gentle orchestral, Joe Hisaishi-inspired"
|
||||
music_volume: 0.15
|
||||
sfx_style: "nature sounds (wind, water, birds), subtle magical chimes, no harsh effects"
|
||||
ducking_threshold_db: -3
|
||||
|
||||
asset_generation:
|
||||
image_prompt_prefix: >
|
||||
Studio Ghibli anime style, hand-painted watercolor aesthetic, soft diffused lighting,
|
||||
lush natural environment, warm color palette, painterly brushstrokes visible,
|
||||
high detail in foliage and atmospheric effects,
|
||||
image_negative_prompt: "photorealistic, 3D render, CGI, harsh shadows, dark gritty, cyberpunk, neon"
|
||||
scene_type: "anime_scene"
|
||||
default_particles: "fireflies"
|
||||
default_particle_color: "#FFE082"
|
||||
default_vignette: true
|
||||
consistency_anchors:
|
||||
- "Deep forest greens (#2D5016) and warm golds (#FFB347) in every scene"
|
||||
- "Soft watercolor edges — no sharp digital lines"
|
||||
- "Layered depth: clear foreground, midground, background separation"
|
||||
- "Warm diffused lighting, never harsh or clinical"
|
||||
- "Magical elements glow with teal (#A8E6CF) or warm gold (#FFF3B0)"
|
||||
- "Nature always present: trees, flowers, water, sky"
|
||||
multi_image_per_scene: true
|
||||
images_per_scene: 2
|
||||
image_variation_guidance: >
|
||||
Generate 2-3 images per scene with subtle variations: slightly different lighting
|
||||
angle, leaves/particles in different positions, or gentle pose changes for characters.
|
||||
Keep composition and color palette identical. These crossfade to simulate motion.
|
||||
|
||||
overlays:
|
||||
stat_card:
|
||||
bg: "rgba(10, 10, 26, 0.7)"
|
||||
border: "#A8E6CF"
|
||||
radius: 16
|
||||
shadow: "0 4px 30px rgba(168, 230, 207, 0.15)"
|
||||
key_term:
|
||||
bg: "rgba(45, 80, 22, 0.5)"
|
||||
text: "#A8E6CF"
|
||||
radius: 8
|
||||
section_title:
|
||||
bg: "rgba(10, 10, 26, 0.6)"
|
||||
text: "#FFF3B0"
|
||||
accent: "#FFB347"
|
||||
|
||||
quality_rules:
|
||||
- "Minimum contrast ratio 4.5:1 for all text against backgrounds"
|
||||
- "Maximum 3 images per scene for crossfade (more creates jumpiness)"
|
||||
- "Every scene must have at least one animated element (particles or camera motion)"
|
||||
- "Scene pacing: hold each scene 4-6 seconds — never rush"
|
||||
- "Crossfade between images within a scene must be 1-1.5 seconds"
|
||||
- "Vignette on every image scene — creates cinematic depth"
|
||||
- "Color consistency: same palette across all generated images (use seed management)"
|
||||
- "No text burned into generated images — use Remotion overlays instead"
|
||||
|
||||
chart_palette:
|
||||
- "#A8E6CF"
|
||||
- "#FFB347"
|
||||
- "#FF6B9D"
|
||||
- "#87CEEB"
|
||||
- "#DDA0DD"
|
||||
- "#F5F0E8"
|
||||
|
||||
color_rules:
|
||||
harmony_type: "analogous"
|
||||
contrast_validation: true
|
||||
colorblind_safe: true
|
||||
@@ -0,0 +1,303 @@
|
||||
"""Analyze audio energy profile to find optimal playback offset.
|
||||
|
||||
Uses ffmpeg's ebur128 loudness meter to measure momentary loudness
|
||||
at 100ms intervals, then identifies where the music "gets interesting"
|
||||
(crosses a configurable energy threshold). Returns a recommended offset
|
||||
in seconds plus the full energy profile.
|
||||
|
||||
Key use cases:
|
||||
- Skip quiet intros in ambient/cinematic music tracks
|
||||
- Find the peak energy section for a 30-second video from a 3-minute track
|
||||
- Determine if music needs looping (total duration vs video duration)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from tools.base_tool import (
|
||||
BaseTool,
|
||||
Determinism,
|
||||
ExecutionMode,
|
||||
ResourceProfile,
|
||||
RetryPolicy,
|
||||
ToolResult,
|
||||
ToolRuntime,
|
||||
ToolStability,
|
||||
ToolStatus,
|
||||
ToolTier,
|
||||
)
|
||||
|
||||
|
||||
class AudioEnergy(BaseTool):
|
||||
name = "audio_energy"
|
||||
version = "0.1.0"
|
||||
tier = ToolTier.CORE
|
||||
capability = "analysis"
|
||||
provider = "ffmpeg"
|
||||
stability = ToolStability.PRODUCTION
|
||||
execution_mode = ExecutionMode.SYNC
|
||||
determinism = Determinism.DETERMINISTIC
|
||||
runtime = ToolRuntime.LOCAL
|
||||
|
||||
dependencies = ["binary:ffmpeg"]
|
||||
install_instructions = (
|
||||
"Install ffmpeg:\n"
|
||||
" Windows: winget install ffmpeg\n"
|
||||
" macOS: brew install ffmpeg\n"
|
||||
" Linux: sudo apt install ffmpeg"
|
||||
)
|
||||
|
||||
capabilities = [
|
||||
"find_music_offset",
|
||||
"energy_profile",
|
||||
"best_window",
|
||||
"loop_recommendation",
|
||||
]
|
||||
best_for = [
|
||||
"finding where ambient music gets interesting (skip quiet intros)",
|
||||
"choosing the best offset for a music track in a video",
|
||||
"determining if a music track needs looping for a longer video",
|
||||
]
|
||||
|
||||
input_schema = {
|
||||
"type": "object",
|
||||
"required": ["input_path"],
|
||||
"properties": {
|
||||
"input_path": {
|
||||
"type": "string",
|
||||
"description": "Path to audio file (mp3, wav, ogg, etc.)",
|
||||
},
|
||||
"video_duration_seconds": {
|
||||
"type": "number",
|
||||
"description": "Duration of the video this music will accompany. "
|
||||
"Used to recommend looping and find the best offset window.",
|
||||
},
|
||||
"energy_threshold_lufs": {
|
||||
"type": "number",
|
||||
"description": "Momentary loudness threshold in LUFS to consider "
|
||||
"music 'active' (default: -40). Higher = stricter. "
|
||||
"Typical: -50 for very quiet, -30 for energetic.",
|
||||
"default": -40,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
resource_profile = ResourceProfile(
|
||||
cpu_cores=1, ram_mb=128, vram_mb=0, disk_mb=0, network_required=False
|
||||
)
|
||||
retry_policy = RetryPolicy(max_retries=0, retryable_errors=[])
|
||||
idempotency_key_fields = ["input_path"]
|
||||
side_effects = []
|
||||
|
||||
def get_status(self) -> ToolStatus:
|
||||
if shutil.which("ffmpeg"):
|
||||
return ToolStatus.AVAILABLE
|
||||
return ToolStatus.UNAVAILABLE
|
||||
|
||||
def estimate_cost(self, inputs: dict[str, Any]) -> float:
|
||||
return 0.0
|
||||
|
||||
def execute(self, inputs: dict[str, Any]) -> ToolResult:
|
||||
input_path = Path(inputs["input_path"])
|
||||
if not input_path.exists():
|
||||
return ToolResult(success=False, error=f"File not found: {input_path}")
|
||||
|
||||
ffmpeg = shutil.which("ffmpeg")
|
||||
if not ffmpeg:
|
||||
return ToolResult(success=False, error="ffmpeg not found on PATH")
|
||||
|
||||
threshold_lufs = inputs.get("energy_threshold_lufs", -40)
|
||||
video_duration = inputs.get("video_duration_seconds")
|
||||
|
||||
start = time.time()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Step 1: Get audio duration
|
||||
# ------------------------------------------------------------------
|
||||
ffprobe = shutil.which("ffprobe")
|
||||
if not ffprobe:
|
||||
return ToolResult(success=False, error="ffprobe not found on PATH")
|
||||
|
||||
try:
|
||||
probe_result = subprocess.run(
|
||||
[
|
||||
ffprobe, "-v", "quiet", "-print_format", "json",
|
||||
"-show_format", str(input_path),
|
||||
],
|
||||
capture_output=True, text=True, timeout=10,
|
||||
)
|
||||
probe_data = json.loads(probe_result.stdout)
|
||||
audio_duration = float(probe_data["format"]["duration"])
|
||||
except Exception as e:
|
||||
return ToolResult(success=False, error=f"Failed to probe duration: {e}")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Step 2: Run ebur128 loudness analysis
|
||||
# ------------------------------------------------------------------
|
||||
# ebur128 outputs momentary loudness (M:) every 100ms — very precise.
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[
|
||||
ffmpeg, "-i", str(input_path),
|
||||
"-af", "ebur128",
|
||||
"-f", "null", "-",
|
||||
],
|
||||
capture_output=True, text=True, timeout=120,
|
||||
)
|
||||
stderr = result.stderr
|
||||
except subprocess.TimeoutExpired:
|
||||
return ToolResult(success=False, error="ebur128 analysis timed out (120s)")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Step 3: Parse momentary loudness (M:) values
|
||||
# ------------------------------------------------------------------
|
||||
# Pattern: t: 0.0999773 TARGET:-23 LUFS M:-120.7 S:-120.7 ...
|
||||
pattern = re.compile(r"t:\s*([\d.]+)\s+.*?M:\s*(-?[\d.]+)")
|
||||
raw_points: list[tuple[float, float]] = []
|
||||
|
||||
for line in stderr.split("\n"):
|
||||
match = pattern.search(line)
|
||||
if match:
|
||||
t = float(match.group(1))
|
||||
m_lufs = float(match.group(2))
|
||||
raw_points.append((t, m_lufs))
|
||||
|
||||
if not raw_points:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="Failed to parse ebur128 output — no loudness data found",
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Step 4: Downsample to 1-second intervals (average per second)
|
||||
# ------------------------------------------------------------------
|
||||
max_sec = int(raw_points[-1][0]) + 1
|
||||
energy_profile: list[dict[str, Any]] = []
|
||||
|
||||
for sec in range(max_sec):
|
||||
# Collect all 100ms points within this second
|
||||
points_in_sec = [
|
||||
m for t, m in raw_points
|
||||
if sec <= t < sec + 1 and m > -120 # -120 = silence marker
|
||||
]
|
||||
|
||||
if points_in_sec:
|
||||
avg_lufs = sum(points_in_sec) / len(points_in_sec)
|
||||
else:
|
||||
avg_lufs = -120.0
|
||||
|
||||
energy_profile.append({
|
||||
"time_seconds": sec,
|
||||
"loudness_lufs": round(avg_lufs, 1),
|
||||
"active": avg_lufs > threshold_lufs,
|
||||
})
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Step 5: Find key moments
|
||||
# ------------------------------------------------------------------
|
||||
# First active second (music becomes meaningful)
|
||||
first_active_sec = 0.0
|
||||
for seg in energy_profile:
|
||||
if seg["active"]:
|
||||
first_active_sec = float(seg["time_seconds"])
|
||||
break
|
||||
|
||||
# Peak loudness second
|
||||
active_segments = [s for s in energy_profile if s["loudness_lufs"] > -120]
|
||||
if active_segments:
|
||||
peak_seg = max(active_segments, key=lambda s: s["loudness_lufs"])
|
||||
peak_sec = float(peak_seg["time_seconds"])
|
||||
peak_lufs = peak_seg["loudness_lufs"]
|
||||
else:
|
||||
peak_sec = 0.0
|
||||
peak_lufs = -120.0
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Step 6: Find best window for video duration
|
||||
# ------------------------------------------------------------------
|
||||
recommended_offset = first_active_sec
|
||||
offset_reason = (
|
||||
f"First active music at {first_active_sec}s "
|
||||
f"(threshold: {threshold_lufs} LUFS)"
|
||||
)
|
||||
|
||||
if video_duration and video_duration < audio_duration:
|
||||
window_size = int(video_duration)
|
||||
loudness_values = [
|
||||
s["loudness_lufs"] if s["loudness_lufs"] > -120 else -60
|
||||
for s in energy_profile
|
||||
]
|
||||
|
||||
if len(loudness_values) >= window_size:
|
||||
best_avg = -999.0
|
||||
best_start = 0
|
||||
|
||||
for i in range(len(loudness_values) - window_size + 1):
|
||||
window = loudness_values[i : i + window_size]
|
||||
avg = sum(window) / len(window)
|
||||
if avg > best_avg:
|
||||
best_avg = avg
|
||||
best_start = i
|
||||
|
||||
recommended_offset = float(best_start)
|
||||
offset_reason = (
|
||||
f"Best {window_size}s window starts at {best_start}s "
|
||||
f"(avg loudness: {round(best_avg, 1)} LUFS)"
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Step 7: Loop recommendation
|
||||
# ------------------------------------------------------------------
|
||||
needs_loop = False
|
||||
loop_info = None
|
||||
if video_duration:
|
||||
available_from_offset = audio_duration - recommended_offset
|
||||
if available_from_offset < video_duration:
|
||||
needs_loop = True
|
||||
loop_info = {
|
||||
"music_available_from_offset": round(available_from_offset, 1),
|
||||
"video_duration": round(video_duration, 1),
|
||||
"shortfall_seconds": round(
|
||||
video_duration - available_from_offset, 1
|
||||
),
|
||||
"recommendation": (
|
||||
f"Music from offset {recommended_offset}s provides only "
|
||||
f"{round(available_from_offset, 1)}s but video is "
|
||||
f"{round(video_duration, 1)}s. Set loop=true and "
|
||||
f"offsetSeconds={recommended_offset} in audio config."
|
||||
),
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Result
|
||||
# ------------------------------------------------------------------
|
||||
result_data = {
|
||||
"file": str(input_path),
|
||||
"audio_duration_seconds": round(audio_duration, 1),
|
||||
"analysis": {
|
||||
"threshold_lufs": threshold_lufs,
|
||||
"total_seconds": len(energy_profile),
|
||||
"active_seconds": sum(1 for s in energy_profile if s["active"]),
|
||||
"quiet_intro_seconds": first_active_sec,
|
||||
"peak_loudness_at_seconds": peak_sec,
|
||||
"peak_loudness_lufs": peak_lufs,
|
||||
},
|
||||
"recommended_offset_seconds": recommended_offset,
|
||||
"offset_reason": offset_reason,
|
||||
"needs_loop": needs_loop,
|
||||
"loop_info": loop_info,
|
||||
"energy_profile": energy_profile,
|
||||
}
|
||||
|
||||
return ToolResult(
|
||||
success=True,
|
||||
data=result_data,
|
||||
duration_seconds=round(time.time() - start, 2),
|
||||
)
|
||||
Reference in New Issue
Block a user