freedak/OpenMontage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Enforce Remotion-first composition engine, fix FFmpeg fallback bugs

Browse Source 
Remotion is now the default composition engine for ALL final renders
when available — video clips, images, mixed content. FFmpeg is only
used as fallback when Remotion is not installed or for standalone
operations (trim, transcode). Also fixes three FFmpeg fallback bugs:
profile + copy codec conflict, stream order mapping, and segment
seeking for audio-first containers.
This commit is contained in:
calesthio
2026-04-04 12:39:24 -07:00
parent 38b6ec5212
commit 16647d2d36
3 changed files with 94 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/core/remotion.md
+22 -11
View File
@@ -13,22 +13,33 @@ Remotion API usage — imports, timing, animation constraints, code patterns.
**This file** teaches how OpenMontage uses Remotion — which compositions map to pipeline
stages, how artifacts flow in, and how renders are triggered.
## When to Use Remotion vs FFmpeg
## Remotion-First Routing
**Remotion is the DEFAULT composition engine for ALL final renders when available.**
It handles video clips (via `<OffthreadVideo>`), still images, animated scenes,
component types, transitions, and mixed content — all in a single React-based
render pass.
FFmpeg is the **fallback** — used only when Remotion is unavailable, or for
simple standalone operations that don't benefit from React rendering.
| Use Case | Backend | Why |
|----------|---------|-----|
| Simple cuts, trims, concat | FFmpeg | Instant, no Node dependency |
| Subtitle burn-in | FFmpeg | Proven, fast |
| Final video render (any content type) | **Remotion** | Default for all compositions |
| Video clips + animated stills + text cards | **Remotion** | Mixed content in one pass |
| Video-only cuts with transitions | **Remotion** | Native `<OffthreadVideo>` + transitions |
| Animated diagrams/text cards | **Remotion** | Frame-by-frame control |
| Data-driven batch videos | **Remotion** | Zod props + parametric renders |
| Simple trim, concat (no composition) | FFmpeg | Instant, no Node dependency |
| Subtitle burn-in (standalone) | FFmpeg | Proven, fast |
| Face enhance, color grade | FFmpeg | Filter-based, deterministic |
| Multi-layer overlays + transitions | Remotion | React composability |
| Animated diagrams/text cards | Remotion | Frame-by-frame control |
| Data-driven batch videos | Remotion | Zod props + parametric renders |
| Generated explainer pipeline | Remotion | Full scene graph needed |
| Talking-head (video-only cuts) | FFmpeg | No images/animations needed |
| Remotion unavailable | FFmpeg | Automatic fallback |
**Note:** The `render` operation auto-routes — if any cut contains images,
animations, transitions, or component types, it delegates to Remotion
automatically. No need to manually select backend.
**Note:** The `render` operation auto-routes to Remotion by default. FFmpeg is
only selected when Remotion is not installed or the agent explicitly calls
`operation='compose'` for standalone operations. The agent can also write custom
Remotion compositions on the fly via the capability-extension protocol when no
existing composition covers the layout (e.g., custom PiP, split-screen).
## Supported Scene Types (Cut Types)
skills/meta/video-reference-analyst.md
+6 -6
View File
@@ -103,12 +103,12 @@ Composition engine Remotion: available READY (preferred)
FFmpeg: available READY (fallback only)
```
**Composition engine priority:** Always check Remotion availability at this step.
When Remotion is available, it is the **primary** composition engine — use it for
transitions, animated text, still-image animation, and scene assembly. FFmpeg is
the fallback for when Remotion is unavailable, or for simple operations that don't
benefit from Remotion (pure concat, trim, audio mux). Never default to FFmpeg when
Remotion is available.
**Composition engine priority:** Remotion is the **default** composition engine for
ALL final renders — video clips, images, animated scenes, mixed content. It embeds
video natively via `<OffthreadVideo>` and handles transitions, overlays, and profile
scaling in a single React render pass. FFmpeg is only used when Remotion is
unavailable, or for standalone operations (trim, transcode, subtitle burn) outside
the composition pipeline. **Never default to FFmpeg when Remotion is available.**
Be honest about gaps. If video generation is needed but unavailable, say so clearly: