freedak/OpenMontage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add per-scene motion classification to video analyzer

Browse Source 
Video analyzer now uses Farneback dense optical flow to classify each
scene as motion_clip, animated_still, or static_image. This lets the
agent correctly identify whether a reference video uses AI-generated
video clips vs still images with pan/zoom — and plan the right pipeline.

Changes:
- video_analyzer.py: new Step 3b with _classify_scene_motion() and
  _read_frame_at() helpers; updated _needs_motion() to use per-scene
  motion data instead of pacing heuristic alone
- video-reference-analyst.md: added Motion line to summary template
  and instructions to read motion_type field before proposing tools
This commit is contained in:
calesthio
2026-04-04 11:33:33 -07:00
parent 6ba79390e9
commit 286c26e33d
2 changed files with 135 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/meta/video-reference-analyst.md
+15
View File
@@ -46,12 +46,27 @@ user. This is NOT a raw dump. It's a conversational interpretation:
**Content:** [2-sentence summary of what the video is about]
**Style:** [1 sentence — pacing, visual treatment, energy]
**Structure:** [X scenes over Y seconds, pacing style]
**Motion:** [N of M scenes are motion clips / animated stills / static images.
This video uses [AI-generated video clips / still images with pan-zoom / a mix].]
**What makes it work:** [2-3 specific things — the hook technique, the pacing,
the visual transitions, the narration style]
Now let me check what I can do with your current setup..."
```
**Motion classification is critical.** The VideoAnalysisBrief now includes per-scene
`motion_type` ("motion_clip", "animated_still", "static_image") and `flow_variance`.
Use this to determine the production approach:
- If most scenes are `motion_clip` → the reference uses **video generation** (Kling,
MiniMax, etc.) → plan around video gen tools, not image gen
- If most scenes are `animated_still` → the reference uses **still images with
Ken Burns / pan-zoom** → image gen + Remotion/FFmpeg composition is appropriate
- If mixed → note which sections use motion and which use stills
**Never guess** whether a reference uses images or video. Read the `motion_type` field.
Getting this wrong leads to proposing the wrong pipeline and wrong tool path.
**Vision analysis:** After presenting the structural data, examine the extracted
keyframes yourself. You ARE a multimodal model — look at the keyframe images and
enrich the VideoAnalysisBrief with: