Add documentary-montage pipeline for retrieval-first motion-clip montage

New end-to-end pipeline for building thematic documentary montages from
a locally-indexed corpus of free stock footage (Pexels, Archive.org,
NASA). The agent builds a project-local corpus, CLIP-ranks candidates
per scene slot, edits with motion-aware arc logic, and composes via
ffmpeg. No paid APIs required for the full path.

Pipeline definition and director skills:
- pipeline_defs/documentary-montage.yaml: 5-stage manifest
  (idea -> scene_plan -> assets -> edit -> compose)
- skills/pipelines/documentary-montage/: 6 director skills
  (executive-producer + idea/scene/asset/edit/compose directors)

Corpus and retrieval infrastructure:
- tools/video/corpus_builder.py: multi-source stock fan-out with
  resumable append-only corpus index
- tools/video/clip_search.py: CLIP ViT-B/32 retrieval —
  rank_for_slot, find_similar_set, diversify, stats
- tools/video/stock_sources/: base + pexels + archive_org + nasa
  adapters with a pluggable BaseStockSource contract
- lib/clip_embedder.py: CLIP wrapper
- lib/corpus.py: corpus schema, jsonl append/read, motion-score
  caching

video_compose fix rolled in because any concat-based pipeline depends
on it:
- Replace ambiguous -to with -t duration (was double-trimming cuts)
- Force re-encode + normalize to 1920x1080 @ 30fps (was keyframe-
  snapping with -c copy and breaking concat on mixed-source corpora)
- Add silent-audio anullsrc fallback for clips without an audio
  stream

README: add Documentary Montage row to the pipeline table and bump
the pipeline count from 11 to 12.
This commit is contained in:
calesthio
2026-04-10 16:04:28 -07:00
parent e815126f5a
commit 44baede67f
18 changed files with 4446 additions and 10 deletions
+165
View File
@@ -0,0 +1,165 @@
name: documentary-montage
version: "1.0"
description: >
Retrieval-first thematic montage pipeline. Builds a semantic corpus of real-world
footage from Pexels, Archive.org (Prelinger et al.), and NASA, then uses CLIP-based
retrieval to fill slot descriptions from a thematic brief. The edit arranges clips
by narrative beat with music sync and uniform color grade across mixed-era footage.
Inspired by Adam Curtis / Chris Marker / Errol Morris tone poems.
category: documentary
stability: beta
default_checkpoint_policy: guided
# Reference video input support
reference_input:
supported: false
orchestration:
mode: executive-producer
skill: pipelines/documentary-montage/executive-producer
budget_default_usd: 1.00
max_revisions_per_stage: 3
max_send_backs: 2
max_wall_time_minutes: 60
extensions:
custom_scripts: true
custom_playbooks: true
custom_skills: true
custom_tools: false
required_skills:
- pipelines/documentary-montage/executive-producer
- pipelines/documentary-montage/idea-director
- pipelines/documentary-montage/scene-director
- pipelines/documentary-montage/asset-director
- pipelines/documentary-montage/edit-director
- pipelines/documentary-montage/compose-director
- meta/reviewer
- meta/checkpoint-protocol
compatible_playbooks:
custom_allowed: true
stages:
- name: idea
skill: pipelines/documentary-montage/idea-director
produces:
- brief
tools_available: []
checkpoint_required: true
human_approval_default: true
review_focus:
- Thematic question is ONE sentence
- Tone register is ONE value from the fixed list
- Duration and shape are concrete
- Music plan is resolved or explicitly silent
success_criteria:
- Schema-valid brief artifact
- thematic_question present in metadata
- music_plan present in metadata
- name: scene_plan
skill: pipelines/documentary-montage/scene-director
required_artifacts_in:
- brief
produces:
- scene_plan
tools_available: []
checkpoint_required: true
human_approval_default: true
review_focus:
- Slot descriptions use concrete noun-and-adjective language
- Every slot has 2-3 short search queries
- At least 2 slots are marked hero
- Slot count and target holds match the tone/duration math
- era_mix is reflected in preferred_sources distribution
success_criteria:
- Schema-valid scene_plan artifact
- metadata.slots[] present with per-slot queries and preferred_sources
- Sum of target_hold_seconds within 10% of brief.duration_seconds
- name: assets
skill: pipelines/documentary-montage/asset-director
required_artifacts_in:
- scene_plan
- brief
produces:
- asset_manifest
required_tools:
- corpus_builder
- clip_search
optional_tools:
- music_gen
tools_available:
- corpus_builder
- clip_search
- music_gen
checkpoint_required: true
human_approval_default: false
review_focus:
- Corpus size is at least 8x the slot count
- Every slot has exactly one picked clip with score >= 0.22
- No clip_id is picked for two slots (exclude_ids enforced)
- diversify ran clean on the final timeline
- rejected_picks log has reasons for passes
- Provenance (provider, original_url, license) present on every asset
success_criteria:
- Schema-valid asset_manifest artifact
- One video asset per scene slot
- metadata.corpus_stats present
- name: edit
skill: pipelines/documentary-montage/edit-director
required_artifacts_in:
- scene_plan
- asset_manifest
produces:
- edit_decisions
tools_available: []
checkpoint_required: true
human_approval_default: true
review_focus:
- Hero slots hold longest; mid-sequence cutaways shortest
- No two adjacent cuts share subject AND scale
- Transition vocabulary is at most 4 distinct values
- Music config is present OR brief explicitly says no music
- total_duration_seconds matches sum of cut durations
- Every cut has a reason
success_criteria:
- Schema-valid edit_decisions artifact
- Timeline duration within 10% of brief.duration_seconds
- name: compose
skill: pipelines/documentary-montage/compose-director
required_artifacts_in:
- edit_decisions
- asset_manifest
- brief
produces:
- render_report
required_tools:
- video_compose
optional_tools:
- audio_mixer
- color_grade
- video_trimmer
- video_stitch
tools_available:
- video_compose
- audio_mixer
- color_grade
- video_trimmer
- video_stitch
checkpoint_required: true
human_approval_default: false
review_focus:
- Output duration matches planned within 1s
- Resolution matches target_platform canvas
- Uniform LUT applied across the timeline
- Music present iff brief planned for it
- First and last frames verified
- No silent fallback from a motion-led promise
success_criteria:
- Schema-valid render_report artifact
- Output file exists and passes ffprobe validation