calesthio
a8d1ebdf6f
All of these changes came out of running the P1 and P2 documentary-montage
audits end-to-end and watching specific things break. Grouping them into
one commit because they share a theme: making the corpus builder and its
stock source adapters robust enough that a real brief can produce a
real corpus without special-casing.
corpus_builder.py + new clip_cache.py + test_clip_cache.py
Phase 1 of the shared-corpus architecture. Adds
~/.openmontage/clips_cache/ — a process-safe, LRU-evicted cache of
downloaded clip files keyed by clip_id. Before each candidate download,
corpus_builder asks the cache whether the bytes already exist on disk
from a previous run; on a hit it hard-links (or copies on cross-drive)
the blob into the caller's corpus dir and skips the network fetch. On
a miss it downloads as usual and ingests the fresh file. Motivation:
re-running the P1 audit after every tool fix was re-downloading gigs
of archive.org footage that had already been fetched in the prior run.
Cache faults never block the pipeline — they degrade gracefully to
normal downloads. The cache bubbles counters into the corpus_builder
return payload so the production report can show hit/miss/bytes-saved.
Default 20 GB cap, overridable via OPENMONTAGE_CACHE_MAX_GB.
Full test coverage: try_link, ingest, stats, LRU eviction, manifest
persistence, lock behavior. 23 tests, tmp_path-scoped.
stock_sources/archive_org.py
1. Three-strategy query cascade (phrase_prox_10 → distinctive_and →
distinctive_or). Motivation: natural-language documentary queries
against archive.org Solr were zeroing out — "1950s family watching
television" returned 0 hits because Solr's default multi-term AND
over-intersects. Walks strict to loose and returns the first
non-empty strategy. Stop words, source hints ("prelinger",
"archive", "footage"), and year tokens ("1950s") are excluded from
the distinctive-token picks since they don't correlate with
Prelinger title tokens.
2. 150 MB per-rendition size cap. archive.org routinely hosts
multi-hundred-megabyte h.264 masters and one 2 GB Prelinger item
poisons corpus build wall-time and disk. Within a format bucket
we now pick the largest rendition under the cap; if nothing fits
we fall through to the next format rather than dropping the item.
3. 180 s default max-duration ceiling when the caller hasn't set one
— archive.org is the only source that routinely hosts feature-
length material and a naive fan-out pulls them into corpora that
only ever want a few seconds per clip.
stock_sources/wikimedia.py
Parallel 3-strategy cascade (full → top2_or → single_best). Motivation:
Commons CirrusSearch also defaults to AND across multi-word queries;
our first P2 diagnostic pass returned 0 video results for 10/10
queries. Same stop-word / source-hint / year-token stripping as
archive_org so the two adapters stay symmetric.
test_stock_source_adapters.py
Rewrote the wikimedia query-builder tests against the new cascade
API. Added coverage for multi-word fallback + source-hint/year
stripping.
video_compose.py
Two small fixes for the Remotion renderer on Windows:
1. Resolve output_path to absolute before invoking the CLI so the
binary can write wherever the caller asked regardless of cwd.
2. Pass cwd=composer_dir to run_command so npx can find the local
Remotion binary under node_modules/.bin. Without this, Windows
npx returns "could not determine executable to run" because it
resolves .bin relative to the process cwd rather than the
script's parent.
75 lines
1.6 KiB
Plaintext
75 lines
1.6 KiB
Plaintext
# Python
|
|
__pycache__/
|
|
*.pyc
|
|
*.pyo
|
|
*.egg-info/
|
|
dist/
|
|
build/
|
|
|
|
# Test/IDE
|
|
.pytest_cache/
|
|
.mypy_cache/
|
|
.ruff_cache/
|
|
htmlcov/
|
|
.coverage
|
|
.trae/
|
|
.idea/
|
|
.vscode/
|
|
|
|
# Pipeline outputs (regenerable — never commit large media)
|
|
output/
|
|
pipeline/
|
|
tests/qa/output/
|
|
|
|
# Project workspaces (generated assets, renders — all regenerable)
|
|
projects/
|
|
|
|
# Corpus directories (downloaded stock clips, thumbnails, embeddings —
|
|
# regenerable from source APIs via corpus_builder). Explicit rule even
|
|
# though most corpora live under projects/ since agents sometimes
|
|
# build scratch corpora at the repo root.
|
|
corpus/
|
|
**/corpus/
|
|
|
|
# User music library (personal royalty-free tracks — not part of repo)
|
|
music_library/
|
|
|
|
# Environment
|
|
.env
|
|
.env.local
|
|
*.env
|
|
|
|
# Internal docs (strategy, handoffs, playbooks — not shipped)
|
|
internal/
|
|
|
|
# Local agent config (user-specific)
|
|
.claude/memory/
|
|
.claude/settings.local.json
|
|
|
|
# Agent skills lock (local install tracking)
|
|
skills-lock.json
|
|
|
|
# Generated images (logo drafts — final logo lives in assets/)
|
|
logo_*.png
|
|
generated_image.png
|
|
|
|
# Downloaded music files (Pixabay, Freesound — not source code)
|
|
pixabay_music_*
|
|
|
|
# OS
|
|
.DS_Store
|
|
Thumbs.db
|
|
|
|
# Node (for remotion-composer/)
|
|
node_modules/
|
|
|
|
# Remotion build output and copied project assets (regenerable at render time)
|
|
remotion-composer/out/
|
|
remotion-composer/public/*
|
|
# But keep demo props (shipped with the project for zero-key demos)
|
|
!remotion-composer/public/demo-props/
|
|
# Ignore test/scratch compositions in demo-props (keep only curated demos)
|
|
remotion-composer/public/demo-props/test-*
|
|
remotion-composer/public/demo-props/talking-head-*
|
|
remotion-composer/public/demo-props/caption-burn-*
|