Commit Graph

11 Commits

Author SHA1 Message Date
calesthio fdd6457fed docs(prompting): adopt 5-aspect video specification across skills
Incorporate the structured taxonomy from Lin et al. "Building a Precise
Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2).
The paper proves prompts structured around five aspects (Subject /
Subject Motion / Scene / Spatial Framing / Camera) unlock controllable
cinematography in fine-tuned video generation models. Off-the-shelf VLMs
already nail subject and scene; the gains live in motion, spatial, and
camera, which prompts routinely omit.

Universal layer (skills/creative/video-gen-prompting.md, +125 lines):
- 5-aspect prompt skeleton replaces flat formula
- Camera movements regrouped (translation / rotation / lens-only) with
  dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations
- New primitive tables: camera height, camera angle, POV, lens
  distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking),
  playback speed (6 modes), subject transitions
- Order-matters and self-contained-prompt rules
- Identity anchoring rule for multi-shot
- Strict static-shot rule, anti-subjective callout, overlays-not-depth
- Per-model word-count guidance

Per-model guides (sora, veo, hunyuan, ltx, seedance):
- Add the primitives each model honors literally
- Word-count sweet spots per model
- Strengthen seedance verbatim-identity and subject-transition guidance

Pipeline directors (cinematic / explainer / animation scene-director,
cinematic / explainer asset-director):
- 5-aspect scene-plan checklist (per-pipeline adapted)
- Overlays-not-depth callout
- Pre / critique / post self-review loop for generation prompts

Reviewer (skills/meta/reviewer.md):
- CHAI critique-quality rules: accurate / complete / constructive
- Critical findings now require a proposed_fix

Storytelling, cinematic, broll, video-reference-analyst:
- Anti-subjective rule (replace mood adjectives with visual causes)
- Camera-intent-per-beat for script writers
- POV column in stock-footage query templates
- 5-aspect structured output mandatory for reference-video analysis

skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.
2026-04-28 08:11:31 -07:00
calesthio b4f7ec4eee hyperframes: add HTML/CSS/GSAP as a parallel composition runtime
Separates creative grammar (renderer_family) from technical engine
(render_runtime) so HyperFrames can stand alongside Remotion as a
first-class runtime instead of masquerading as a Remotion sub-case.
Locks runtime choice at proposal stage and enforces it end-to-end: the
schemas require it, video_compose routes by it, the reviewer fails
closed on silent swaps, and a parametrized contract test walks every
pipeline manifest to ensure each planning-stage skill explains the
conversation to the user. Adds hyperframes_compose (scaffold/lint/
validate/render/doctor/add_block), a playbook -> CSS style bridge, and
vendored HyperFrames Layer 3 skills from commit d291358, pinned via
PROVENANCE.md for future re-sync. Final_review now records
render_runtime_used and runtime_swap_detected so compose lies are
catchable after the fact.
2026-04-18 18:37:00 -07:00
calesthio a37b58199a skills: adopt GSAP Layer 3 + Layer 2 animation-runtime routing
Mirror the official GSAP AI skills (greensock/gsap-skills, MIT) into
.agents/skills/ and add the Layer 2 wiring that makes them discoverable
from a fresh context.

GSAP covers animation needs that Remotion primitives strain at:
per-character text reveals (SplitText), SVG shape morphs (MorphSVG),
curved camera paths (MotionPath), stroke-reveal line drawing (DrawSVG),
layout-to-layout flight (Flip), and custom bezier easings (CustomEase).
Also becomes mandatory day-1 knowledge if we wire in HyperFrames later
(HF uses GSAP timelines as its native animation runtime).

Layer 3 adds (.agents/skills/):
- gsap-core, gsap-timeline, gsap-plugins, gsap-utils
- gsap-react, gsap-performance
- gsap-scrolltrigger, gsap-frameworks (situational)
- gsap/README.md — OpenMontage-specific framing and Remotion-safe usage

Layer 2 wiring (the discovery triggers):
- skills/meta/animation-runtime-selector.md — NEW routing meta-skill.
  Decision matrix covering Remotion primitives, GSAP plugins, framer-
  motion, Lottie, Manim, D3, TerminalScene. Enforces the "keep it
  simple" bias: reach for GSAP only when the plugin genuinely earns
  its bundle weight.
- skills/pipelines/explainer/asset-director.md — references GSAP for
  kinetic typography, multi-step choreography, SVG line draws.
- skills/pipelines/animation/asset-director.md — references GSAP for
  logo morphs, motion paths, FLIP transitions, custom easings.
- skills/pipelines/cinematic/asset-director.md — references GSAP for
  cinematic camera moves, per-char title reveals, prestige easings.
- AGENT_GUIDE.md — adds a categorized Layer 3 skills table so a fresh-
  context agent can find the right skill by what they're trying to do,
  plus a pointer to animation-runtime-selector.md for routing.

Determinism: every GSAP use inside Remotion must drive timeline progress
from useCurrentFrame(), never requestAnimationFrame. Three Remotion-safe
patterns are documented in both the gsap/README and the selector skill.

Attribution: https://github.com/greensock/gsap-skills (MIT).
2026-04-16 20:07:00 -07:00
calesthio 1b7e13d24b Fix Remotion-first rendering docs and post-render verification gaps
Compose-director had contradictory instructions: Step 2 described Remotion
captions/audio, but Steps 5/5b gave detailed FFmpeg code that agents followed
instead. This caused three failures in production: FFmpeg subtitles instead of
Remotion CaptionOverlay, missing audio (mixed externally but never embedded in
Remotion props), and skipped audio verification in post-render review.

Changes:
- compose-director: Remotion is now DEFAULT for audio, captions, text overlays;
  FFmpeg is labeled FALLBACK only. Post-render review has mandatory ffprobe gate
  and audio transcription with explicit stop conditions.
- remotion.md: routing table updated (captions/audio → Remotion), added universal
  Post-Render Verification Protocol for all pipelines (only 2/10 had one).
- scene-director, asset-director: added pitfall for AI-generated text in CTA
  screens — must use Remotion text_card for verbatim text.
- image-provider-usage: added Recraft V4 caveat (style param causes 422 on fal.ai).
- recraft_image.py: documented the style parameter 422 issue inline.
2026-04-06 08:21:48 -07:00
calesthio b0917d2d84 Add reference video input analysis workflow 2026-04-04 10:01:11 -07:00
calesthio fd4c58f30b Refine visual identity guidance across pipelines 2026-04-03 10:13:39 -07:00
calesthio 2cd36fa8e0 Implementation spec: governance, decision intelligence, theme system, and E2E bug fixes
Implements the 2026-04-02 transformation spec (Phases 1-8) and fixes all
critical bugs found during 5-pipeline E2E testing.

Governance & Decision Intelligence:
- Pipeline-specific stage order in checkpoint (replaces global STAGES list)
- Provider scoring engine (lib/scoring.py) with 7-dimension weighted ranking
- Decision log artifact enforced at proposal/idea stage across all 10 pipelines
- Delivery promise classifier prevents silent motion-to-still downgrades
- Structured shot language in scene_plan schema (camera, lens, lighting, DOF)
- Variation checker and slideshow risk scorer block samey output before render
- Creative intake, capability extension, and creative-intake meta skills
- Final self-review artifact with 5 mandatory checks before presenting output
- Source media review contract for user-supplied footage

Render & Theme System:
- Remotion AnimatedBackground now derives colors from playbook (no more hardcoded
  dark blue fintech gradient on every video)
- video_compose builds custom ThemeConfig from playbook YAML colors/fonts —
  custom playbooks flow through to Remotion automatically
- Explainer component wires theme to all child components (charts, cards, etc.)
- resolveAsset() handles absolute paths on Windows/Unix via file:// URIs
- RENDERER_FAMILY_MAP synced with actual Remotion compositions

Critical Bug Fixes:
- Windows npx subprocess: run_command() resolves .cmd wrappers via shutil.which()
- Silent renderer downgrade: Remotion failure now returns explicit error with
  options instead of silently falling back to FFmpeg
- .env inline comment parsing strips trailing # comments from API keys
- concat_path UnboundLocalError in video_compose finally block
- audio_mixer and showcase_card capture=True kwarg bug
- Selector estimate_cost() calls fixed (_select_tool -> _select_best_tool)
- asset_manifest schema expanded with provider, license, subtype fields
- screen-demo subtitle_gen moved from required to optional tools
- Duration drift detection in post-render final review (>25% warns)
2026-04-03 09:35:09 -07:00
calesthio 8cac647193 One-key formula: AI images, TTS narration, auto music, subtitles, and self-review pipeline
Prove that adding one API key (OPENAI_API_KEY) to the zero-key foundation
produces dramatically better videos for ~$0.69 each. Two compositions built:
The Abyss (deep ocean visual essay) and VOID (neural interface product ad).

New tools:
- audio_probe: ffprobe wrapper with probe_duration() helper
- composition_validator: pre-render checks (asset existence, audio-video sync)
- pixabay_music: royalty-free music scraper (no API key needed)
- freesound_music: Freesound API search + download

Remotion upgrades:
- BackgroundImageLayer: AI images behind data scenes with ken-burns + dark overlay
- Gradient support: all 9 components changed from backgroundColor to background CSS
- CaptionOverlay: word spacing fix, WhisperX word-level subtitles
- HeroTitle: reduced overlay opacity so background images show through

Process codified in agent skills:
- compose-director: audio acquisition flow (present user with voice/music/subtitle
  options), mandatory pre-render validation, post-render self-review (extract
  frames + transcribe + inspect + present findings to user)
- scene-director: narration duration budgeting (word budget from video duration)
- remotion skill: pre-render validation section
- TTS tool: now returns audio_duration_seconds in result

README updated with VOID product ad video embed.
2026-03-30 16:54:13 -07:00
calesthio 5223eec21c Zero-key video formula: fix rendering bugs, add 8 demo compositions, update skills
Fix ComparisonCard and CalloutBox dark-theme passthrough in SceneRenderer.
Add 8 demo-props JSON files (3 fixed originals + 5 new compositions).
Update scene-director and compose-director skills with all 10 Remotion
scene types and zero-key rendering guidance. Document the proven
all-dark-background formula in remotion.md. Add cinematic renderer,
onboarding skill, prompt gallery, and demo render script.
2026-03-30 13:23:28 -07:00
calesthio 2c16c6e547 Upgrade Remotion composition engine with cinematic enhancements
- Upgrade Remotion packages to 4.0.441, add transitions/captions/fonts/media
- Add spring-animated image scenes, stat reveals, section titles, hero cards
- Add TikTok-style word-by-word captions with highlight effect
- Add Google Fonts (Space Grotesk) and dynamic duration via calculateMetadata
- Fix Remotion false-positive: check node_modules/ in _remotion_available()
- Add project directory convention, music library, tool naming docs
- Add music transparency to proposal stage, subtitle pipeline to compose stage
- Add README showcase montage and Windows npm troubleshooting note
2026-03-29 12:06:06 -07:00
calesthio a3e735cc7a Initial release — OpenMontage: the first open-source agentic video production system
11 production pipelines, 47 tools, 124 agent skills.
Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and
free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 08:25:17 -07:00