Commit Graph

20 Commits

Author SHA1 Message Date
calesthio fdd6457fed docs(prompting): adopt 5-aspect video specification across skills
Incorporate the structured taxonomy from Lin et al. "Building a Precise
Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2).
The paper proves prompts structured around five aspects (Subject /
Subject Motion / Scene / Spatial Framing / Camera) unlock controllable
cinematography in fine-tuned video generation models. Off-the-shelf VLMs
already nail subject and scene; the gains live in motion, spatial, and
camera, which prompts routinely omit.

Universal layer (skills/creative/video-gen-prompting.md, +125 lines):
- 5-aspect prompt skeleton replaces flat formula
- Camera movements regrouped (translation / rotation / lens-only) with
  dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations
- New primitive tables: camera height, camera angle, POV, lens
  distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking),
  playback speed (6 modes), subject transitions
- Order-matters and self-contained-prompt rules
- Identity anchoring rule for multi-shot
- Strict static-shot rule, anti-subjective callout, overlays-not-depth
- Per-model word-count guidance

Per-model guides (sora, veo, hunyuan, ltx, seedance):
- Add the primitives each model honors literally
- Word-count sweet spots per model
- Strengthen seedance verbatim-identity and subject-transition guidance

Pipeline directors (cinematic / explainer / animation scene-director,
cinematic / explainer asset-director):
- 5-aspect scene-plan checklist (per-pipeline adapted)
- Overlays-not-depth callout
- Pre / critique / post self-review loop for generation prompts

Reviewer (skills/meta/reviewer.md):
- CHAI critique-quality rules: accurate / complete / constructive
- Critical findings now require a proposed_fix

Storytelling, cinematic, broll, video-reference-analyst:
- Anti-subjective rule (replace mood adjectives with visual causes)
- Camera-intent-per-beat for script writers
- POV column in stock-footage query templates
- 5-aspect structured output mandatory for reference-video analysis

skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.
2026-04-28 08:11:31 -07:00
calesthio b4f7ec4eee hyperframes: add HTML/CSS/GSAP as a parallel composition runtime
Separates creative grammar (renderer_family) from technical engine
(render_runtime) so HyperFrames can stand alongside Remotion as a
first-class runtime instead of masquerading as a Remotion sub-case.
Locks runtime choice at proposal stage and enforces it end-to-end: the
schemas require it, video_compose routes by it, the reviewer fails
closed on silent swaps, and a parametrized contract test walks every
pipeline manifest to ensure each planning-stage skill explains the
conversation to the user. Adds hyperframes_compose (scaffold/lint/
validate/render/doctor/add_block), a playbook -> CSS style bridge, and
vendored HyperFrames Layer 3 skills from commit d291358, pinned via
PROVENANCE.md for future re-sync. Final_review now records
render_runtime_used and runtime_swap_detected so compose lies are
catchable after the fact.
2026-04-18 18:37:00 -07:00
calesthio a37b58199a skills: adopt GSAP Layer 3 + Layer 2 animation-runtime routing
Mirror the official GSAP AI skills (greensock/gsap-skills, MIT) into
.agents/skills/ and add the Layer 2 wiring that makes them discoverable
from a fresh context.

GSAP covers animation needs that Remotion primitives strain at:
per-character text reveals (SplitText), SVG shape morphs (MorphSVG),
curved camera paths (MotionPath), stroke-reveal line drawing (DrawSVG),
layout-to-layout flight (Flip), and custom bezier easings (CustomEase).
Also becomes mandatory day-1 knowledge if we wire in HyperFrames later
(HF uses GSAP timelines as its native animation runtime).

Layer 3 adds (.agents/skills/):
- gsap-core, gsap-timeline, gsap-plugins, gsap-utils
- gsap-react, gsap-performance
- gsap-scrolltrigger, gsap-frameworks (situational)
- gsap/README.md — OpenMontage-specific framing and Remotion-safe usage

Layer 2 wiring (the discovery triggers):
- skills/meta/animation-runtime-selector.md — NEW routing meta-skill.
  Decision matrix covering Remotion primitives, GSAP plugins, framer-
  motion, Lottie, Manim, D3, TerminalScene. Enforces the "keep it
  simple" bias: reach for GSAP only when the plugin genuinely earns
  its bundle weight.
- skills/pipelines/explainer/asset-director.md — references GSAP for
  kinetic typography, multi-step choreography, SVG line draws.
- skills/pipelines/animation/asset-director.md — references GSAP for
  logo morphs, motion paths, FLIP transitions, custom easings.
- skills/pipelines/cinematic/asset-director.md — references GSAP for
  cinematic camera moves, per-char title reveals, prestige easings.
- AGENT_GUIDE.md — adds a categorized Layer 3 skills table so a fresh-
  context agent can find the right skill by what they're trying to do,
  plus a pointer to animation-runtime-selector.md for routing.

Determinism: every GSAP use inside Remotion must drive timeline progress
from useCurrentFrame(), never requestAnimationFrame. Three Remotion-safe
patterns are documented in both the gsap/README and the selector skill.

Attribution: https://github.com/greensock/gsap-skills (MIT).
2026-04-16 20:07:00 -07:00
calesthio a919fde450 Remove fal-first provider guidance 2026-04-08 13:04:32 -07:00
calesthio 7b40edb82c Fix gap #22 and add CinematicRenderer captions + music support
- AGENT_GUIDE.md: add "never read source code" rule — skills are the
  interface, not .py files
- animation.yaml: add Layer 2 skill-first guardrail in assets stage
- video-reference-analyst.md: Step 4b now mandates Layer 2 before Layer 3,
  explicitly forbids reading implementation code
- CinematicRenderer: add TikTok-style CaptionOverlay and separate music
  track support (narration + music as independent audio layers)
- cinematic/types.ts: add CinematicCaptionConfig and music prop types
2026-04-04 14:23:46 -07:00
calesthio 370f2f11b7 Default TTS recommendation to Google Chirp3-HD over ElevenLabs
Chirp3-HD is near-free ($0.003 vs $0.30+ per script), expressive,
24kHz, and available without quota limits. ElevenLabs demoted to
voice-cloning-only recommendation.
2026-04-04 14:08:34 -07:00
calesthio fc6ff2248a Add UAT guardrails to pipeline YAML and fix playbook test
- animation.yaml: enforce audio architecture + provider comparison in
  proposal stage, Layer 3 skill gate + clip duration in assets stage
- video-reference-analyst.md: Step 6 is now a hard redirect forcing
  stage-by-stage pipeline execution
- Fix test_compatible_with_manifest for nested compatible_playbooks dict
2026-04-04 13:29:24 -07:00
calesthio 93de8effbe Add mandatory Layer 3 skill gate before any asset generation
New Step 4b requires the agent to read agent_skills from every tool
before writing generation prompts. Discovered during UAT: agent
generated video clips, images, and TTS without reading provider-specific
prompting guidance, violating AGENT_GUIDE governance.
2026-04-04 13:20:19 -07:00
calesthio 97097e61a1 Present video gen provider options with costs to user, don't auto-pick
Agent must show a provider comparison table (quality, speed, cost per
clip) and recommend one, but let the user choose.
2026-04-04 13:18:15 -07:00
calesthio 1451f4fd69 Add clip duration optimization and Layer 3 skill reminder to proposals
Proposals must now specify clip duration strategy (prefer 10s over 5s
to halve API costs) and list Layer 3 skills that must be read before
generating assets. Both were governance gaps found during UAT.
2026-04-04 13:17:31 -07:00
calesthio 24a85337c3 List all TTS providers in proposal template, not just ElevenLabs
Agent must run tts_selector preflight to check available providers
instead of defaulting to ElevenLabs. Supports ElevenLabs, Google TTS,
OpenAI TTS, and Piper (offline).
2026-04-04 12:53:34 -07:00
calesthio cb1423a61f Add audio architecture decision to video-reference-analyst planning
The agent must lock the audio approach (single narrator vs. character
dialogue vs. both) during Step 3, not defer it to script/compose stage.
Proposals now include voice casting with specific voice IDs.
2026-04-04 12:51:47 -07:00
calesthio 16647d2d36 Enforce Remotion-first composition engine, fix FFmpeg fallback bugs
Remotion is now the default composition engine for ALL final renders
when available — video clips, images, mixed content. FFmpeg is only
used as fallback when Remotion is not installed or for standalone
operations (trim, transcode). Also fixes three FFmpeg fallback bugs:
profile + copy codec conflict, stream order mapping, and segment
seeking for audio-first containers.
2026-04-04 12:39:24 -07:00
calesthio 38b6ec5212 Add mandatory lightweight research step to video-reference-analyst
The reference analyst skill went straight from capability audit to
creative proposals with no research. This caused the agent to propose
concepts based solely on the reference analysis and its own knowledge,
missing content landscape context, technique best practices, and
subject-matter depth.

Added Step 3b between critical questions and creative proposals:
- Content landscape scan (3-5 similar existing videos)
- Style/technique research (AI model strengths, prompting patterns)
- Subject-matter research (facts, tropes, hooks)
- 2-3 minute time budget — lightweight, not full research-director
2026-04-04 11:59:11 -07:00
calesthio 61c704149e Enforce Remotion-first composition in video-reference-analyst skill
The AGENT_GUIDE establishes Remotion as the preferred composition engine
over FFmpeg, but the reference analyst skill was presenting them as peer
options. This caused the agent to default to FFmpeg during proposals.

- Capability audit template now labels Remotion as "preferred" and FFmpeg
  as "fallback only"
- Added explicit composition engine priority note
- Proposal template separates Composition and Motion as distinct lines
2026-04-04 11:43:17 -07:00
calesthio 286c26e33d Add per-scene motion classification to video analyzer
Video analyzer now uses Farneback dense optical flow to classify each
scene as motion_clip, animated_still, or static_image. This lets the
agent correctly identify whether a reference video uses AI-generated
video clips vs still images with pan/zoom — and plan the right pipeline.

Changes:
- video_analyzer.py: new Step 3b with _classify_scene_motion() and
  _read_frame_at() helpers; updated _needs_motion() to use per-scene
  motion data instead of pacing heuristic alone
- video-reference-analyst.md: added Motion line to summary template
  and instructions to read motion_type field before proposing tools
2026-04-04 11:33:33 -07:00
calesthio b0917d2d84 Add reference video input analysis workflow 2026-04-04 10:01:11 -07:00
calesthio 2cd36fa8e0 Implementation spec: governance, decision intelligence, theme system, and E2E bug fixes
Implements the 2026-04-02 transformation spec (Phases 1-8) and fixes all
critical bugs found during 5-pipeline E2E testing.

Governance & Decision Intelligence:
- Pipeline-specific stage order in checkpoint (replaces global STAGES list)
- Provider scoring engine (lib/scoring.py) with 7-dimension weighted ranking
- Decision log artifact enforced at proposal/idea stage across all 10 pipelines
- Delivery promise classifier prevents silent motion-to-still downgrades
- Structured shot language in scene_plan schema (camera, lens, lighting, DOF)
- Variation checker and slideshow risk scorer block samey output before render
- Creative intake, capability extension, and creative-intake meta skills
- Final self-review artifact with 5 mandatory checks before presenting output
- Source media review contract for user-supplied footage

Render & Theme System:
- Remotion AnimatedBackground now derives colors from playbook (no more hardcoded
  dark blue fintech gradient on every video)
- video_compose builds custom ThemeConfig from playbook YAML colors/fonts —
  custom playbooks flow through to Remotion automatically
- Explainer component wires theme to all child components (charts, cards, etc.)
- resolveAsset() handles absolute paths on Windows/Unix via file:// URIs
- RENDERER_FAMILY_MAP synced with actual Remotion compositions

Critical Bug Fixes:
- Windows npx subprocess: run_command() resolves .cmd wrappers via shutil.which()
- Silent renderer downgrade: Remotion failure now returns explicit error with
  options instead of silently falling back to FFmpeg
- .env inline comment parsing strips trailing # comments from API keys
- concat_path UnboundLocalError in video_compose finally block
- audio_mixer and showcase_card capture=True kwarg bug
- Selector estimate_cost() calls fixed (_select_tool -> _select_best_tool)
- asset_manifest schema expanded with provider, license, subtype fields
- screen-demo subtitle_gen moved from required to optional tools
- Duration drift detection in post-render final review (>25% warns)
2026-04-03 09:35:09 -07:00
calesthio 5223eec21c Zero-key video formula: fix rendering bugs, add 8 demo compositions, update skills
Fix ComparisonCard and CalloutBox dark-theme passthrough in SceneRenderer.
Add 8 demo-props JSON files (3 fixed originals + 5 new compositions).
Update scene-director and compose-director skills with all 10 Remotion
scene types and zero-key rendering guidance. Document the proven
all-dark-background formula in remotion.md. Add cinematic renderer,
onboarding skill, prompt gallery, and demo render script.
2026-03-30 13:23:28 -07:00
calesthio a3e735cc7a Initial release — OpenMontage: the first open-source agentic video production system
11 production pipelines, 47 tools, 124 agent skills.
Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and
free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 08:25:17 -07:00