Initial release — OpenMontage: the first open-source agentic video production system

11 production pipelines, 47 tools, 124 agent skills. Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 08:25:17 -07:00
commit a3e735cc7a
1147 changed files with 240221 additions and 0 deletions
@@ -0,0 +1,482 @@
+# OpenMontage Architecture
+
+> Last updated: 2026-03-28 | Derived from code exploration, not prior documentation.
+
+OpenMontage is an **agent-orchestrated video production platform**. An LLM coding assistant (Claude Code, Cursor, Copilot, etc.) acts as the orchestrator — reading pipeline manifests, following skill instructions, calling Python tools, and checkpointing state. There is no runtime Python orchestrator; the agent _is_ the control plane.
+
+---
+
+## High-Level Flow
+
+```
+User gives topic/idea
+        |
+        v
+Agent reads pipeline manifest (YAML)
+        |
+        v
+For each stage:
+   1. Agent reads stage-director skill (Markdown)
+   2. Agent calls Python tools via tool registry
+   3. Agent writes checkpoint (JSON) with artifacts
+   4. Agent self-reviews using meta/reviewer skill
+   5. Human approval gate (if configured)
+        |
+        v
+Final video output
+```
+
+---
+
+## Repository Layout
+
+```
+OpenMontage/
+├── lib/                    # Core runtime infrastructure (Python)
+│   ├── config_model.py     # Pydantic config: LLM, budget, checkpoint, output, paths
+│   ├── checkpoint.py       # Pipeline state persistence & stage transitions
+│   ├── pipeline_loader.py  # YAML manifest loading & validation
+│   ├── media_profiles.py   # Platform-specific render profiles (YouTube, TikTok, etc.)
+│   ├── env_loader.py       # .env variable management
+│   └── providers/          # (Reserved for future provider abstractions)
+│
+├── tools/                  # 55+ Python tool implementations
+│   ├── base_tool.py        # Abstract base class — the tool contract
+│   ├── tool_registry.py    # Auto-discovery singleton registry
+│   ├── cost_tracker.py     # Budget governance (estimate → reserve → reconcile)
+│   ├── analysis/           # Transcription, scene detection, frame sampling, video understanding
+│   ├── audio/              # TTS (ElevenLabs, OpenAI, Piper), music gen, mixing, enhancement
+│   ├── avatar/             # Talking head animation, lip sync
+│   ├── enhancement/        # Upscale, bg removal, face enhance/restore, color grading
+│   ├── graphics/           # Image gen (FLUX, DALL-E, Recraft, local diffusion), stock, diagrams, code snippets, math animation
+│   ├── publishers/         # (Reserved)
+│   ├── subtitle/           # SRT/VTT generation from timestamps
+│   └── video/              # 12 video gen providers, composition, stitching, trimming
+│
+├── pipeline_defs/          # 11 YAML pipeline manifests
+├── schemas/                # JSON Schema definitions for validation
+│   ├── artifacts/          # 11 artifact schemas (brief → publish_log)
+│   ├── checkpoints/        # Checkpoint state schema
+│   ├── pipelines/          # Pipeline manifest schema
+│   ├── styles/             # Style playbook schema
+│   └── tools/              # Tool-specific schemas
+│
+├── skills/                 # Layer 2: OpenMontage-specific agent instructions
+│   ├── core/               # FFmpeg, Remotion, WhisperX, color grading skills
+│   ├── creative/           # Video editing, enhancement, data viz, prompt engineering
+│   ├── meta/               # reviewer, checkpoint-protocol, skill-creator
+│   └── pipelines/          # Per-pipeline stage-director skills (10 pipelines)
+│
+├── .agents/skills/         # Layer 3: 47 external technology skills (FFmpeg, ElevenLabs, FLUX, etc.)
+├── styles/                 # Visual style playbooks (YAML) + loader
+├── remotion-composer/      # Node.js/React — Remotion video composition renderer
+├── tests/                  # Contract tests, QA integration tests, eval harness
+├── docs/                   # Best-practices guides, session handoffs, audits
+└── config.yaml             # Global runtime configuration
+```
+
+---
+
+## Core Architectural Principles
+
+### 1. Agent-First Orchestration
+
+There is **no Python orchestrator**. The LLM agent:
+- Reads the pipeline manifest to know the stage order
+- Reads each stage-director skill for detailed instructions
+- Calls tools, evaluates results, makes creative decisions
+- Writes checkpoints to persist state between stages
+
+Python provides **tools and persistence only**. All intelligence lives in skill instructions (Markdown) and pipeline manifests (YAML).
+
+### 2. No LLM API Key in Runtime
+
+OpenMontage does not call LLM APIs at runtime. The coding assistant running in the user's IDE _is_ the LLM. Tools that need generation (images, video, TTS) call domain-specific APIs directly (ElevenLabs, fal.ai, HeyGen, etc.), not general-purpose LLM endpoints.
+
+### 3. Dual-Provider Support
+
+Every capability must support both **API providers** (cloud, paid) and **local/open-source alternatives** (free, GPU-dependent). The selector pattern enforces this by routing to whatever is available.
+
+---
+
+## The Tool System
+
+### BaseTool Contract
+
+All tools inherit from `BaseTool` (ABC) and declare:
+
+| Field | Purpose |
+|-------|---------|
+| `name`, `version` | Identity |
+| `tier` | CORE, VOICE, ENHANCE, GENERATE, SOURCE, ANALYZE, PUBLISH |
+| `capability` | What it does (e.g., `tts`, `image_generation`, `video_post`) |
+| `provider` | Which service (e.g., `elevenlabs`, `ffmpeg`, `selector`) |
+| `runtime` | LOCAL, LOCAL_GPU, API, HYBRID |
+| `stability` | EXPERIMENTAL, BETA, PRODUCTION |
+| `dependencies` | Required binaries (`cmd:ffmpeg`), env vars (`env:ELEVENLABS_API_KEY`), Python packages (`python:torch`) |
+| `input_schema`, `output_schema` | JSON Schema for inputs/outputs |
+| `fallback_tools` | Ordered fallback chain |
+| `agent_skills` | Links to Layer 3 knowledge skills |
+| `resource_profile` | CPU, RAM, VRAM, disk, network requirements |
+| `retry_policy` | Max retries, backoff strategy |
+
+**Required method:** `execute(inputs) -> ToolResult`
+
+`ToolResult` carries: `success`, `data`, `artifacts` (file paths), `error`, `cost_usd`, `duration_seconds`, `seed`, `model`.
+
+### Tool Registry
+
+`ToolRegistry` is a singleton that auto-discovers all `BaseTool` subclasses via `pkgutil.walk_packages()`. No manual registration.
+
+Key queries:
+- `get_by_capability("tts")` — all TTS tools
+- `get_by_provider("elevenlabs")` — all ElevenLabs tools
+- `get_available()` — tools whose dependencies are satisfied
+- `find_fallback("elevenlabs_tts")` — resolve fallback chain
+- `support_envelope()` — full capability report for agent consumption
+- `gpu_required_tools()`, `network_required_tools()`
+
+### Selector Pattern
+
+Three selector tools abstract multi-provider capabilities:
+
+| Selector | Capability | Providers (priority order) |
+|----------|-----------|---------------------------|
+| `tts_selector` | Text-to-speech | ElevenLabs > OpenAI > Piper (offline) |
+| `image_selector` | Image generation | FLUX > DALL-E > Recraft > LocalDiffusion > Pexels/Pixabay (stock) |
+| `video_selector` | Video generation | Kling > Runway > VEO > MiniMax > HeyGen > LTX (modal) > LTX (local) > CogVideo > Hunyuan > WAN > Pexels/Pixabay (stock) |
+
+Selectors route based on: user preference > availability > fallback order. They adapt input schemas between providers transparently.
+
+### Tool Inventory by Category
+
+**Analysis (4):** transcriber (WhisperX), scene_detect, frame_sampler, video_understand (CLIP/BLIP-2)
+
+**Audio (7):** elevenlabs_tts, openai_tts, piper_tts, tts_selector, music_gen, audio_mixer, audio_enhance
+
+**Avatar (2):** talking_head (SadTalker/MuseTalk), lip_sync (Wav2Lip)
+
+**Enhancement (5):** upscale (Real-ESRGAN), bg_remove (rembg/U2Net), face_enhance, face_restore (CodeFormer/GFPGAN), color_grade (FFmpeg LUTs)
+
+**Graphics (11):** flux_image, openai_image, recraft_image, local_diffusion, pexels_image, pixabay_image, image_selector, code_snippet, diagram_gen, math_animate (ManimCE), image_gen (deprecated)
+
+**Subtitle (1):** subtitle_gen
+
+**Video (17):** heygen_video, veo_video, kling_video, runway_video, minimax_video, wan_video, hunyuan_video, cogvideo_video, ltx_video_local, ltx_video_modal, pexels_video, pixabay_video, video_selector, video_compose (FFmpeg), video_stitch, video_trimmer
+
+---
+
+## Pipeline System
+
+### Pipeline Manifests
+
+Each pipeline is a YAML file in `pipeline_defs/` defining:
+
+```yaml
+name: animated-explainer
+version: "2.0"
+category: generated          # talking_head | generated | hybrid | screen_recording | animation | cinematic | custom
+default_checkpoint_policy: guided
+
+orchestration:
+  mode: executive-producer
+  skill: pipelines/explainer/executive-producer
+  budget_default_usd: 2.00
+  max_revisions_per_stage: 3
+
+compatible_playbooks:
+  - clean-professional
+  - flat-motion-graphics
+
+stages:
+  - name: research
+    skill: pipelines/explainer/research-director
+    produces: [research_brief]
+    tools_available: []
+    checkpoint_required: false
+    human_approval_default: false
+    review_focus: [...]
+    success_criteria: [...]
+  # ... through publish
+```
+
+### Available Pipelines (11)
+
+| Pipeline | Category | Description |
+|----------|----------|-------------|
+| `animated-explainer` | generated | AI-produced explainer with research, narration, visuals, music |
+| `animation` | animation | Motion graphics, kinetic typography |
+| `avatar-spokesperson` | talking_head | Avatar-driven presenter videos |
+| `cinematic` | cinematic | Trailer, teaser, mood-driven edits |
+| `clip-factory` | custom | Batch short-form clips from long source |
+| `hybrid` | hybrid | Source footage + AI-generated support visuals |
+| `localization-dub` | custom | Subtitle, dub, and translate existing video |
+| `podcast-repurpose` | hybrid | Podcast highlights to video |
+| `screen-demo` | screen_recording | Software screen recordings and walkthroughs |
+| `talking-head` | talking_head | Footage-led speaker videos |
+| `framework-smoke` | custom | Minimal smoke test for framework validation |
+
+### Standard Stage Progression
+
+All production pipelines follow a canonical 8-stage flow:
+
+```
+research → proposal → script → scene_plan → assets → edit → compose → publish
+```
+
+Each stage:
+1. Has a **stage-director skill** (Markdown instructions for the agent)
+2. Declares **tools_available** (what the agent can call)
+3. **Produces** one or more canonical artifacts
+4. Has **review_focus** criteria and **success_criteria**
+5. Can require **human approval** before proceeding
+
+---
+
+## Checkpoint System
+
+Checkpoints persist pipeline state as JSON in the project's `pipeline/` directory.
+
+```json
+{
+  "version": "1.0",
+  "project_id": "my-video",
+  "stage": "script",
+  "status": "completed",
+  "timestamp": "2026-03-28T10:00:00Z",
+  "checkpoint_policy": "guided",
+  "human_approval_required": false,
+  "human_approved": true,
+  "artifacts": { "script": { ... } },
+  "review": { ... },
+  "cost_snapshot": { ... }
+}
+```
+
+**Status values:** `pending` | `in_progress` | `awaiting_human` | `completed` | `failed`
+
+**Checkpoint policies:**
+- `guided` — checkpoint at key creative stages, auto-proceed on mechanical ones
+- `manual_all` — human approval at every stage
+- `auto_noncreative` — auto-proceed unless stage is creative (assets, edit)
+
+**Functions:** `write_checkpoint()`, `read_checkpoint()`, `get_latest_checkpoint()`, `get_completed_stages()`, `get_next_stage()`
+
+### Canonical Artifacts (11 types, all JSON-schema validated)
+
+| Artifact | Stage | Contains |
+|----------|-------|----------|
+| `research_brief` | research | Landscape analysis, data points, audience insights, angles |
+| `proposal_packet` | proposal | Concept options, production plan, cost estimates, approval gate |
+| `brief` | idea | Title, hook, key points, tone, style, platform, duration |
+| `script` | script | Timestamped sections with enhancement cues, pronunciation guides |
+| `scene_plan` | scene_plan | Scene definitions with type, description, timing |
+| `asset_manifest` | assets | Generated assets with path, source tool, scene association |
+| `edit_decisions` | edit | Editorial cuts with in/out timings |
+| `render_report` | compose | Output metadata (format, resolution, duration) |
+| `publish_log` | publish | Platform publication entries with status |
+| `review` | (any) | Reviewer feedback and approval records |
+| `cost_log` | (any) | Budget tracking entries |
+
+---
+
+## Budget Governance
+
+The `CostTracker` enforces spending controls across the pipeline.
+
+### Lifecycle
+
+```
+estimate(tool, operation, $) → entry_id
+        |
+reserve(entry_id)          # locks budget
+        |
+reconcile(entry_id, $)     # records actual spend
+```
+
+### Budget Modes
+
+| Mode | Behavior |
+|------|----------|
+| `observe` | Track costs, no enforcement |
+| `warn` | Log warnings on overruns, allow execution |
+| `cap` | Reject operations that exceed remaining budget |
+
+### Controls
+- **Total budget** (default: $10.00)
+- **Reserve holdback** (default: 10%) — kept as safety margin
+- **Single-action approval threshold** (default: $0.50) — pause for approval above this
+- **New paid tool approval** — first-time use of any paid tool requires confirmation
+- Persists to `cost_log.json` per project
+
+---
+
+## 3-Layer Knowledge Architecture
+
+```
+Layer 3: .agents/skills/          External technology knowledge (47 skills)
+         "How the technology works"    FFmpeg, ElevenLabs API, FLUX, Remotion, Three.js, etc.
+              ^
+              | agent_skills[] references
+              |
+Layer 2: skills/                  OpenMontage conventions
+         "How this project uses the tech"  Pipeline integration, quality checklists, artifact mappings
+              ^
+              | stage skill references
+              |
+Layer 1: tools/ + pipeline_defs/  Executable capabilities + orchestration definitions
+         "What exists and when to use it"  BaseTool contracts, pipeline manifests
+```
+
+Each tool's `agent_skills[]` field links Layer 1 to Layers 2 and 3. For example:
+- `video_compose.agent_skills = ["remotion-best-practices", "remotion", "ffmpeg"]`
+- `tts_selector.agent_skills = ["text-to-speech", "elevenlabs", "openai-docs"]`
+
+---
+
+## Configuration
+
+### config.yaml
+
+```yaml
+llm:
+  provider: anthropic
+  temperature: 0.7
+  max_tokens: 4096
+
+budget:
+  mode: warn
+  total_usd: 10.00
+  reserve_pct: 0.10
+  single_action_approval_usd: 0.50
+
+checkpoint:
+  policy: guided
+  storage_dir: pipeline
+
+output:
+  default_format: mp4
+  default_codec: libx264
+  default_audio_codec: aac
+  default_resolution: 1920x1080
+  default_fps: 30
+  default_crf: 23
+
+paths:
+  pipeline_dir: pipeline
+  library_dir: library
+  styles_dir: styles
+  skills_dir: skills
+  output_dir: output
+```
+
+All config is validated via Pydantic models in `lib/config_model.py`.
+
+### Environment Variables (.env)
+
+| Variable | Used By | Purpose |
+|----------|---------|---------|
+| `ELEVENLABS_API_KEY` | elevenlabs_tts, music_gen | TTS, music, sound effects |
+| `OPENAI_API_KEY` | openai_tts, openai_image | TTS fallback, DALL-E 3 |
+| `FAL_KEY` | flux_image, kling_video, veo_video, minimax_video, recraft_image | fal.ai hosted models (FLUX, Veo, Kling, MiniMax, Recraft) |
+| `HEYGEN_API_KEY` | heygen_video | Multi-provider video generation |
+| `PEXELS_API_KEY` | pexels_image, pexels_video | Stock media |
+| `PIXABAY_API_KEY` | pixabay_image, pixabay_video | Stock media |
+| `RUNWAY_API_KEY` | runway_video | Runway Gen-4 direct |
+| `MODAL_LTX2_ENDPOINT_URL` | ltx_video_modal | Self-hosted LTX-2 |
+| `VIDEO_GEN_LOCAL_ENABLED` | local video tools | Enable local GPU generation |
+| `VIDEO_GEN_LOCAL_MODEL` | wan, hunyuan, ltx, cogvideo | Select local model |
+
+---
+
+## Visual Style System
+
+Style playbooks in `styles/` define visual language for pipelines:
+
+- `clean-professional.yaml` — Corporate, polished look
+- `flat-motion-graphics.yaml` — Modern flat design
+- `minimalist-diagram.yaml` — Technical, minimal diagrams
+
+Loaded by `styles/playbook_loader.py`. Each pipeline declares `compatible_playbooks` in its manifest. Validated against `schemas/styles/playbook.schema.json`.
+
+---
+
+## Media Profiles
+
+Platform-specific render configurations in `lib/media_profiles.py`:
+
+| Profile | Resolution | Aspect | Notes |
+|---------|-----------|--------|-------|
+| `youtube_landscape` | 1920x1080 | 16:9 | Standard YouTube |
+| `youtube_4k` | 3840x2160 | 16:9 | 4K YouTube |
+| `youtube_shorts` | 1080x1920 | 9:16 | Max 60s |
+| `instagram_reels` | 1080x1920 | 9:16 | Max 90s |
+| `instagram_feed` | 1080x1080 | 1:1 | Square |
+| `tiktok` | 1080x1920 | 9:16 | Vertical |
+| `linkedin` | 1920x1080 | 16:9 | Landscape |
+| `cinematic` | 2560x1080 | 21:9 | Ultrawide |
+
+Each profile specifies codec, audio codec, CRF, pixel format, max file size, max duration, and caption format. `ffmpeg_output_args(profile)` generates the corresponding FFmpeg flags.
+
+---
+
+## Remotion Composer
+
+A standalone Node.js/React subproject in `remotion-composer/` for programmatic video composition using [Remotion](https://www.remotion.dev/).
+
+- **React 18** + **Remotion 4.0** + **TypeScript 5.3**
+- Used by `video_compose` tool for complex compositions
+- Scripts: `start` (studio), `build` (render), `upgrade`
+
+---
+
+## Test Architecture
+
+```
+tests/
+├── contracts/              # Phase 0-3: tool contract validation, schema checks, registry tests
+├── qa/                     # Integration tests: TTS, image gen, music, audio mix, video compose/stitch, E2E
+├── eval/                   # Golden scenario replay harness for regression testing
+├── pipelines/              # Pipeline-level tests
+├── tools/                  # Individual tool tests
+└── styles/                 # Style playbook tests
+```
+
+**Contract tests** verify every tool satisfies the `BaseTool` contract: identity fields, schemas, dependency declarations, inheritance.
+
+**QA tests** call real tools (with real APIs/binaries) and inspect output quality.
+
+**Eval harness** (`tests/eval/replay_harness/`) replays golden scenarios with tolerance-based comparison for stochastic outputs.
+
+---
+
+## System Dependencies
+
+**Required:**
+- Python >= 3.10
+- FFmpeg (used by ~15 tools)
+
+**Optional (extend capabilities):**
+- Node.js (for Remotion composer)
+- GPU + CUDA (for local video/image generation)
+- Piper (offline TTS)
+- ManimCE (math animations)
+- Mermaid CLI (diagram generation)
+
+**Python packages:** pyyaml, pydantic, jsonschema, python-dotenv (core); pytest, pytest-asyncio (dev); torch, torchvision, torchaudio (GPU)
+
+---
+
+## Key Design Decisions
+
+1. **No runtime orchestrator** — The LLM agent reads YAML + Markdown and drives everything. This makes the system debuggable (just read the skill) and model-agnostic.
+
+2. **Checkpoint-based resumption** — Any stage can fail and the pipeline resumes from the last checkpoint. No re-running completed stages.
+
+3. **Schema-validated artifacts** — Every stage output is validated against a JSON Schema before the checkpoint is written. Prevents garbage propagation.
+
+4. **Budget as a first-class concept** — Cost estimation before execution, budget reservation, and reconciliation. The agent cannot silently overspend.
+
+5. **Selector pattern over hard-coded providers** — Capabilities degrade gracefully. Missing an API key? The selector falls through to the next provider or a local alternative.
+
+6. **Skills over code for intelligence** — Creative decisions, quality checklists, review criteria, and prompt templates live in Markdown skills, not Python. This means the agent's behavior can be tuned by editing text files, not code.
@@ -0,0 +1,402 @@
+# OpenMontage Capability Audit
+
+> 2026-03-28 | Code-verified. Every finding traced to specific file + line.
+
+**Core question:** When an agent discovers its capabilities via the registry and reads
+the skills/manifests, does it get an accurate picture of what it can and can't do?
+
+**Verdict:** No. The agent receives a mostly-accurate tool inventory but is actively
+misled at the skill-to-tool boundary. Skills promise inputs tools don't accept, schemas
+reject structures skills tell the agent to produce, and the discovery mechanism omits
+the one thing the agent needs most: what inputs each tool takes.
+
+---
+
+## 1. Discovery: What the Agent Learns at Boot
+
+The agent runs:
+```python
+from tools.tool_registry import registry
+registry.discover()
+envelope = registry.support_envelope()
+```
+
+**What it gets per tool** (via `BaseTool.get_info()`):
+- Identity: name, version, tier, capability, provider, stability
+- Status: available/unavailable/degraded, dependencies list
+- Runtime: execution_mode, determinism, runtime type, resource_profile
+- Decision hints: best_for, not_good_for, provider_matrix, fallback_tools
+- Skill links: agent_skills, related_skills
+- Side effects, resume_support, user_visible_verification
+
+**What it does NOT get:**
+| Missing Field | Why It Matters |
+|---|---|
+| `input_schema` | Agent can't know what parameters a tool accepts without reading source code |
+| `output_schema` | Agent can't know what a tool returns |
+| `artifact_schema` | Agent can't know what files a tool produces |
+| `estimate_cost()` results | Agent can't compare costs without calling each tool individually |
+
+**This is the root cause of most downstream issues.** The agent knows *what tools exist* but not *what they accept or produce*. It must rely on skills for that information — and skills get it wrong in several places.
+
+### Fix
+
+Add to `BaseTool.get_info()`:
+```python
+"input_schema": self.input_schema,
+"output_schema": self.output_schema,
+"artifact_schema": self.artifact_schema,
+```
+
+This single change would let the agent validate skill instructions against actual tool contracts at runtime.
+
+---
+
+## 2. Skills Tell the Agent to Pass Inputs Tools Don't Accept
+
+These are cases where a stage-director skill instructs the agent to call a tool with
+specific parameters, but the tool's `input_schema` and `execute()` don't read them.
+The agent follows the skill, the extra fields are silently ignored, and the output
+doesn't reflect the agent's intent.
+
+### 2a. tts_selector: voice style, speaker directions, pronunciation
+
+**Skill says** (`skills/pipelines/explainer/asset-director.md` lines 54-62):
+> Apply speaker directions from the script (pace, emphasis, emotion).
+> Apply the playbook's `audio.voice_style`.
+> Include a pronunciation map in the TTS request for technical terms.
+
+**Tool accepts** (`tools/audio/tts_selector.py` input_schema):
+`text`, `preferred_provider`, `allowed_providers`, `output_path`
+
+No fields for voice_style, speaker_directions, pace, emphasis, emotion, or pronunciation.
+None of the concrete TTS tools (elevenlabs, openai, piper) accept these either — except
+elevenlabs_tts which accepts `stability`, `similarity_boost`, `style` (numeric 0-1 floats),
+not the semantic concepts the skill describes.
+
+**Agent impact:** Agent is told to express creative direction through tool parameters that
+don't exist. The narration will be generated with default voice settings regardless of what
+the script specifies.
+
+### 2b. image_selector: negative_prompt, consistency_anchors
+
+**Skill says** (`skills/pipelines/explainer/asset-director.md` lines 68-73):
+> Build the prompt: playbook.image_prompt_prefix + scene description + style cues.
+> Add negative prompt from playbook.
+> Include consistency anchors (same palette, same style across all images).
+
+**Tool accepts** (`tools/graphics/image_selector.py` input_schema):
+`prompt`, `preferred_provider`, `allowed_providers`, `output_path`
+
+No `negative_prompt` or `consistency_anchors` field. The selector passes `prompt` through
+to whichever provider it selects. Some downstream providers (flux_image) DO accept
+`negative_prompt`, but the selector doesn't forward it.
+
+**Agent impact:** Agent must flatten everything into a single `prompt` string. Negative
+prompts and consistency anchors are lost unless the agent bypasses the selector and calls
+flux_image directly — which contradicts the skill's guidance.
+
+### 2c. video_compose "render" operation: options, output_profile
+
+**Skill says** (`skills/pipelines/explainer/compose-director.md` lines 66-80):
+```json
+{
+  "operation": "render",
+  "edit_decisions": {...},
+  "asset_manifest": {...},
+  "output_profile": "youtube_landscape",
+  "options": {
+    "subtitle_burn": true,
+    "audio_normalize": true,
+    "two_pass_encode": true
+  }
+}
+```
+
+**Tool accepts** (`tools/video/video_compose.py` input_schema):
+`operation`, `input_path`, `output_path`, `edit_decisions`, `subtitle_path`,
+`subtitle_style`, `overlays`, `audio_path`, `profile`, `codec`, `crf`, `preset`
+
+- `asset_manifest`: NOT in input_schema but IS read by `_render()` (line 271). Works but undocumented.
+- `output_profile`: Not accepted. The field is `profile` and only used by `_encode()`, not `_render()`.
+- `options`: Not accepted at all. `subtitle_burn`, `audio_normalize`, `two_pass_encode` are silently dropped.
+
+**Agent impact:** Agent follows the skill, passes these fields, they vanish. Output won't
+match target platform. No subtitle burn, no normalization, no two-pass encode.
+
+### 2d. audio_mixer: skill implies single call, tool requires multiple
+
+**Skill says** (`skills/pipelines/explainer/compose-director.md` lines 87-95):
+> Call the audio_mixer tool to:
+> 1. Layer narration segments in order
+> 2. Mix background music at playbook volume
+> 3. Apply ducking (music dips during narration)
+> 4. Normalize overall audio levels
+
+**Tool accepts** (`tools/audio/audio_mixer.py`):
+Three discrete operations: `mix`, `duck`, `extract`. Each is a separate `execute()` call.
+There is no single-call "mix everything together" mode.
+
+**Agent impact:** Agent attempts a single call with all requirements. The tool handles one
+operation per call. Agent must orchestrate: mix narration tracks first, then duck with
+music, then the compose step muxes audio into video. The skill doesn't explain this
+multi-step choreography.
+
+---
+
+## 3. Skills Tell the Agent to Produce Artifacts Schemas Reject
+
+These are cases where a skill shows the agent an example JSON structure for an artifact,
+but the corresponding JSON Schema has `additionalProperties: false` and rejects the
+skill's fields.
+
+### 3a. edit_decisions: transforms and transitions
+
+**Skill says** (`skills/pipelines/explainer/edit-director.md` lines 38-53):
+```json
+{
+  "id": "cut-1",
+  "source": "img-scene-1",
+  "in_seconds": 0,
+  "out_seconds": 10,
+  "layer": "primary",
+  "transform": {
+    "scale": 1.0,
+    "position": "center",
+    "animation": "ken-burns-slow-zoom"
+  },
+  "transition_in": "fade",
+  "transition_out": "dissolve",
+  "transition_duration": 0.4
+}
+```
+
+**Schema allows** (`schemas/artifacts/edit_decisions.schema.json` lines 12-24):
+```json
+{
+  "required": ["id", "source", "in_seconds", "out_seconds"],
+  "properties": {
+    "id": {}, "source": {}, "in_seconds": {}, "out_seconds": {},
+    "speed": {}, "reason": {}
+  },
+  "additionalProperties": false
+}
+```
+
+**Rejected fields:** `layer`, `transform` (and all sub-fields), `transition_in`,
+`transition_out`, `transition_duration`.
+
+**Agent impact:** Agent builds the edit_decisions artifact per skill instructions.
+Checkpoint validation rejects it. Agent is stuck between following the skill and
+passing schema validation. The schema also means `video_compose._compose()` will
+never receive transition/transform data even if it could handle it — which it can't
+(it only does hard cuts via concat).
+
+### 3b. edit_decisions: subtitle styling
+
+**Skill says** (edit-director.md lines 62-78):
+```json
+"subtitles": {
+  "enabled": true,
+  "style": "word-by-word",
+  "font": "Inter",
+  "font_size": 48,
+  "color": "#FFFFFF",
+  "background": "#00000088",
+  "position": "bottom-center",
+  "max_words_per_line": 8
+}
+```
+
+**Schema allows** (edit_decisions.schema.json lines 51-58):
+```json
+"subtitles": {
+  "properties": {
+    "enabled": { "type": "boolean" },
+    "style": { "type": "string" },
+    "source": { "type": "string" }
+  }
+}
+```
+
+**Rejected fields:** `font`, `font_size`, `color`, `background`, `position`,
+`max_words_per_line`.
+
+**Agent impact:** All subtitle styling information is lost at schema validation.
+The compose stage gets `enabled: true` and `style: "word-by-word"` but no font,
+color, or positioning data. Subtitle styling must come from video_compose's
+`subtitle_style` input instead — but the edit-director skill doesn't mention this path.
+
+### 3c. edit_decisions: narration and SFX audio configuration
+
+**Skill says** (edit-director.md lines 84-110):
+```json
+"audio": {
+  "narration": {
+    "segments": [
+      { "asset_id": "narration-s1", "start_seconds": 0 }
+    ]
+  },
+  "music": {
+    "asset_id": "...",
+    "ducking": {
+      "enabled": true,
+      "threshold_db": -3,
+      "reduction_db": -8,
+      "attack_ms": 200,
+      "release_ms": 500
+    }
+  },
+  "sfx": []
+}
+```
+
+**Schema allows** (edit_decisions.schema.json lines 59-67):
+`music` with `asset_id`, `volume`, `ducking` (boolean only), `fade_in_seconds`, `fade_out_seconds`.
+
+**Rejected:** Entire `audio.narration` section, entire `audio.sfx` section, structured
+`ducking` object (schema only accepts boolean). Narration segment timing and SFX
+have no home in the validated artifact.
+
+---
+
+## 4. Tools Lie About What They Can Do
+
+### 4a. video_compose._render() claims image-to-video conversion
+
+**Docstring** (`tools/video/video_compose.py` line 267):
+> "It orchestrates: image-to-video conversion, concatenation, audio mixing,
+> subtitle burn-in, and final encoding to target profile."
+
+**Actual code** (lines 280-302): Resolves asset IDs from asset_manifest to file paths,
+then calls `_compose()`. No image-to-video conversion. `_compose()` uses `ffmpeg -ss/-to`
+on each source, which fails on still images (PNG/JPG have no temporal dimension to seek into).
+
+**Agent impact:** Agent trusts that the "render" operation handles images. It doesn't.
+Explainer pipelines where most cuts are generated images will produce broken output.
+
+### 4b. subtitle_gen declares highlight_style but ignores it
+
+**Input schema** (`tools/subtitle/subtitle_gen.py`): includes `highlight_style` parameter.
+
+**Execute method:** Never reads `highlight_style`. It's a phantom parameter.
+
+**Agent impact:** Low — agent might pass highlight_style expecting highlighted captions
+and get plain ones.
+
+### 4c. video_selector declares operation parameter but ignores it
+
+**Input schema** (`tools/video/video_selector.py`): includes `operation` field.
+
+**Execute method:** Reads it but never passes it to the selected provider tool.
+
+**Agent impact:** Low — operation context is lost during delegation but most video
+providers only do one thing anyway.
+
+---
+
+## 5. Phantom Tools in Pipeline Manifests
+
+`pipeline_defs/framework-smoke.yaml` references three tools that don't exist:
+
+| Tool Name | Stage | Field |
+|---|---|---|
+| `idea_explorer_llm` | research | preferred_tools |
+| `script_writer_llm` | script | preferred_tools |
+| `script_writer_template` | script | fallback_tools |
+
+`registry.discover()` will not find these. If the agent tries to resolve them, it gets
+nothing. These appear to be placeholders from an earlier design where LLM capabilities
+were modeled as tools.
+
+**Agent impact:** framework-smoke pipeline is broken. Other 10 pipelines are clean —
+all their tool references resolve to real tools.
+
+---
+
+## 6. Unused Tools the Agent Doesn't Know to Use
+
+Five tools exist in `tools/` but no pipeline manifest references them:
+
+| Tool | What It Does | Why It's Missing |
+|---|---|---|
+| `bg_remove` | Background removal (rembg/U2Net) | Useful for compositing, overlays. No skill mentions it. |
+| `face_restore` | Face restoration (CodeFormer/GFPGAN) | Useful for low-res faces in talking-head. No skill mentions it. |
+| `upscale` | Image/video upscaling (Real-ESRGAN) | Useful for enhancement pass. No skill mentions it. |
+| `video_understand` | Vision-language analysis (CLIP/BLIP-2) | Could be used for quality review. No skill mentions it. |
+| `image_gen` | Legacy multi-provider image gen | Deprecated, replaced by image_selector. Expected. |
+
+The agent discovers these via `support_envelope()` but has no skill guidance on when or
+how to use them. They're invisible capabilities — the agent CAN use them but WON'T
+unless it independently decides to.
+
+---
+
+## 7. The Playbook Gap: Declared Knowledge the Agent Can't Apply
+
+Style playbooks (`styles/*.yaml`) define detailed visual contracts:
+`image_prompt_prefix`, `negative_prompt`, `consistency_anchors`, `typography`,
+`motion.transition_duration_seconds`, `audio.music_mood`, `quality_rules`.
+
+Skills extensively reference these: "apply the playbook's voice_style", "use playbook's
+image_prompt_prefix", "set music volume per playbook". **30+ skill references across
+all pipelines.**
+
+But:
+- Zero tools import `playbook_loader` or read playbook data
+- The selector tools (tts_selector, image_selector) have no playbook-aware parameters
+- The agent must: load the YAML, extract values, manually inject them into tool calls
+
+This works IF the agent knows to do it. But the tools give no signal that they expect
+playbook-derived values. The playbook system is a skill-layer convention that the tool
+layer is unaware of. This is architecturally fine for an agent-first system — but the
+skills need to be explicit about HOW the agent should bridge this (e.g., "read
+`playbook.asset_generation.image_prompt_prefix` and prepend it to the `prompt` field
+when calling image_selector"). Currently skills say WHAT to apply but not HOW to
+translate playbook values into tool parameters.
+
+---
+
+## 8. Missing Skill: Talking-Head Executive Producer
+
+`pipeline_defs/talking-head.yaml` declares `orchestration.skill: pipelines/talking-head/executive-producer` but the file `skills/pipelines/talking-head/executive-producer.md` does not exist. All other 9 pipelines with orchestration have their EP skill.
+
+**Agent impact:** Agent following the talking-head pipeline reads the manifest, tries to
+load the EP skill, gets a file-not-found. Must improvise orchestration without guidance.
+
+---
+
+## Summary: What the Agent Gets Right vs. Wrong
+
+### Accurate (agent can trust these)
+- Tool inventory: 44/47 tool references in manifests resolve correctly (3 phantoms in framework-smoke only)
+- Tool status: `get_status()` accurately reports availability based on real dependency checks
+- Fallback chains: All `fallback_tools` references point to tools that exist
+- Tool self-description: 10/12 tools audited have accurate input_schema vs execute() behavior
+- Pipeline stage ordering and artifact flow: checkpoint system matches manifests
+- Cost estimates: Tools that implement `estimate_cost()` return realistic numbers
+
+### Inaccurate (agent will be misled by these)
+| Issue | Count | Root Cause |
+|---|---|---|
+| Skills pass inputs tools don't accept | 4 cases | Skills written to aspirational API, not actual tool schema |
+| Skills produce artifacts schemas reject | 3 cases | Schemas stricter than skills expect (additionalProperties: false) |
+| Tool overclaims capability | 1 case | video_compose._render() docstring lies about image handling |
+| Discovery omits input/output schemas | All tools | get_info() doesn't include schemas |
+| Phantom tools in manifests | 3 tools | framework-smoke has placeholder references |
+| Missing EP skill | 1 pipeline | talking-head pipeline incomplete |
+
+### Recommended Fix Priority
+
+| # | Fix | Effort | Effect |
+|---|---|---|---|
+| 1 | Add input_schema/output_schema to `get_info()` | 3 lines | Agent can self-validate against skills |
+| 2 | Align edit_decisions.schema.json with edit-director skills | Medium | Unblocks transitions, subtitle styling, narration config |
+| 3 | Add image-to-video loop in video_compose._compose() | Small | Unblocks all image-based pipelines |
+| 4 | Add negative_prompt passthrough to image_selector | Small | Enables playbook integration for image gen |
+| 5 | Add voice_id/style passthrough to tts_selector | Small | Enables creative direction for narration |
+| 6 | Fix compose-director to match video_compose's actual interface | Small | Agent stops passing phantom options |
+| 7 | Fix compose-director to document multi-call audio_mixer flow | Small | Agent knows to call mix then duck |
+| 8 | Remove phantom tools from framework-smoke.yaml | Trivial | Clean manifest |
+| 9 | Create talking-head executive-producer.md | Medium | Completes pipeline |
+| 10 | Document playbook-to-tool bridging pattern in skills | Medium | Agent knows HOW to apply playbook values |