Implementation spec: governance, decision intelligence, theme system, and E2E bug fixes
Implements the 2026-04-02 transformation spec (Phases 1-8) and fixes all critical bugs found during 5-pipeline E2E testing. Governance & Decision Intelligence: - Pipeline-specific stage order in checkpoint (replaces global STAGES list) - Provider scoring engine (lib/scoring.py) with 7-dimension weighted ranking - Decision log artifact enforced at proposal/idea stage across all 10 pipelines - Delivery promise classifier prevents silent motion-to-still downgrades - Structured shot language in scene_plan schema (camera, lens, lighting, DOF) - Variation checker and slideshow risk scorer block samey output before render - Creative intake, capability extension, and creative-intake meta skills - Final self-review artifact with 5 mandatory checks before presenting output - Source media review contract for user-supplied footage Render & Theme System: - Remotion AnimatedBackground now derives colors from playbook (no more hardcoded dark blue fintech gradient on every video) - video_compose builds custom ThemeConfig from playbook YAML colors/fonts — custom playbooks flow through to Remotion automatically - Explainer component wires theme to all child components (charts, cards, etc.) - resolveAsset() handles absolute paths on Windows/Unix via file:// URIs - RENDERER_FAMILY_MAP synced with actual Remotion compositions Critical Bug Fixes: - Windows npx subprocess: run_command() resolves .cmd wrappers via shutil.which() - Silent renderer downgrade: Remotion failure now returns explicit error with options instead of silently falling back to FFmpeg - .env inline comment parsing strips trailing # comments from API keys - concat_path UnboundLocalError in video_compose finally block - audio_mixer and showcase_card capture=True kwarg bug - Selector estimate_cost() calls fixed (_select_tool -> _select_best_tool) - asset_manifest schema expanded with provider, license, subtype fields - screen-demo subtitle_gen moved from required to optional tools - Duration drift detection in post-render final review (>25% warns)
This commit is contained in:
calesthio
@@ -0,0 +1,115 @@
|
||||
# Capability Extension Protocol
|
||||
|
||||
## When to Use
|
||||
|
||||
When you encounter a production need that no existing tool covers. The agent can extend the system — but with guardrails. This replaces the blanket "do NOT write ad-hoc Python scripts" rule with a structured protocol.
|
||||
|
||||
## Assessment First
|
||||
|
||||
Before writing anything, classify the gap:
|
||||
|
||||
| Gap Type | Example | Action |
|
||||
|----------|---------|--------|
|
||||
| **One-off transform** | Custom image crop, color adjustment, format conversion | Write a project-scoped Python script |
|
||||
| **Recurring visual need** | New illustration style, custom chart type | Generate a custom playbook or Remotion component |
|
||||
| **Missing provider** | User wants a specific API not in the registry | Create a minimal tool wrapper |
|
||||
| **Missing knowledge** | Agent doesn't know how to prompt a specific model | Use web search to learn, then document as a Layer 3 skill |
|
||||
|
||||
## Rules for Ad-Hoc Scripts
|
||||
|
||||
Scripts are allowed ONLY when:
|
||||
1. No existing tool covers the need (verified against registry via preflight)
|
||||
2. The script is idempotent (safe to re-run)
|
||||
3. The script produces a file artifact in the project workspace
|
||||
4. The script is logged in the decision log: `category: "capability_extension"`
|
||||
5. The user is informed: "I wrote a custom script for X because no existing tool handles Y"
|
||||
6. The script does NOT call external APIs without user approval
|
||||
|
||||
Scripts go in: `projects/<project-name>/scripts/`
|
||||
|
||||
### Script Template
|
||||
|
||||
```python
|
||||
"""<One-line description of what this script does>
|
||||
|
||||
Created by capability extension protocol because: <reason no existing tool covers this>
|
||||
Decision log entry: <decision_id>
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
def main(input_path: str, output_path: str) -> None:
|
||||
# Idempotent: check if output already exists
|
||||
out = Path(output_path)
|
||||
if out.exists():
|
||||
print(f"Output already exists: {out}")
|
||||
return
|
||||
|
||||
# ... transformation logic ...
|
||||
|
||||
print(f"Created: {out}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1], sys.argv[2])
|
||||
```
|
||||
|
||||
## Rules for Custom Playbooks
|
||||
|
||||
When the existing playbooks don't match the brief:
|
||||
1. Use `lib/playbook_generator.py` to create a new playbook
|
||||
2. Base it on the closest existing playbook if possible
|
||||
3. Validate against `schemas/styles/playbook.schema.json`
|
||||
4. Save to `styles/custom/<project-name>.yaml`
|
||||
5. Log as decision: `category: "playbook_selection"`, `subject: "custom playbook created"`
|
||||
|
||||
## Rules for New Skills (Technique Learning)
|
||||
|
||||
When the agent discovers technique knowledge during web research:
|
||||
1. Document it as a project-scoped skill: `projects/<project-name>/skills/<name>.md`
|
||||
2. Follow the Layer 3 skill format:
|
||||
- Provider name and version
|
||||
- Provider-specific prompting patterns
|
||||
- Optimal parameters for this use case
|
||||
- Quality tips and known failure modes
|
||||
- Source URLs for the information
|
||||
3. Reference it in the decision log
|
||||
4. Suggest promoting to `.agents/skills/` if it's generally useful
|
||||
|
||||
## Rules for Tool Wrappers
|
||||
|
||||
When a user needs a specific provider that isn't in the registry:
|
||||
1. The agent can create a minimal `BaseTool` subclass
|
||||
2. Save to `projects/<project-name>/tools/<name>.py`
|
||||
3. It MUST inherit from `BaseTool` and implement the full contract (input_schema, execute, capabilities, etc.)
|
||||
4. It MUST be registered before use
|
||||
5. Log as decision: `category: "capability_extension"`
|
||||
6. Requires user approval before first paid API call
|
||||
|
||||
## What Is Still Forbidden
|
||||
|
||||
- Bypassing the pipeline (all production still goes through stages)
|
||||
- Calling external APIs without user knowledge
|
||||
- Modifying existing tools in `tools/` (create wrappers, don't modify originals)
|
||||
- Skipping the decision log
|
||||
- Writing scripts that have side effects beyond their output file (no sending emails, no pushing to remote, no deleting files outside project workspace)
|
||||
|
||||
## Decision Log Entry Format
|
||||
|
||||
Every extension must be logged:
|
||||
|
||||
```json
|
||||
{
|
||||
"decision_id": "ext-001",
|
||||
"stage": "<current stage>",
|
||||
"category": "capability_extension",
|
||||
"subject": "Created custom <script|playbook|skill|tool> for <purpose>",
|
||||
"options_considered": [
|
||||
{"option_id": "existing-tool", "label": "<closest existing tool>", "rejected_because": "<why it doesn't work>"},
|
||||
{"option_id": "extension", "label": "<what was created>", "reason": "<why this approach>"}
|
||||
],
|
||||
"selected": "extension",
|
||||
"reason": "<concise justification>",
|
||||
"user_visible": true,
|
||||
"confidence": 0.8
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,64 @@
|
||||
# Creative Intake
|
||||
|
||||
Before the research stage, gather user intent through targeted questions.
|
||||
Do NOT start production on a vague brief.
|
||||
|
||||
## Required Questions (ask conversationally, not as a survey)
|
||||
|
||||
1. **Purpose**: What is this video FOR? (educate, sell, inspire, document, entertain)
|
||||
2. **Audience**: Who will watch it? (age, expertise, context — "my team" vs "YouTube public")
|
||||
3. **Platform**: Where will it live? (YouTube, internal Slack, social media, presentation, website)
|
||||
4. **Tone**: What should it FEEL like? (serious, playful, cinematic, raw, warm, provocative)
|
||||
5. **References**: Any videos you admire or want this to feel like?
|
||||
6. **Outcome**: What should the viewer DO or FEEL after watching?
|
||||
7. **Constraints**: Budget ceiling? Timeline? Must-include content?
|
||||
|
||||
## How to Ask
|
||||
|
||||
Don't dump all 7 questions at once. Start with purpose and audience,
|
||||
then let the conversation flow. Fill in gaps naturally.
|
||||
|
||||
If the user gives a detailed brief, skip questions they've already answered.
|
||||
|
||||
Identify what the user has already told you. If they said "I want a
|
||||
cinematic brand film for Instagram," you already have purpose (inspire/sell),
|
||||
platform (Instagram), and tone (cinematic). Ask what's missing.
|
||||
|
||||
## Handling Vague Briefs
|
||||
|
||||
When the user says something like "make me a video about X":
|
||||
|
||||
1. Acknowledge the topic — show you understood.
|
||||
2. Ask the single most important missing question first (usually purpose or audience).
|
||||
3. Based on their answer, ask the next most important gap.
|
||||
4. Stop asking when you have enough to start research. You don't need perfect answers — research will fill in details.
|
||||
|
||||
## Handling Detailed Briefs
|
||||
|
||||
When the user provides a multi-paragraph brief or a document:
|
||||
|
||||
1. Summarize what you understood (1-2 sentences).
|
||||
2. Call out any gaps: "I have a clear picture of the audience and tone, but I'd love to know — is there a specific outcome you're hoping for?"
|
||||
3. Confirm the platform and constraints if not stated.
|
||||
|
||||
## Output
|
||||
|
||||
Produce an `intake_brief` (informal, not schema-validated) that the
|
||||
research stage uses as its starting context. Include:
|
||||
|
||||
- Direct quotes from the user where their language reveals intent
|
||||
- Explicit answers to each of the 7 questions (mark any that were inferred vs stated)
|
||||
- Any reference videos/images the user mentioned
|
||||
- Constraints that must be honored (budget, timeline, must-include)
|
||||
|
||||
The intake_brief is passed as context to the research-director, not as a
|
||||
formal artifact. It exists to prevent the research stage from inventing
|
||||
intent that the user never expressed.
|
||||
|
||||
## What NOT To Do
|
||||
|
||||
- Do not present a numbered survey. This is a conversation, not a form.
|
||||
- Do not ask questions the user already answered in their initial message.
|
||||
- Do not delay production unnecessarily — if the brief is clear, move on.
|
||||
- Do not invent answers for questions the user didn't address. Mark them as "not specified" and let the research stage handle ambiguity.
|
||||
- Do not assume the user wants an explainer. Many users want cinematic, animation, or source-led work. Listen for signals.
|
||||
@@ -81,9 +81,9 @@ Based on the user's tier, present **3 ready-to-use prompts** they can copy right
|
||||
>
|
||||
> This will research the topic, write a script, find stock visuals, generate narration with Piper, and compose an animated video with transitions and captions — all free.
|
||||
|
||||
> **Also try:** "Create a 60-second data-driven video about coffee consumption around the world"
|
||||
> **Also try:** "I have a screen recording of a dashboard workflow — make it a polished product demo with captions and a voiceover" *(Screen Demo pipeline)*
|
||||
|
||||
> **Or:** "Make a short explainer about how the internet works, with narration and animated captions"
|
||||
> **Or:** "Turn this interview recording into 3 short clips for TikTok and YouTube Shorts" *(Clip Factory pipeline)*
|
||||
|
||||
**Starter-tier prompts (image gen available):**
|
||||
|
||||
@@ -91,19 +91,19 @@ Based on the user's tier, present **3 ready-to-use prompts** they can copy right
|
||||
>
|
||||
> I'll use FLUX to generate custom images for each scene — much more visually striking than stock.
|
||||
|
||||
> **Also try:** "Make a product launch teaser for a fictional smart water bottle called AquaPulse"
|
||||
> **Also try:** "Make a short documentary-style video about urban beekeeping — keep it grounded and textural, not flashy" *(Hybrid pipeline — source + generated support)*
|
||||
|
||||
> **Or:** "Build a 90-second explainer about the psychology of color in marketing"
|
||||
> **Or:** "Create a classroom-ready video teaching photosynthesis to 8th graders — simple, clear, and engaging" *(Explainer pipeline — teacher mode)*
|
||||
|
||||
**Full-tier prompts (video gen available):**
|
||||
|
||||
> **Try this:** "Create a cinematic 30-second trailer for a sci-fi concept: humanity receives a warning from 1000 years in the future"
|
||||
>
|
||||
> I'll generate actual motion video clips, compose a soundtrack, and deliver a finished cinematic trailer.
|
||||
> I'll generate actual motion video clips, compose a soundtrack, and deliver a finished cinematic trailer. *(Cinematic pipeline)*
|
||||
|
||||
> **Also try:** "Make a 60-second avatar spokesperson video announcing a company rebrand"
|
||||
> **Also try:** "Make a 60-second avatar spokesperson video announcing a company rebrand" *(Avatar Spokesperson pipeline)*
|
||||
|
||||
> **Or:** "Create a 90-second animated explainer about quantum computing for middle school students, with a fun narrator voice and custom soundtrack"
|
||||
> **Or:** "I recorded a founder update on my webcam — make it feel polished, confident, and premium without looking fake" *(Talking Head pipeline)*
|
||||
|
||||
**Rules for prompt suggestions:**
|
||||
- Present exactly 3 prompts.
|
||||
|
||||
+136
-3
@@ -107,10 +107,143 @@ Structure your review as:
|
||||
|
||||
| Stage | What matters most |
|
||||
|-------|-----------------|
|
||||
| research | Source diversity, claim verifiability, visual reference quality |
|
||||
| proposal | Delivery promise clarity, renderer family selection, music/voice plan, decision log started |
|
||||
| idea | Hook uniqueness, research depth, angle diversity |
|
||||
| script | Timing accuracy, narrative arc, enhancement cue density |
|
||||
| scene_plan | Full coverage, visual variety, asset feasibility |
|
||||
| scene_plan | Full coverage, visual variety, asset feasibility, slideshow risk score |
|
||||
| assets | File existence, style consistency, budget adherence |
|
||||
| edit | Timeline coverage, audio sync, subtitle presence |
|
||||
| compose | Playability, duration accuracy, audio quality |
|
||||
| edit | Timeline coverage, audio sync, subtitle presence, delivery promise compliance |
|
||||
| compose | Playability, duration accuracy, audio quality, pre-compose validation pass |
|
||||
| publish | SEO quality, metadata completeness, export packaging |
|
||||
|
||||
## Slideshow Risk Review
|
||||
|
||||
Run at **scene_plan** and **edit** stages. Use `lib/slideshow_risk.py` to compute the score.
|
||||
|
||||
### At scene_plan stage:
|
||||
1. Compute `score_slideshow_risk(scenes, renderer_family=renderer_family)`
|
||||
2. If verdict is **"fail"** (average ≥ 4.0): **CRITICAL** — scene plan must be revised before proceeding
|
||||
3. If verdict is **"revise"** (average ≥ 3.0): **SUGGESTION** — flag specific dimensions scoring ≥ 3.5
|
||||
4. If verdict is **"strong"** or **"acceptable"**: note in review summary, no finding needed
|
||||
|
||||
### At edit stage:
|
||||
1. Recompute with full edit_decisions: `score_slideshow_risk(scenes, edit_decisions, renderer_family)`
|
||||
2. Same thresholds apply — if the edit stage made things worse (higher score than scene_plan), flag it
|
||||
|
||||
### What to flag per dimension:
|
||||
| Dimension | What to say when score ≥ 3.0 |
|
||||
|-----------|------------------------------|
|
||||
| repetition | "X scenes use the same layout/shot size — vary the visual grammar" |
|
||||
| decorative_visuals | "X scenes have no stated purpose (no information_role or shot_intent)" |
|
||||
| weak_motion | "Camera movement exists but lacks narrative justification" |
|
||||
| weak_shot_intent | "X scenes are missing shot_intent — why does this frame exist?" |
|
||||
| typography_overreliance | "X% of scenes are text/stat cards — video feels like animated slides" |
|
||||
| unsupported_cinematic_claims | "Claiming cinematic but missing hero moments / lighting / movement" |
|
||||
|
||||
## Decision Log Review
|
||||
|
||||
Run at **every stage** after proposal. The decision log (`schemas/artifacts/decision_log.schema.json`) is a cumulative audit trail.
|
||||
|
||||
### Checks:
|
||||
1. **Existence**: Does the checkpoint reference a `decision_log_ref`? If not after proposal stage, flag as **SUGGESTION**.
|
||||
2. **Coverage**: Does every major choice have an entry? Key decisions that MUST be logged:
|
||||
- Provider selection (which image/video/audio tool and why)
|
||||
- Style/playbook selection
|
||||
- Music track selection
|
||||
- Voice selection
|
||||
- Renderer family selection
|
||||
- Any fallback or downgrade (e.g., motion → still)
|
||||
3. **Quality**: Each decision should have:
|
||||
- At least 2 `options_considered` (not just the one picked)
|
||||
- A `reason` that isn't boilerplate ("best option" is not a reason)
|
||||
- Correct `confidence` (0.0–1.0) — flag if everything is 1.0 (unrealistic)
|
||||
4. **User visibility**: Decisions marked `user_visible: true` should be ones the user would actually care about (not internal routing)
|
||||
|
||||
### Severity:
|
||||
- Missing decision log after proposal: **SUGGESTION** (first time), **CRITICAL** (if still missing at edit stage)
|
||||
- Decision with only 1 option considered: **SUGGESTION** — "Log rejected alternatives for auditability"
|
||||
- All decisions at confidence 1.0: **SUGGESTION** — "Unrealistic confidence — at least provider selection involves tradeoffs"
|
||||
|
||||
## Creative Differentiation Review
|
||||
|
||||
Run at **scene_plan** and **edit** stages. Prevents the "every video looks the same" failure mode.
|
||||
|
||||
### Checks:
|
||||
1. **Variation check** (scene_plan only): Use `lib/variation_checker.py` → `check_scene_variation(scenes)`.
|
||||
- If verdict is "poor" (score ≤ 2): **CRITICAL** — "Scene plan lacks variety: [list violations]"
|
||||
- If verdict is "fair" (score ≤ 3): **SUGGESTION** — note specific suggestions from the checker
|
||||
|
||||
2. **Playbook alignment**: Is the active playbook appropriate for this content?
|
||||
- Cinematic trailer using "clean-professional" theme → flag mismatch
|
||||
- Educational explainer using "anime-ghibli" theme without user request → flag
|
||||
|
||||
3. **Shot language completeness** (scene_plan):
|
||||
- Every scene should have at least `shot_size` and `shot_intent`
|
||||
- Hero moments should have full shot_language (all 6 fields)
|
||||
- Flag scenes with empty shot_language as **SUGGESTION**
|
||||
|
||||
4. **Renderer family match** (edit stage):
|
||||
- Does `renderer_family` in edit_decisions match what was set at proposal?
|
||||
- If changed without documented reason in decision log → **CRITICAL**
|
||||
|
||||
## Delivery Promise Review
|
||||
|
||||
Run at **edit** and **compose** stages. Uses `lib/delivery_promise.py`.
|
||||
|
||||
### At edit stage:
|
||||
1. Extract delivery promise from proposal packet or edit_decisions metadata
|
||||
2. Run `promise.validate_cuts(cuts)` against the resolved cut list
|
||||
3. If `valid` is False: **CRITICAL** — "Delivery promise violation: [violations]"
|
||||
4. Check `motion_ratio`: if a motion-led promise has < 50% motion cuts, flag even if technically valid
|
||||
|
||||
### At compose stage:
|
||||
1. The `_pre_compose_validation()` in video_compose.py enforces this automatically
|
||||
2. Review should verify the validation was not bypassed (check render report for warnings)
|
||||
3. If render succeeded despite low motion ratio on a motion-led promise, flag as **SUGGESTION**
|
||||
|
||||
## Source Understanding Review
|
||||
|
||||
Run at **research** and **proposal** stages when user-supplied media files exist.
|
||||
|
||||
### Checks:
|
||||
1. **Existence**: If user-supplied files were provided to the project, does a `source_media_review` artifact exist?
|
||||
- If user media exists but no `source_media_review`: **CRITICAL** — "User supplied media but the agent did not inspect it before planning. Run `lib/source_media_review.review_source_media()` before proceeding."
|
||||
2. **Actual inspection**: Does every file entry have `reviewed: true` and a non-empty `technical_probe`?
|
||||
- If `reviewed` is missing or `technical_probe` is empty: **CRITICAL** — "The source_media_review claims review but contains no probe data. The file was not actually inspected."
|
||||
3. **Planning reflection**: Do the `planning_implications` appear in the proposal's production plan?
|
||||
- If quality risks were identified (e.g. low resolution, mono audio) but the proposal doesn't mention them: **SUGGESTION** — "Source media has quality risks that the proposal does not address."
|
||||
4. **Content accuracy**: Does the plan rely on content that the source media does not actually contain?
|
||||
- E.g. plan assumes interview dialogue but transcript_summary shows no speech: **CRITICAL** — "Plan assumes dialogue but source media contains no speech."
|
||||
5. **No hallucinated content**: The agent must not infer unsupported content from filenames alone. If `content_summary` says "interview footage" but the probe only shows 3s of silent video, flag as **CRITICAL**.
|
||||
|
||||
### Severity:
|
||||
- Missing `source_media_review` when user files exist: **CRITICAL** at proposal stage
|
||||
- Unreviewed files (no probe): **CRITICAL**
|
||||
- Plan doesn't reflect quality risks: **SUGGESTION**
|
||||
- Plan assumes content not in source: **CRITICAL**
|
||||
|
||||
## Final Self-Review Review
|
||||
|
||||
Run at **compose** and **publish** stages. Ensures the agent reviewed the actual rendered output.
|
||||
|
||||
### At compose stage:
|
||||
1. **Existence**: Does a `final_review` artifact exist alongside the `render_report`?
|
||||
- If missing: **CRITICAL** — "Compose produced a render_report but no final_review. The agent must inspect the rendered output before presenting it."
|
||||
2. **Status check**: What is `final_review.status`?
|
||||
- `pass` → OK, proceed
|
||||
- `revise` → The agent should have fixed issues before presenting. If the pipeline continued anyway: **CRITICAL** — "Self-review found revise-worthy issues but the agent presented anyway."
|
||||
- `fail` → The pipeline MUST NOT proceed. If it did: **CRITICAL**
|
||||
3. **Check completeness**: All 5 required checks must have data:
|
||||
- `technical_probe` must show a valid container with plausible duration/resolution
|
||||
- `visual_spotcheck` must have `frames_sampled >= 4`
|
||||
- `audio_spotcheck` must report narration/music presence
|
||||
- `promise_preservation` must confirm `delivery_promise_honored`
|
||||
- `subtitle_check` must report presence/absence
|
||||
- Any check with missing data: **SUGGESTION** — "Self-review check [X] has incomplete data"
|
||||
4. **Promise preservation**: If `promise_preservation.silent_downgrade_detected` is true: **CRITICAL** — "Self-review detected silent downgrade from motion-led to still-led."
|
||||
|
||||
### At publish stage:
|
||||
1. Verify that `final_review` was passed through as a required artifact
|
||||
2. If `final_review.status` is not `pass`: **CRITICAL** — "Cannot publish with a non-passing self-review"
|
||||
3. If `final_review.issues_found` is non-empty and `recommended_action` is not `present_to_user`: **SUGGESTION** — "Self-review found issues; verify they were resolved before publishing"
|
||||
|
||||
Reference in New Issue
Block a user