Fix gaps #18-20: multi-gateway video skill, selector-first routing, self-review

- ai-video-gen skill: document fal.ai as primary gateway alongside HeyGen,
  add video_selector-first guidance
- animation.yaml: add selector-first rule in assets stage, mandatory
  post-render self-review in compose stage
- CinematicRenderer: add resolveAsset() for staticFile path resolution,
  fixing file:// URI errors in OffthreadVideo and Audio components
This commit is contained in:
calesthio
2026-04-04 13:53:19 -07:00
parent fc6ff2248a
commit 20dcc785e5
4 changed files with 49 additions and 14 deletions
+16 -6
View File
@@ -1,23 +1,33 @@
---
name: ai-video-gen
description: |
Generate AI videos from text prompts using the HeyGen API. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance), (5) Working with HeyGen's /v1/workflows/executions endpoint for video generation.
Generate AI videos from text prompts using multiple provider gateways. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance, MiniMax). Supports two gateways: HeyGen API and fal.ai API.
allowed-tools: mcp__heygen__*
metadata:
openclaw:
requires:
env:
env_any:
- HEYGEN_API_KEY
primaryEnv: HEYGEN_API_KEY
- FAL_KEY
primaryEnv: FAL_KEY
---
# Video Generation (HeyGen API)
# Video Generation (Multi-Gateway)
Generate AI videos from text prompts. Supports multiple providers (VEO 3.1, Kling, Sora, Runway, Seedance), configurable aspect ratios, and optional reference images for image-to-video generation.
Generate AI videos from text prompts. Supports multiple providers via two API gateways:
| Gateway | Env Variable | Providers | Tool |
|---------|-------------|-----------|------|
| **fal.ai** | `FAL_KEY` | Kling v3/v2.1, MiniMax, VEO | `kling_video`, `minimax_video`, `veo_video` |
| **HeyGen** | `HEYGEN_API_KEY` | VEO 3.1, Kling Pro, Sora v2, Runway Gen-4, Seedance | `heygen_video` |
**IMPORTANT:** Always use `video_selector` instead of calling provider tools directly. The selector handles availability checks, cost comparison, and automatic fallback.
## Authentication
All requests require the `X-Api-Key` header. Set the `HEYGEN_API_KEY` environment variable.
**fal.ai (recommended primary):** Set `FAL_KEY` environment variable. Get a key at https://fal.ai/dashboard/keys
**HeyGen (secondary):** Set `HEYGEN_API_KEY` environment variable.
```bash
curl -X POST "https://api.heygen.com/v1/workflows/executions" \
+16 -6
View File
@@ -1,23 +1,33 @@
---
name: ai-video-gen
description: |
Generate AI videos from text prompts using the HeyGen API. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance), (5) Working with HeyGen's /v1/workflows/executions endpoint for video generation.
Generate AI videos from text prompts using multiple provider gateways. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance, MiniMax). Supports two gateways: HeyGen API and fal.ai API.
allowed-tools: mcp__heygen__*
metadata:
openclaw:
requires:
env:
env_any:
- HEYGEN_API_KEY
primaryEnv: HEYGEN_API_KEY
- FAL_KEY
primaryEnv: FAL_KEY
---
# Video Generation (HeyGen API)
# Video Generation (Multi-Gateway)
Generate AI videos from text prompts. Supports multiple providers (VEO 3.1, Kling, Sora, Runway, Seedance), configurable aspect ratios, and optional reference images for image-to-video generation.
Generate AI videos from text prompts. Supports multiple providers via two API gateways:
| Gateway | Env Variable | Providers | Tool |
|---------|-------------|-----------|------|
| **fal.ai** | `FAL_KEY` | Kling v3/v2.1, MiniMax, VEO | `kling_video`, `minimax_video`, `veo_video` |
| **HeyGen** | `HEYGEN_API_KEY` | VEO 3.1, Kling Pro, Sora v2, Runway Gen-4, Seedance | `heygen_video` |
**IMPORTANT:** Always use `video_selector` instead of calling provider tools directly. The selector handles availability checks, cost comparison, and automatic fallback.
## Authentication
All requests require the `X-Api-Key` header. Set the `HEYGEN_API_KEY` environment variable.
**fal.ai (recommended primary):** Set `FAL_KEY` environment variable. Get a key at https://fal.ai/dashboard/keys
**HeyGen (secondary):** Set `HEYGEN_API_KEY` environment variable.
```bash
curl -X POST "https://api.heygen.com/v1/workflows/executions" \
+3
View File
@@ -197,6 +197,7 @@ stages:
- Total cost within approved budget from proposal_packet
- "Layer 3 skills read for EVERY generation tool before writing prompts (check agent_skills field)"
- "Clip duration maximized (prefer 10s over 5s) to reduce API calls and cost"
- "Use selector tools (video_selector, tts_selector, image_selector) — never call provider tools directly"
success_criteria:
- Schema-valid asset_manifest artifact
- All referenced asset files exist on disk
@@ -249,9 +250,11 @@ stages:
- Motion timing survives the final render
- Audio sync matches the animatic intent
- Duration within +/-5% of target
- "Post-render self-review MANDATORY: extract keyframes, visually inspect each scene, flag weak frames before presenting to user"
success_criteria:
- Schema-valid render_report artifact
- Output file exists and passes ffprobe validation
- self_review_completed is true with per-scene quality notes
- name: publish
skill: pipelines/animation/publish-director
+14 -2
View File
@@ -8,9 +8,21 @@ import {
Sequence,
interpolate,
spring,
staticFile,
useCurrentFrame,
useVideoConfig,
} from "remotion";
function resolveAsset(src: string): string {
if (src.startsWith("http://") || src.startsWith("https://") || src.startsWith("data:")) {
return src;
}
const clean = src.replace(/^file:\/\/\/?/, "");
if (clean.startsWith("/") || /^[A-Za-z]:[/\\]/.test(clean)) {
return `file:///${clean.replace(/\\/g, "/")}`;
}
return staticFile(clean);
}
import { CinematicRendererProps, CinematicTone, CinematicVideoScene } from "./cinematic/types";
const FPS = 30;
@@ -74,7 +86,7 @@ const SceneVideo: React.FC<{ scene: CinematicVideoScene }> = ({ scene }) => {
<AbsoluteFill style={{ backgroundColor: "#020407", opacity }}>
<OffthreadVideo
muted
src={scene.src}
src={resolveAsset(scene.src)}
trimBefore={trimBefore}
trimAfter={trimAfter}
style={{
@@ -296,7 +308,7 @@ const Soundtrack: React.FC<{
return (
<Audio
src={src}
src={resolveAsset(src)}
trimBefore={trimBefore}
trimAfter={trimAfter}
volume={() => Math.min(fadeIn, fadeOut)}