Fix gaps #18-20: multi-gateway video skill, selector-first routing, self-review
- ai-video-gen skill: document fal.ai as primary gateway alongside HeyGen, add video_selector-first guidance - animation.yaml: add selector-first rule in assets stage, mandatory post-render self-review in compose stage - CinematicRenderer: add resolveAsset() for staticFile path resolution, fixing file:// URI errors in OffthreadVideo and Audio components
This commit is contained in:
@@ -1,23 +1,33 @@
|
||||
---
|
||||
name: ai-video-gen
|
||||
description: |
|
||||
Generate AI videos from text prompts using the HeyGen API. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance), (5) Working with HeyGen's /v1/workflows/executions endpoint for video generation.
|
||||
Generate AI videos from text prompts using multiple provider gateways. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance, MiniMax). Supports two gateways: HeyGen API and fal.ai API.
|
||||
allowed-tools: mcp__heygen__*
|
||||
metadata:
|
||||
openclaw:
|
||||
requires:
|
||||
env:
|
||||
env_any:
|
||||
- HEYGEN_API_KEY
|
||||
primaryEnv: HEYGEN_API_KEY
|
||||
- FAL_KEY
|
||||
primaryEnv: FAL_KEY
|
||||
---
|
||||
|
||||
# Video Generation (HeyGen API)
|
||||
# Video Generation (Multi-Gateway)
|
||||
|
||||
Generate AI videos from text prompts. Supports multiple providers (VEO 3.1, Kling, Sora, Runway, Seedance), configurable aspect ratios, and optional reference images for image-to-video generation.
|
||||
Generate AI videos from text prompts. Supports multiple providers via two API gateways:
|
||||
|
||||
| Gateway | Env Variable | Providers | Tool |
|
||||
|---------|-------------|-----------|------|
|
||||
| **fal.ai** | `FAL_KEY` | Kling v3/v2.1, MiniMax, VEO | `kling_video`, `minimax_video`, `veo_video` |
|
||||
| **HeyGen** | `HEYGEN_API_KEY` | VEO 3.1, Kling Pro, Sora v2, Runway Gen-4, Seedance | `heygen_video` |
|
||||
|
||||
**IMPORTANT:** Always use `video_selector` instead of calling provider tools directly. The selector handles availability checks, cost comparison, and automatic fallback.
|
||||
|
||||
## Authentication
|
||||
|
||||
All requests require the `X-Api-Key` header. Set the `HEYGEN_API_KEY` environment variable.
|
||||
**fal.ai (recommended primary):** Set `FAL_KEY` environment variable. Get a key at https://fal.ai/dashboard/keys
|
||||
|
||||
**HeyGen (secondary):** Set `HEYGEN_API_KEY` environment variable.
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v1/workflows/executions" \
|
||||
|
||||
@@ -1,23 +1,33 @@
|
||||
---
|
||||
name: ai-video-gen
|
||||
description: |
|
||||
Generate AI videos from text prompts using the HeyGen API. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance), (5) Working with HeyGen's /v1/workflows/executions endpoint for video generation.
|
||||
Generate AI videos from text prompts using multiple provider gateways. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance, MiniMax). Supports two gateways: HeyGen API and fal.ai API.
|
||||
allowed-tools: mcp__heygen__*
|
||||
metadata:
|
||||
openclaw:
|
||||
requires:
|
||||
env:
|
||||
env_any:
|
||||
- HEYGEN_API_KEY
|
||||
primaryEnv: HEYGEN_API_KEY
|
||||
- FAL_KEY
|
||||
primaryEnv: FAL_KEY
|
||||
---
|
||||
|
||||
# Video Generation (HeyGen API)
|
||||
# Video Generation (Multi-Gateway)
|
||||
|
||||
Generate AI videos from text prompts. Supports multiple providers (VEO 3.1, Kling, Sora, Runway, Seedance), configurable aspect ratios, and optional reference images for image-to-video generation.
|
||||
Generate AI videos from text prompts. Supports multiple providers via two API gateways:
|
||||
|
||||
| Gateway | Env Variable | Providers | Tool |
|
||||
|---------|-------------|-----------|------|
|
||||
| **fal.ai** | `FAL_KEY` | Kling v3/v2.1, MiniMax, VEO | `kling_video`, `minimax_video`, `veo_video` |
|
||||
| **HeyGen** | `HEYGEN_API_KEY` | VEO 3.1, Kling Pro, Sora v2, Runway Gen-4, Seedance | `heygen_video` |
|
||||
|
||||
**IMPORTANT:** Always use `video_selector` instead of calling provider tools directly. The selector handles availability checks, cost comparison, and automatic fallback.
|
||||
|
||||
## Authentication
|
||||
|
||||
All requests require the `X-Api-Key` header. Set the `HEYGEN_API_KEY` environment variable.
|
||||
**fal.ai (recommended primary):** Set `FAL_KEY` environment variable. Get a key at https://fal.ai/dashboard/keys
|
||||
|
||||
**HeyGen (secondary):** Set `HEYGEN_API_KEY` environment variable.
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.heygen.com/v1/workflows/executions" \
|
||||
|
||||
@@ -197,6 +197,7 @@ stages:
|
||||
- Total cost within approved budget from proposal_packet
|
||||
- "Layer 3 skills read for EVERY generation tool before writing prompts (check agent_skills field)"
|
||||
- "Clip duration maximized (prefer 10s over 5s) to reduce API calls and cost"
|
||||
- "Use selector tools (video_selector, tts_selector, image_selector) — never call provider tools directly"
|
||||
success_criteria:
|
||||
- Schema-valid asset_manifest artifact
|
||||
- All referenced asset files exist on disk
|
||||
@@ -249,9 +250,11 @@ stages:
|
||||
- Motion timing survives the final render
|
||||
- Audio sync matches the animatic intent
|
||||
- Duration within +/-5% of target
|
||||
- "Post-render self-review MANDATORY: extract keyframes, visually inspect each scene, flag weak frames before presenting to user"
|
||||
success_criteria:
|
||||
- Schema-valid render_report artifact
|
||||
- Output file exists and passes ffprobe validation
|
||||
- self_review_completed is true with per-scene quality notes
|
||||
|
||||
- name: publish
|
||||
skill: pipelines/animation/publish-director
|
||||
|
||||
@@ -8,9 +8,21 @@ import {
|
||||
Sequence,
|
||||
interpolate,
|
||||
spring,
|
||||
staticFile,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
} from "remotion";
|
||||
|
||||
function resolveAsset(src: string): string {
|
||||
if (src.startsWith("http://") || src.startsWith("https://") || src.startsWith("data:")) {
|
||||
return src;
|
||||
}
|
||||
const clean = src.replace(/^file:\/\/\/?/, "");
|
||||
if (clean.startsWith("/") || /^[A-Za-z]:[/\\]/.test(clean)) {
|
||||
return `file:///${clean.replace(/\\/g, "/")}`;
|
||||
}
|
||||
return staticFile(clean);
|
||||
}
|
||||
import { CinematicRendererProps, CinematicTone, CinematicVideoScene } from "./cinematic/types";
|
||||
|
||||
const FPS = 30;
|
||||
@@ -74,7 +86,7 @@ const SceneVideo: React.FC<{ scene: CinematicVideoScene }> = ({ scene }) => {
|
||||
<AbsoluteFill style={{ backgroundColor: "#020407", opacity }}>
|
||||
<OffthreadVideo
|
||||
muted
|
||||
src={scene.src}
|
||||
src={resolveAsset(scene.src)}
|
||||
trimBefore={trimBefore}
|
||||
trimAfter={trimAfter}
|
||||
style={{
|
||||
@@ -296,7 +308,7 @@ const Soundtrack: React.FC<{
|
||||
|
||||
return (
|
||||
<Audio
|
||||
src={src}
|
||||
src={resolveAsset(src)}
|
||||
trimBefore={trimBefore}
|
||||
trimAfter={trimAfter}
|
||||
volume={() => Math.min(fadeIn, fadeOut)}
|
||||
|
||||
Reference in New Issue
Block a user