Add Grok media providers and improve selector routing
This commit is contained in:
calesthio
@@ -143,8 +143,8 @@ Three selector tools abstract multi-provider capabilities:
|
||||
| Selector | Capability | Providers (priority order) |
|
||||
|----------|-----------|---------------------------|
|
||||
| `tts_selector` | Text-to-speech | ElevenLabs > Google TTS > OpenAI > Piper (offline) |
|
||||
| `image_selector` | Image generation | FLUX > Google Imagen > DALL-E > Recraft > LocalDiffusion > Pexels/Pixabay (stock) |
|
||||
| `video_selector` | Video generation | Kling > Runway > VEO > MiniMax > HeyGen > LTX (modal) > LTX (local) > CogVideo > Hunyuan > WAN > Pexels/Pixabay (stock) |
|
||||
| `image_selector` | Image generation | FLUX > Grok > Google Imagen > DALL-E > Recraft > LocalDiffusion > Pexels/Pixabay (stock) |
|
||||
| `video_selector` | Video generation | Grok > Kling > Runway > VEO > MiniMax > HeyGen > LTX (modal) > LTX (local) > CogVideo > Hunyuan > WAN > Pexels/Pixabay (stock) |
|
||||
|
||||
Selectors route based on: user preference > availability > fallback order. They adapt input schemas between providers transparently.
|
||||
|
||||
@@ -158,11 +158,11 @@ Selectors route based on: user preference > availability > fallback order. They
|
||||
|
||||
**Enhancement (5):** upscale (Real-ESRGAN), bg_remove (rembg/U2Net), face_enhance, face_restore (CodeFormer/GFPGAN), color_grade (FFmpeg LUTs)
|
||||
|
||||
**Graphics (12):** flux_image, google_imagen, openai_image, recraft_image, local_diffusion, pexels_image, pixabay_image, image_selector, code_snippet, diagram_gen, math_animate (ManimCE), image_gen (deprecated)
|
||||
**Graphics (13):** flux_image, grok_image, google_imagen, openai_image, recraft_image, local_diffusion, pexels_image, pixabay_image, image_selector, code_snippet, diagram_gen, math_animate (ManimCE), image_gen (deprecated)
|
||||
|
||||
**Subtitle (1):** subtitle_gen
|
||||
|
||||
**Video (17):** heygen_video, veo_video, kling_video, runway_video, minimax_video, wan_video, hunyuan_video, cogvideo_video, ltx_video_local, ltx_video_modal, pexels_video, pixabay_video, video_selector, video_compose (FFmpeg), video_stitch, video_trimmer
|
||||
**Video (17):** grok_video, heygen_video, veo_video, kling_video, runway_video, minimax_video, wan_video, hunyuan_video, cogvideo_video, ltx_video_local, ltx_video_modal, pexels_video, pixabay_video, video_selector, video_compose (FFmpeg), video_stitch, video_trimmer
|
||||
|
||||
---
|
||||
|
||||
@@ -378,6 +378,7 @@ All config is validated via Pydantic models in `lib/config_model.py`.
|
||||
|----------|---------|---------|
|
||||
| `ELEVENLABS_API_KEY` | elevenlabs_tts, music_gen | TTS, music, sound effects |
|
||||
| `OPENAI_API_KEY` | openai_tts, openai_image | TTS fallback, DALL-E 3 |
|
||||
| `XAI_API_KEY` | grok_image, grok_video | Grok image editing/generation, Grok video generation |
|
||||
| `FAL_KEY` | flux_image, kling_video, veo_video, minimax_video, recraft_image | fal.ai hosted models (FLUX, Veo, Kling, MiniMax, Recraft) |
|
||||
| `HEYGEN_API_KEY` | heygen_video | Multi-provider video generation |
|
||||
| `PEXELS_API_KEY` | pexels_image, pexels_video | Stock media |
|
||||
|
||||
+39
-2
@@ -38,6 +38,7 @@ GOOGLE_API_KEY= # Google TTS + Google Imagen
|
||||
# VOICE + MUSIC
|
||||
ELEVENLABS_API_KEY= # TTS, music, sound effects (10K chars/month free)
|
||||
OPENAI_API_KEY= # OpenAI TTS + DALL-E 3 images
|
||||
XAI_API_KEY= # xAI Grok image generation/editing + Grok video generation
|
||||
|
||||
# MULTI-MODEL GATEWAY (one key, 6+ tools)
|
||||
FAL_KEY= # FLUX, Recraft, Kling, Veo, MiniMax video
|
||||
@@ -56,6 +57,41 @@ VIDEO_GEN_LOCAL_MODEL= # wan2.1-1.3b, wan2.1-14b, hunyuan-1.5, ltx2-local,
|
||||
|
||||
## Cloud Providers
|
||||
|
||||
### xAI — Grok Image + Video
|
||||
|
||||
> **Best if you want one provider for image edits and reference-conditioned short video.** Grok covers both image generation/editing and video generation under one key.
|
||||
|
||||
**Tools unlocked:** `grok_image`, `grok_video`
|
||||
**Env var:** `XAI_API_KEY`
|
||||
|
||||
#### Setup
|
||||
|
||||
1. Create an xAI developer account
|
||||
2. Generate an API key in the xAI developer console
|
||||
3. Add to `.env`: `XAI_API_KEY=xai-...`
|
||||
|
||||
#### What it's best for
|
||||
|
||||
- Image editing and style transfer
|
||||
- Multi-image composites into one generated frame
|
||||
- Short reference-image videos where a person, garment, or product must carry into motion
|
||||
|
||||
#### Pricing
|
||||
|
||||
Current xAI docs pricing for the Grok media models:
|
||||
|
||||
| Model | Price |
|
||||
|------|-------|
|
||||
| `grok-imagine-image` | $0.02 per generated image |
|
||||
| `grok-imagine-image` input images (edits/composites) | $0.002 per input image |
|
||||
| `grok-imagine-video` at 480p | $0.05/sec |
|
||||
| `grok-imagine-video` at 720p | $0.07/sec |
|
||||
| `grok-imagine-video` input images | $0.002 per input image |
|
||||
|
||||
OpenMontage now uses those published rates in the Grok tool estimators.
|
||||
|
||||
---
|
||||
|
||||
### fal.ai — Multi-Model Gateway
|
||||
|
||||
> **Best bang for buck.** One API key unlocks 6 tools across image and video generation.
|
||||
@@ -565,6 +601,7 @@ These tools require only FFmpeg or Python packages — no GPU, no API key.
|
||||
| **ElevenLabs** | `ELEVENLABS_API_KEY` | `elevenlabs_tts`, `music_gen` | Free tier + paid |
|
||||
| **fal.ai** | `FAL_KEY` | `flux_image`, `recraft_image`, `kling_video`, `veo_video`, `minimax_video` | Pay-as-you-go |
|
||||
| **OpenAI** | `OPENAI_API_KEY` | `openai_tts`, `openai_image` | Paid only |
|
||||
| **xAI** | `XAI_API_KEY` | `grok_image`, `grok_video` | Paid only |
|
||||
| **Runway** | `RUNWAY_API_KEY` | `runway_video` | Free trial + paid |
|
||||
| **HeyGen** | `HEYGEN_API_KEY` | `heygen_video` | Pay-as-you-go |
|
||||
| **Suno** | `SUNO_API_KEY` | `suno_music` | Pay-as-you-go |
|
||||
@@ -580,8 +617,8 @@ How many providers cover each capability:
|
||||
|
||||
| Capability | Cloud Providers | Local Providers | Free Options |
|
||||
|-----------|----------------|-----------------|--------------|
|
||||
| **Image Generation** | FLUX, Google Imagen, DALL-E 3, Recraft | Local Diffusion | Pexels, Pixabay (stock) |
|
||||
| **Video Generation** | Kling, Runway, Veo, MiniMax, HeyGen | WAN, Hunyuan, CogVideo, LTX | Pexels, Pixabay (stock) |
|
||||
| **Image Generation** | FLUX, Grok, Google Imagen, DALL-E 3, Recraft | Local Diffusion | Pexels, Pixabay (stock) |
|
||||
| **Video Generation** | Grok, Kling, Runway, Veo, MiniMax, HeyGen | WAN, Hunyuan, CogVideo, LTX | Pexels, Pixabay (stock) |
|
||||
| **Text-to-Speech** | ElevenLabs, Google TTS, OpenAI | Piper | Piper, Google free tier, ElevenLabs free tier |
|
||||
| **Music Generation** | ElevenLabs, Suno | — | ElevenLabs free tier |
|
||||
| **Post-Production** | — | FFmpeg (compose, stitch, trim, mix, enhance, grade) | All free |
|
||||
|
||||
Reference in New Issue
Block a user