Add Grok media providers and improve selector routing
This commit is contained in:
calesthio
@@ -10,6 +10,7 @@
|
||||
| Tool | Provider | Cost | Speed | Best For |
|
||||
|------|----------|------|-------|----------|
|
||||
| `flux_image` | FLUX 2 Pro via fal.ai | ~$0.03-0.05 | ~5-10s | Photorealism, general purpose, workhorse |
|
||||
| `grok_image` | Grok Imagine Image (xAI) | $0.02/output + $0.002/input edit image | ~5-15s | Image edits, style transfer, multi-image compositing |
|
||||
| `openai_image` | GPT Image 1 (OpenAI) | ~$0.01-0.17 | ~5-15s | Complex instructions, text in images, multi-element |
|
||||
| `recraft_image` | Recraft V4 via fal.ai | ~$0.04-0.25 | ~5-10s | Logos, SVG vectors, brand assets, text rendering |
|
||||
| `local_diffusion` | Stable Diffusion (local) | Free | ~30s+ | Offline, privacy, free |
|
||||
@@ -35,6 +36,8 @@
|
||||
| **Real-world photo** (city, nature, people) | `pexels_image` | Real photos > AI for realism | `pixabay_image` → `flux_image` |
|
||||
| **Technical diagram** | `diagram_gen` | Structured, editable | `flux_image` with diagram prompt |
|
||||
| **Abstract/conceptual illustration** | `flux_image` | AI excels at custom concepts | `openai_image` |
|
||||
| **Style transfer / repaint of an existing image** | `grok_image` | Native edit flow, strong promptable transforms | `openai_image` |
|
||||
| **Multi-image merge / composite** | `grok_image` | Can combine multiple source images into one scene | `openai_image` |
|
||||
| **Logo or brand asset** | `recraft_image` | SVG support, text accuracy | `openai_image` |
|
||||
| **Image with text/labels** | `openai_image` | Best text rendering (GPT Image 1) | `recraft_image` |
|
||||
| **Complex multi-element composition** | `openai_image` | Best instruction following | `flux_image` |
|
||||
@@ -46,6 +49,7 @@
|
||||
## Cost-Quality Tradeoff
|
||||
|
||||
```
|
||||
|
||||
PRODUCTION PATH: Premium
|
||||
├── Hero images: flux_image ($0.05/img)
|
||||
├── Supporting visuals: flux_image ($0.03/img)
|
||||
@@ -69,6 +73,8 @@ PRODUCTION PATH: Offline
|
||||
└── Total: $0.00 (but slower, lower quality)
|
||||
```
|
||||
|
||||
Use `generation_mode="edit"` when the task starts from an existing image and should route only to edit-capable providers.
|
||||
|
||||
## Using the Image Selector
|
||||
|
||||
For most cases, use `image_selector` and let it route:
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
# Grok Prompting
|
||||
|
||||
Use this when the chosen provider is `grok_image` or `grok_video`.
|
||||
|
||||
## When Grok Is The Right Pick
|
||||
|
||||
- You need to edit an existing image instead of generating from scratch
|
||||
- You need to merge multiple source images into one output
|
||||
- You need a short video influenced by reference images without locking the first frame
|
||||
- You want one provider for both image and video generation with similar prompt language
|
||||
|
||||
## Grok Image
|
||||
|
||||
### Best Prompt Shape
|
||||
|
||||
```
|
||||
[subject] + [action or change] + [setting] + [one style anchor] + [lighting]
|
||||
```
|
||||
|
||||
### Edit Prompts
|
||||
|
||||
For image edits, describe the intended transformation directly:
|
||||
|
||||
- "Render this as a pencil sketch with detailed shading."
|
||||
- "Replace the plain t-shirt with a dark green bomber jacket."
|
||||
- "Combine these two people into the same sunny park scene."
|
||||
|
||||
Do not over-specify every unchanged detail unless preservation is critical.
|
||||
|
||||
### Multi-Image Composites
|
||||
|
||||
Tell Grok how to combine the inputs:
|
||||
|
||||
- who comes from which source
|
||||
- what should stay separate
|
||||
- where the final scene takes place
|
||||
|
||||
Example:
|
||||
|
||||
```
|
||||
Place the person from image 1 and the person from image 2 on the same subway platform at dusk,
|
||||
standing shoulder to shoulder, cinematic sodium-vapor lighting, realistic photography.
|
||||
```
|
||||
|
||||
## Grok Video
|
||||
|
||||
### Best Prompt Shape
|
||||
|
||||
```
|
||||
[shot] + [camera movement] + [subject] + [main motion beat] + [environment] + [lighting] + [tone]
|
||||
```
|
||||
|
||||
### Reference-Image Video
|
||||
|
||||
Grok supports prompts that refer to source images with placeholders like `<IMAGE_1>`.
|
||||
Use that when you need identity, wardrobe, or product consistency.
|
||||
|
||||
Example:
|
||||
|
||||
```
|
||||
Medium full shot, slow push-in. The model from <IMAGE_1> walks onto a clean white runway wearing
|
||||
the jacket from <IMAGE_2>. Soft studio lighting, premium fashion campaign, confident expression.
|
||||
```
|
||||
|
||||
### Image-to-Video vs Reference-to-Video
|
||||
|
||||
- Use image-to-video when the source image should act like the opening frame.
|
||||
- Use reference-to-video when the source images should influence the content but not freeze the composition.
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
- Treating Grok reference images like strict storyboards. They are influence inputs, not exact frame locks.
|
||||
- Writing multiple scene changes into one clip request.
|
||||
- Combining too many style labels with too little scene information.
|
||||
- Using vague edit prompts like "make it better" instead of naming the change.
|
||||
|
||||
## OpenMontage Guidance
|
||||
|
||||
- For image edits or compositing, prefer `grok_image` over the selector's default workhorse tools.
|
||||
- For reference-conditioned video, prefer `grok_video` when the brief depends on carrying people, clothing, or products from input images into motion.
|
||||
- If the deliverable is pure cinematic motion without reference constraints, compare Grok against Runway, Veo, and Kling before locking the provider.
|
||||
@@ -14,6 +14,7 @@ For model-specific tips, see the linked guides below.
|
||||
|-------|-------|-------------|
|
||||
| **Sora 2 / Sora 2 Pro** | [OpenAI Sora 2 Cookbook](https://developers.openai.com/cookbook/examples/sora/sora2_prompting_guide) | Richest structured template. Advanced fields: lenses, filtration, grade, diegetic sound, wardrobe, finishing. |
|
||||
| **VEO 3.1 / VEO 3** | [Vertex AI Prompt Guide](https://cloud.google.com/vertex-ai/generative-ai/docs/video/video-gen-prompt-guide) | Best vocabulary reference tables. 14-component prompt structure. |
|
||||
| **Grok Imagine Video** | `creative/prompting/grok-prompting.md` | Best when prompts need reference-image placeholders like `<IMAGE_1>` and identity/product carryover. |
|
||||
| **LTX-2** | [LTX Prompting Guide](https://docs.ltx.video/api-documentation/prompting-guide) | 6-element structure. Audio/voice prompting. Strong "what to avoid" section. |
|
||||
| **HunyuanVideo 1.5** | [Tencent Prompt Handbook](https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/main/assets/HunyuanVideo_1_5_Prompt_Handbook_EN.md) | Formula: Subject + Motion + Scene + [Shot] + [Camera] + [Lighting] + [Style] + [Atmosphere]. |
|
||||
| **Runway Gen-4** | [Runway Prompting Guide](https://help.runwayml.com/hc/en-us/articles/39789879462419-Gen-4-Video-Prompting-Guide) | "Focus on motion, not appearance." One scene per clip. Simplicity wins. |
|
||||
@@ -175,6 +176,7 @@ Put dialogue in quotation marks: `Character says: "Hello world."`
|
||||
3. **If a shot misfires** — strip back. Freeze camera, simplify action, try again.
|
||||
4. **For consistency across clips** — repeat the same style/lighting/grade description.
|
||||
5. **Use seed values** — when you find a good result, save the seed for variations.
|
||||
6. **For Grok reference-image video** — assign each source image a clear role in the prompt using `<IMAGE_1>`, `<IMAGE_2>`, etc.
|
||||
|
||||
## Example: Generic Prompt Template
|
||||
|
||||
|
||||
Reference in New Issue
Block a user