Agent Skills

video

AI video generation: text-to-video, image-to-video, video-to-video, model selection. Use when generating a short video clip from a prompt or reference (e.g. 5s clip of a cat in rain, animate this photo, restyle this video).

Install

npx skills add https://github.com/starchild-ai-agent/official-skills --skill video
SKILL.md

video

Use this skill for all video-generation requests on Starchild.

Core principle: call the provided scripts. Do not re-implement proxy/billing/upload plumbing.


1. Text-to-video (most common)

โš ๏ธ Execution context โ€” read this first. The code blocks below are Python, not shell commands. Starchild's bash tool runs /bin/bash -c, which cannot parse exec(open(...)) โ€” pasting them directly into a bash command will fail with syntax error near unexpected token 'open'. Also, exec(open(...)) inside python3 -c fails with NameError: __file__ because the script uses __file__ for path resolution.

Use python3 - <<'EOF' with from exports import when calling via the bash tool:

python3 - <<'EOF'
import sys
sys.path.insert(0, "skills/video")
from generate_video import generate_video
result = generate_video(
    prompt="A cinematic drone shot over snowy mountains at sunrise",
    model="balanced",
    duration=5,
)
print(result)
EOF

The heredoc (<<'EOF') preserves all quotes and newlines โ€” no escaping needed. Note: video skill has no exports.py โ€” import directly from generate_video.

exec(open('skills/video/generate_video.py').read())
result = generate_video(
    prompt="A cinematic drone shot over snowy mountains at sunrise",
    model="balanced",   # "budget" | "balanced" | "premium"
    duration=5,
)
# result -> {"success": True, "cost": 0.70, "video_url": "...", "local_path": "output/videos/..."}

generate_video automatically: submits โ†’ polls โ†’ fetches result โ†’ downloads mp4 to output/videos/.

Delivering the result to the user โ€” IMPORTANT

Never hand the user the raw video_url (e.g. https://*.fal.media/.../*.mp4). fal serves these files with Content-Security-Policy: sandbox; default-src 'none', which means:

  • Opening the link in a browser shows a blank page (no inline player triggered).
  • Embedding via <video> / <iframe> is blocked by CSP.
  • There is no Content-Disposition: attachment header, so the browser does not auto-download either.
  • URL-side tweaks (query params, ?download=1, etc.) cannot fix this โ€” only a server-side header change would, and we don't control fal's CDN.

The only reliable user-facing delivery path is the already-downloaded local file:

  1. Use result["local_path"] (e.g. output/videos/xxx.mp4) โ€” generate_video always downloads on success.
  2. Tell the user the file is saved to output/videos/<filename> and is viewable in the workspace file panel / file browser.
  3. On Web channel, also embed it inline so the user can preview it in chat:
    ![video](output/videos/<filename>.mp4)
    
    (or link as [video](output/videos/<filename>.mp4) โ€” the workspace serves these directly with the right headers).
  4. On Telegram / WeChat: send the file via send_to_telegram(file_path="output/videos/...", message_type="video") or send_to_wechat(file_path="output/videos/...", message_type="video").

If the download somehow failed (local_path missing) โ€” re-fetch with:

curl -L -o output/videos/<filename>.mp4 "<video_url>"

Then deliver the local path. Still do not give the user the raw fal URL as the primary deliverable.


2. Image-to-video / video-to-video (reference assets)

fal.ai needs the reference asset as a public https URL. fal storage upload requires a Serverless permission your key currently does not have. The reliable path is to expose the asset via a published Starchild preview.

Standard procedure

  1. Drop or copy the asset into output/fal_assets/ using publish_asset.py.
  2. Make sure a preview named fal-assets is running and published (one-time setup, see ยง3).
  3. Build the public URL as <preview_base>/<filename>.
  4. Call generate_video(... image_url=public_url).
# Step 1: publish a local image into the asset folder
exec(open('skills/video/publish_asset.py').read())
asset = publish_local('/path/to/your/photo.jpg')
# or: publish_from_url('https://example.com/photo.jpg')

filename = asset['filename']

# Step 2: combine with the preview's public base URL (see ยง3)
public_url = f"https://community.iamstarchild.com/<user_slug>-fal-assets/{filename}"

# Step 3: image-to-video
exec(open('skills/video/generate_video.py').read())
result = generate_video(
    prompt="gentle cinematic camera push-in",
    model="balanced",
    duration=5,
    image_url=public_url,
)

generate_video auto-rewrites the model path from */text-to-video to */image-to-video whenever image_url is provided. The same approach works for video-to-video models โ€” pass an mp4 URL instead.

Asset constraints (enforced by publish_asset.py)

  • Image: .jpg .jpeg .png .webp .gif .bmp, max 10 MB
  • Video: .mp4 .mov .webm .mkv .m4v, max 100 MB
  • Anything outside these is rejected before publish

3. One-time fal-assets public preview setup

Run this once per workspace. The preview keeps running across sessions.

# 3.1 ensure the asset folder exists with a placeholder index
import os, pathlib
pathlib.Path('output/fal_assets').mkdir(parents=True, exist_ok=True)
if not os.path.exists('output/fal_assets/index.html'):
    open('output/fal_assets/index.html', 'w').write(
        '<!doctype html><html><body><h1>fal asset host</h1></body></html>'
    )

# 3.2 start the preview
preview(action='serve', dir='output/fal_assets', title='fal-assets')

# 3.3 publish to a public URL
preview(action='publish', preview_id='<id from step 3.2>', slug='fal-assets', title='fal-assets')
# โ†’ public base: https://community.iamstarchild.com/<user_slug>-fal-assets/

After publish, the public base URL is reusable for every future image-to-video / video-to-video task. Files dropped into output/fal_assets/ become reachable as <base>/<filename> immediately โ€” no re-publish needed.

Verify with:

curl -sI https://community.iamstarchild.com/<user_slug>-fal-assets/<filename>
# expect: HTTP/2 200, content-type: image/* or video/*

If preview(action='serve') returns No available ports in pool, ask the user which existing preview can be stopped to free a port โ€” never silently kill one.


4. Model selection

Tier Model Cost / 5s Notes
budget fal-ai/wan/v2.5/text-to-video $0.25 Fastest, cheapest; good for prompt iteration
balanced alibaba/happy-horse/text-to-video $0.70 Default; best lip-sync, most use cases
premium bytedance/seedance-2.0/fast/text-to-video $1.20 Best motion + camera direction
mini bytedance/seedance-2.0/mini/text-to-video $0.36 (480p) / $0.77 (720p) Cheapest Seedance; resolution-tiered, no 1080p. Duration must be a string ("5", not 5 or "5s") โ€” see gotcha below
premium-25 bytedance/seedance-2.5/text-to-video token-based Supports text-to-video, image-to-video, and reference-to-video. Requires resolution (480p/720p), aspect_ratio (six supported ratios), and integer duration from 4โ€“30 seconds. Estimate with estimate_cost(..., aspect_ratio=...).
โ€” xai/grok-imagine-video/v1.5/image-to-video $0.41 (480p) / $0.71 (720p) per 5s image-to-video ONLY (single required image_url, no image_urls); +$0.01 input-image surcharge included in estimate. โš ๏ธ resolution="1080p" is schema-valid upstream but has NO published price โ€” the proxy rejects it 400 fail-closed
โ€” fal-ai/kling-video/v3/turbo/standard/text-to-video $0.56 per 5s Kling v3 Turbo Standard, flat $0.112/s; .../turbo/pro/... = $0.14/s ($0.70/5s); .../v3/4k/... = $0.42/s ($2.10/5s). i2v variants exist for all
โ€” alibaba/happy-horse/v1.1/text-to-video $0.70 (720p) / $0.90 (1080p) per 5s v1.1 has its own 1080p tier $0.18/s (NOT the v1.0 2ร— rule); also /image-to-video, /reference-to-video
โ€” fal-ai/minimax_h3/text-to-video proxy pricing applies Supports text-to-video, image-to-video, and reference-to-video. For reference-to-video pass image_urls=[...]; the payload is translated to upstream reference_image_urls.

โš ๏ธ Happy Horse default resolution is 1080p upstream (v1.0 and v1.1): omitting resolution bills the 1080p tier (v1.1 5s = $0.90; v1.0 ref2v 5s = $1.40). Pass resolution="720p" explicitly for the cheaper rate. Invalid resolution values are rejected 400 by the proxy.

Reference-to-video: pass image_urls=[...] (list of 1โ€“9 public HTTP(S) URLs) โ€” NOT the single image_url param. generate_video() validates count and URL scheme. Happy Horse and Seedance 2.5 submit the image_urls field; MiniMax H3 (fal-ai/minimax_h3/reference-to-video) submits upstream's reference_image_urls field.

Seedance 2.5 example:

result = generate_video(
    prompt="A cinematic close-up of a paper crane unfolding",
    model="bytedance/seedance-2.5/text-to-video",
    duration=5,
    resolution="720p",
    aspect_ratio="16:9",
)

Use an integer duration from 4โ€“30 seconds. resolution must be 480p or 720p; aspect_ratio must be one of 21:9, 16:9, 4:3, 1:1, 3:4, 9:16. The proxy rejects auto values because they cannot be priced safely.

Override by passing the full model id to generate_video(model=...). Image-to-video variants are auto-derived by replacing text-to-video with image-to-video.

Pricing details and model registry live in generate_video.py::estimate_cost. For models not yet registered there, the legacy fallback is only a rough estimate and may differ from the proxy; do not use it for budgeting new endpoints.


5. Polling an existing request

exec(open('skills/video/poll_status.py').read())
result = poll_video("019ded6c-d871-7290-bbf1-ddc6993f8958")

Use this when an earlier generate_video call timed out or you only have a request_id.


6. Provided scripts

  • generate_video.py โ€” submit โ†’ poll โ†’ download. Handles text-to-video and image-to-video.
  • publish_asset.py โ€” copy local files (or download remote URLs) into output/fal_assets/ so they can be served by the fal-assets preview.
  • poll_status.py โ€” resume polling by request_id, downloads the result on completion.

7. Troubleshooting

Problem Fix
image_url must be a public HTTP(S) URL Use publish_asset.py + fal-assets preview, then pass the public URL
No available ports in pool (preview serve) Ask the user which preview to stop; do not auto-kill
downstream_service_error after COMPLETED Reference asset host failed mid-render โ€” re-encode/resize to 16:9, re-publish, retry
HTTP 402 insufficient_credits Top up balance; cost is pre-charged on submit
HTTP 403 endpoint_not_allowed sc-proxy only allows approved fal video endpoints; pick one from the model table
Generation FAILED upstream Shorten prompt, drop unusual tokens, retry once before changing model
HTTP 422 literal_error on duration (Seedance Mini) Mini requires duration as a string ("5", "10", "auto"), not an int and not "5s". generate_video() encodes this automatically when model contains seedance-2.0/mini โ€” only hit this if you hand-build the request body. Other Seedance variants accept int/"5s" as before.
Seedance 2.5 rejects auto or returns resolution_not_priceable / aspect_ratio_not_priceable Pass explicit resolution="480p" or "720p", an explicit supported aspect_ratio, and integer duration from 4โ€“30. Seedance 2.5 uses token-based pricing; call estimate_cost(model, duration, resolution, aspect_ratio) for a local estimate.
MiniMax H3 reference request returns a parameter error Use image_urls=[...] with the fal-ai/minimax_h3/reference-to-video model. generate_video() translates it to upstream reference_image_urls; do not hand-send image_urls to upstream.
Job stuck IN_PROGRESS >15 min Save request_id, resume later with poll_status.py
User reports the fal.media link "shows nothing" / "blank page" Expected โ€” fal serves with CSP: sandbox; default-src 'none'. Deliver the local file at result["local_path"] instead of the raw URL (see ยง1).

8. Infrastructure (reference)

  • Caller โ†’ sc-proxy โ†’ queue.fal.run (and api.fal.ai) โ†’ fal model providers
  • All requests must include Authorization: Key fake-falai-key-12345 (proxy injects the real FAL_KEY)
  • Pre-charge happens at submit. Poll/result calls are free.
  • Allowed endpoints: video text-to-video / image-to-video / video-to-video / edit-video for the registered models. Anything else returns 403 endpoint_not_allowed.
  • Final mp4 lives at https://*.fal.media/... โ€” public CDN, no auth needed for download.

9. Maintenance

  • Adding a new model โ†’ register price in generate_video.py::estimate_cost and in transparent-proxy/apis/falai.py::_VIDEO_PRICING.
  • Asset hosting via fal storage upload is intentionally not used in this skill: the production FAL_KEY lacks Serverless permission. Keep using the preview-based approach until that changes.

Related skills

video-editgenmedia-labs715KEdit existing video on RunComfy โ€” this skill is a smart router that matches the user's intent to the right edit model in the RunComfy catalog. Picks Wan 2.7 Edit-Video (general restyle / background swap / packaging swap, identity + motion preservation), Kling 2.6 Pro Motion Control (transfer precise motion from a reference video to a target character), or Lucy Edit Restyle (lightweight identity-stable restyle / outfit swap). Bundles each model's documented prompting patterns so the skill gets shai-video-generationgenmedia-labs714KGenerate AI videos on RunComfy via the `runcomfy` CLI โ€” a smart router across the full video-model catalog: HappyHorse 1.0 (Arena #1, native in-pass audio), Wan-AI Wan 2-7 (open weights, audio-driven lip-sync), ByteDance Seedance v2 / 1-5 / 1-0 (multi-modal cinematic), Kling 3.0 / 2-6, Google Veo 3-1, MiniMax Hailuo 2-3, ByteDance Dreamina 3-0. Covers text-to-video (t2v), image-to-video (i2v), and Veo's video-extend endpoint. The skill picks the right model for the user's intent (Arena-#1 qualitai-musicgenmedia-labs714KGenerate AI music on RunComfy via the `runcomfy` CLI โ€” a smart router across the music-model catalog. Routes to ElevenLabs AI Music Generation (premium 44.1 kHz stereo vocal tracks, 5 sโ€“5 min, $0.0083/s) and ACE Step / ACE Step 1.5 (StepFun-AI open-weights, tag-driven composition, multilingual lyrics, $0.0002โ€“0.0003/s, ~27ร— cheaper), plus ACE Step audio-inpaint (regenerate a time range inside an existing track) and ACE Step audio-outpaint (extend a track before or after). Picks the right model fimage-to-videogenmedia-labs713KAnimate any still image on RunComfy โ€” this skill is a smart router that matches the user's intent to the right i2v model in the RunComfy catalog. Picks HappyHorse 1.0 I2V (Arena #1, native audio, identity preservation) for general animations, Wan 2.7 with `audio_url` for custom-voiceover lip-sync, or Seedance 2.0 Pro for multi-modal animation from image + reference video + reference audio. Bundles each model's documented prompting patterns so the caller gets sharper output without burning iterat

Search skills and MCP servers

Fuzzy search across 23,137 skills and servers