xai/grok-imagine-video

xAI's video model. Generate short videos with synchronized sound from text, animate an image, or edit a video.

2.2M runs

Grok Imagine Video

Grok Imagine Video is xAI’s video model. It generates short videos with synchronized sound from a text prompt, animates a still image, or edits an existing video.

What it does

  • Text to video. Pass only a prompt and the model generates the scene, the motion, and the sound.
  • Image to video. Pass an image (jpg, jpeg, png, or webp) and a prompt describing the motion. The model animates the image and keeps its style.
  • Video editing. Pass a video (mp4, mov, or webm, up to 8.7 seconds) and describe the change. The output keeps the input’s duration, aspect ratio, and resolution.

The model generates background music, sound effects, and ambient audio together with the video, so the sound matches what happens on screen.

Inputs

  • prompt: what should happen in the video.
  • image: optional image to animate.
  • video: optional video to edit.
  • duration: length in seconds, from 1 to 15 (default 5). Ignored when you edit a video.
  • aspect_ratio: auto (the default), 16:9, 4:3, 1:1, 9:16, 3:4, 3:2, or 2:3.
  • resolution: 720p (the default) or 480p.

Content filtering

Runs that break the content policy fail. Your prompt and images are checked before generation starts, and xAI also filters finished videos. A blocked run fails with a content policy error. Try rewording the prompt or using a different image.

Prompt guide

The most reliable way to write prompts for Grok Imagine Video is to think like a director. Structure your prompt around these elements:

Subject, action, setting, camera, then lighting and mood.

The model expands your prompt internally, so you don’t need to describe every detail, but you do need to tell it what matters. Write natural sentences, not keyword lists.

Text-to-video prompts

When generating video from text, your prompt carries all the scene information.

Cinematic portrait:

Close-up shot of a young woman turning her head to look at the camera,
golden hour light casting warm shadows across her face, gentle wind
moving through her hair, slow camera push-in, nostalgic mood,
shallow depth of field, 50mm lens feel, photoreal natural light.

Action scene:

Tracking shot of a cloaked thief sprinting through a medieval castle
courtyard at dusk, dust kicking up, torches flickering on stone walls,
camera at shoulder height following the action, dramatic lighting,
crisp motion, cinematic pacing.

Nature scene:

8-second wide establishing shot of a mountain trail at sunrise,
golden light spilling over the ridge, wind moving grass subtly,
distant birds gliding across the sky, slow stabilized pan,
warm color grade, natural realism, gentle film grain.

Product shot:

Close-up of a luxury watch on a marble surface, camera slowly
orbiting 360 degrees, soft studio lighting with subtle reflections
on the metal case, clean minimal background, commercial ad style.

Image-to-video prompts

When animating a still image, the model already has the scene. Focus your prompt on motion, not description.

  • Don’t re-describe what’s in the image. The model sees it. Tell it what should change: the action, the camera movement, the atmosphere.
  • Don’t contradict the image. If there’s a man in the photo, don’t write “a woman dances.” Match your prompt to what’s actually there.
  • Be specific about motion. The model can’t infer the degree of motion from a still image. “Car passing” is vague, and “car racing past at high speed” gives the model something to work with.
  • Mention prominent features to anchor the subject: “the old man wearing glasses” or “the woman in the red jacket.”
  • Negative prompts don’t work. The model ignores them. Describe what you want instead.

Portrait animation:

The woman slowly turns her head to the right and smiles,
soft breeze moving her hair, gentle camera push-in.

Product animation:

The sneaker rotates smoothly on the pedestal, camera orbiting
at eye level, dramatic spotlight sweeping across the surface.

Camera movements

The model understands standard cinematic camera language:

  • Pan left or right: the camera turns horizontally to reveal a scene
  • Tilt up or down: the camera turns vertically for dramatic reveals
  • Zoom in or out: the lens zooms closer or farther
  • Dolly in or out: the camera moves forward or backward, which looks more cinematic than a zoom
  • Tracking or follow shot: the camera follows a moving subject
  • Orbit: the camera circles the subject
  • Aerial or drone: a high, bird’s-eye view
  • Handheld: natural shake for a documentary feel or urgency
  • Slow push-in: gradual forward movement to build tension
  • Static or tripod: no camera movement, for stable, formal compositions

You can describe multiple shots in sequence using “camera switch” or “cut to” as a transition cue:

Close-up of a kitten eating from a bowl. Camera switch.
Close-up of the cat food showing the texture and detail.

Audio prompts

The model generates audio natively alongside the video. Influence it by mentioning sound in your prompt:

  • Background music: “with upbeat electronic music” or “dramatic orchestral score”
  • Sound effects: “footsteps on gravel,” “wind howling,” “engine revving”
  • Ambient audio: “quiet café ambience,” “forest sounds with birdsong”
  • Short dialogue: “a quiet whisper: ‘We made it.’” or “urgent shout: ‘Stop him!’”

You can add an AUDIO: section at the end of your prompt for clarity:

8-second close-up of hands pulling apart a warm cinnamon roll,
steam rising, soft morning window light, slow camera push-in,
cozy kitchen mood.
AUDIO: soft room tone, faint kettle hiss, gentle pastry tear sound,
a quiet satisfied whisper: 'Perfect.'

Multiple actions

The model handles multi-beat sequences well. List actions in order:

The athlete crouches at the starting line, then explodes forward,
legs alternating rapidly, arms pumping powerfully. After crossing
the finish line, the crowd erupts in cheers. Follow-shot perspective.

You can also describe actions for multiple subjects:

The teacher in the background lectures angrily while the student
in the foreground slowly turns away. Camera rack-focuses from
student to teacher.

Intensity and adverbs

The model responds to intensity modifiers. Without them, it fills in its own interpretation, which may be more subtle than you want. Exaggerate slightly to match your intent:

  • “car passing” becomes “car racing past at high speed”
  • “man roaring” becomes “man roaring wildly”
  • “wings flapping” becomes “wings flapping with massive amplitude”

Style keywords

Embed these naturally in your prompts rather than stacking them as tags:

  • Photorealistic: “photoreal detail,” “natural lighting,” “lifelike textures,” “documentary feel”
  • Cinematic: “volumetric lighting,” “film grain,” “dynamic shadows,” “anamorphic lens”
  • Anime: “bold cel-shading,” “expressive linework,” “bright anime palette”
  • Painterly: “oil-painting texture,” “lush brushstrokes,” “impressionist glow”
  • Surreal: “dreamlike distortion,” “ethereal glow,” “otherworldly hues”
  • Retro: “1970s color film,” “VHS grain,” “faded Polaroid tones”
  • Commercial: “clean studio lighting,” “product photography,” “premium feel”

Video editing prompts

When editing an existing video, describe what to change. The model preserves everything you don’t mention.

  • Add: “Add a silver necklace to the woman.”
  • Remove: “Remove the bee from the scene.”
  • Swap: “Replace the bird with a small dragon.”
  • Restyle: “Restyle this as cyberpunk anime with neon-charged lines.”
  • Scene change: “Change the setting to autumn with falling leaves and warm golden light.”
  • Color change: “Change the woman’s outfit from blue to red.”

Input videos for editing can be up to 8.7 seconds long. The output matches the input’s duration, aspect ratio, and resolution, and duration is ignored.

Common mistakes

  • Re-describing the image in image-to-video mode. The model already sees it. Focus on motion.
  • Contradicting the source image. Match your prompt to what’s actually in the photo.
  • Tag stacking (“knight, castle, epic, 8K, cinematic”). Write a natural sentence with intent instead.
  • Too many simultaneous actions. Keep it to one subject, one action, one camera move.
  • No camera direction. Always specify a shot type and camera movement.
  • Vague motion (“the thing moves”). Use specific verbs with intensity modifiers.
  • Using negative prompts. They’re ignored. Describe what you want instead.

Tips for better results

  • Keep it simple. One main subject, one main action, and one camera move.
  • Iterate in small steps. Change one thing at a time: lighting, camera, action, or mood.
  • Use the two-step approach. Generate a great still image first, then animate it with image-to-video for more control.
  • Describe lighting and time of day. “Morning window light,” “golden hour,” “overcast,” “candlelight.”
  • Use specific verbs. “Surges,” “unfurls,” “shatters,” and “drifts” give better motion than “moves” or “goes.”
  • Shorter clips are more stable. 5 to 8 seconds works best. 15-second clips work but are more likely to have artifacts.
  • Match the aspect ratio to your platform. 16:9 for YouTube, 9:16 for Reels and TikTok, 1:1 for social media thumbnails.

You can try this model in the Replicate playground.

Model created
Model updated