lucataco / hotshot-xl

😊 Hotshot-XL is an AI text-to-GIF model trained to work alongside Stable Diffusion XL

Warm

Public
637.5K runs
L40S
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

go-pro video of a polar bear diving in the ocean, 8k, HD, dslr, nature footagego-pro video of a polar bear diving in the ocean, 8k, HD, dslr, nature footage

Input prompt

Default: "a camel smoking a cigarette, hd, high quality"

negative_prompt

string

Shift + Return to add a new line

Negative prompt

Default: "blurry"

width

integer

Width of the output

Default: 672

height

integer

Height of the output

Default: 384

scheduler

string

Select a Scheduler

Default: "EulerAncestralDiscreteScheduler"

steps

integer

(minimum: 1, maximum: 500)

Number of denoising steps

Default: 30

mp4

boolean

Save as mp4, False for GIF

Default: false

seed

integer

Random seed. Leave blank to randomize the seed

Run this model in Node.js with one line of code:

npx create-replicate --model=lucataco/hotshot-xl

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run lucataco/hotshot-xl using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "lucataco/hotshot-xl:78b3a6257e16e4b241245d65c8b2b81ea2e1ff7ed4c55306b511509ddbfd327a",
  {
    input: {
      mp4: false,
      seed: 6226,
      steps: 30,
      width: 672,
      height: 384,
      prompt: "go-pro video of a polar bear diving in the ocean, 8k, HD, dslr, nature footage",
      scheduler: "EulerAncestralDiscreteScheduler",
      negative_prompt: "blurry"
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run lucataco/hotshot-xl using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "lucataco/hotshot-xl:78b3a6257e16e4b241245d65c8b2b81ea2e1ff7ed4c55306b511509ddbfd327a",
    input={
        "mp4": False,
        "seed": 6226,
        "steps": 30,
        "width": 672,
        "height": 384,
        "prompt": "go-pro video of a polar bear diving in the ocean, 8k, HD, dslr, nature footage",
        "scheduler": "EulerAncestralDiscreteScheduler",
        "negative_prompt": "blurry"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run lucataco/hotshot-xl using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "lucataco/hotshot-xl:78b3a6257e16e4b241245d65c8b2b81ea2e1ff7ed4c55306b511509ddbfd327a",
    "input": {
      "mp4": false,
      "seed": 6226,
      "steps": 30,
      "width": 672,
      "height": 384,
      "prompt": "go-pro video of a polar bear diving in the ocean, 8k, HD, dslr, nature footage",
      "scheduler": "EulerAncestralDiscreteScheduler",
      "negative_prompt": "blurry"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2024-12-06T21:16:56.997655Z",
  "created_at": "2024-12-06T21:16:47.040000Z",
  "data_removed": false,
  "error": null,
  "id": "fp0vk4gj81rm80ckkqarajhzk0",
  "input": {
    "mp4": false,
    "seed": 6226,
    "steps": 30,
    "width": 672,
    "height": 384,
    "prompt": "go-pro video of a polar bear diving in the ocean, 8k, HD, dslr, nature footage",
    "scheduler": "EulerAncestralDiscreteScheduler",
    "negative_prompt": "blurry"
  },
  "logs": "Using seed: 6226\nWarning - setting num_images_per_prompt = 1 because video_length = 8\n  0%|          | 0/30 [00:00<?, ?it/s]\n  3%|▎         | 1/30 [00:00<00:08,  3.62it/s]\n  7%|▋         | 2/30 [00:00<00:07,  3.56it/s]\n 10%|█         | 3/30 [00:00<00:07,  3.54it/s]\n 13%|█▎        | 4/30 [00:01<00:07,  3.54it/s]\n 17%|█▋        | 5/30 [00:01<00:07,  3.53it/s]\n 20%|██        | 6/30 [00:01<00:06,  3.53it/s]\n 23%|██▎       | 7/30 [00:01<00:06,  3.52it/s]\n 27%|██▋       | 8/30 [00:02<00:06,  3.52it/s]\n 30%|███       | 9/30 [00:02<00:05,  3.51it/s]\n 33%|███▎      | 10/30 [00:02<00:05,  3.51it/s]\n 37%|███▋      | 11/30 [00:03<00:05,  3.51it/s]\n 40%|████      | 12/30 [00:03<00:05,  3.51it/s]\n 43%|████▎     | 13/30 [00:03<00:04,  3.50it/s]\n 47%|████▋     | 14/30 [00:03<00:04,  3.51it/s]\n 50%|█████     | 15/30 [00:04<00:04,  3.50it/s]\n 53%|█████▎    | 16/30 [00:04<00:03,  3.51it/s]\n 57%|█████▋    | 17/30 [00:04<00:03,  3.51it/s]\n 60%|██████    | 18/30 [00:05<00:03,  3.50it/s]\n 63%|██████▎   | 19/30 [00:05<00:03,  3.50it/s]\n 67%|██████▋   | 20/30 [00:05<00:02,  3.50it/s]\n 70%|███████   | 21/30 [00:05<00:02,  3.50it/s]\n 73%|███████▎  | 22/30 [00:06<00:02,  3.50it/s]\n 77%|███████▋  | 23/30 [00:06<00:02,  3.50it/s]\n 80%|████████  | 24/30 [00:06<00:01,  3.49it/s]\n 83%|████████▎ | 25/30 [00:07<00:01,  3.49it/s]\n 87%|████████▋ | 26/30 [00:07<00:01,  3.50it/s]\n 90%|█████████ | 27/30 [00:07<00:00,  3.49it/s]\n 93%|█████████▎| 28/30 [00:07<00:00,  3.49it/s]\n 97%|█████████▋| 29/30 [00:08<00:00,  3.49it/s]\n100%|██████████| 30/30 [00:08<00:00,  3.49it/s]\n100%|██████████| 30/30 [00:08<00:00,  3.51it/s]\n  0%|          | 0/8 [00:00<?, ?it/s]\n 75%|███████▌  | 6/8 [00:00<00:00, 44.13it/s]\n100%|██████████| 8/8 [00:00<00:00, 32.33it/s]",
  "metrics": {
    "predict_time": 9.949907471,
    "total_time": 9.957655
  },
  "output": "https://replicate.delivery/xezq/dPpsLJkQgUZ5LtiPM9kFmrnrAQPnI2ubuCfzixAkvZKk3I8JA/output.gif",
  "started_at": "2024-12-06T21:16:47.047748Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://stream.replicate.com/v1/files/bcwr-nphtq4xfv2idpug6zr2kgprs6ptagunhqglkb5mqjry6eh6ixq3a",
    "get": "https://api.replicate.com/v1/predictions/fp0vk4gj81rm80ckkqarajhzk0",
    "cancel": "https://api.replicate.com/v1/predictions/fp0vk4gj81rm80ckkqarajhzk0/cancel"
  },
  "version": "78b3a6257e16e4b241245d65c8b2b81ea2e1ff7ed4c55306b511509ddbfd327a"
}

Generated in

9.9 seconds

Tweak it ShareReport

Using seed: 6226
Warning - setting num_images_per_prompt = 1 because video_length = 8
  0%|          | 0/30 [00:00<?, ?it/s]
  3%|▎         | 1/30 [00:00<00:08,  3.62it/s]
  7%|▋         | 2/30 [00:00<00:07,  3.56it/s]
 10%|█         | 3/30 [00:00<00:07,  3.54it/s]
 13%|█▎        | 4/30 [00:01<00:07,  3.54it/s]
 17%|█▋        | 5/30 [00:01<00:07,  3.53it/s]
 20%|██        | 6/30 [00:01<00:06,  3.53it/s]
 23%|██▎       | 7/30 [00:01<00:06,  3.52it/s]
 27%|██▋       | 8/30 [00:02<00:06,  3.52it/s]
 30%|███       | 9/30 [00:02<00:05,  3.51it/s]
 33%|███▎      | 10/30 [00:02<00:05,  3.51it/s]
 37%|███▋      | 11/30 [00:03<00:05,  3.51it/s]
 40%|████      | 12/30 [00:03<00:05,  3.51it/s]
 43%|████▎     | 13/30 [00:03<00:04,  3.50it/s]
 47%|████▋     | 14/30 [00:03<00:04,  3.51it/s]
 50%|█████     | 15/30 [00:04<00:04,  3.50it/s]
 53%|█████▎    | 16/30 [00:04<00:03,  3.51it/s]
 57%|█████▋    | 17/30 [00:04<00:03,  3.51it/s]
 60%|██████    | 18/30 [00:05<00:03,  3.50it/s]
 63%|██████▎   | 19/30 [00:05<00:03,  3.50it/s]
 67%|██████▋   | 20/30 [00:05<00:02,  3.50it/s]
 70%|███████   | 21/30 [00:05<00:02,  3.50it/s]
 73%|███████▎  | 22/30 [00:06<00:02,  3.50it/s]
 77%|███████▋  | 23/30 [00:06<00:02,  3.50it/s]
 80%|████████  | 24/30 [00:06<00:01,  3.49it/s]
 83%|████████▎ | 25/30 [00:07<00:01,  3.49it/s]
 87%|████████▋ | 26/30 [00:07<00:01,  3.50it/s]
 90%|█████████ | 27/30 [00:07<00:00,  3.49it/s]
 93%|█████████▎| 28/30 [00:07<00:00,  3.49it/s]
 97%|█████████▋| 29/30 [00:08<00:00,  3.49it/s]
100%|██████████| 30/30 [00:08<00:00,  3.49it/s]
100%|██████████| 30/30 [00:08<00:00,  3.51it/s]
  0%|          | 0/8 [00:00<?, ?it/s]
 75%|███████▌  | 6/8 [00:00<00:00, 44.13it/s]
100%|██████████| 8/8 [00:00<00:00, 32.33it/s]

Examples

View more examples

Run time and cost

This model costs approximately $0.0098 to run on Replicate, or 102 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 11 seconds.

Readme

About

Attempt at implementing hotshot-XL

Changelog

10/11/23 - Shoutout to @sepal for his PR to speed up build & startup times. Also GIFs now returned by user request

10/18/23 - Added width and height as input options

10/20/23 - Fixed bug that didn’t allow width to be same as height

10/23/23 - Shoutout to @AbiruEkanayaka for the PR that added more Schedulers

Support

Give me a follow if you like my work! @lucataco93