fofr/wan2.1-with-lora – Run with an API on Replicate

fofr / wan2.1-with-lora

Run Wan2.1 14b or 1.3b with a lora

Cold

Public
15.2K runs
H100
GitHub
License

Run with an API

Playground API Examples README Versions

Input

prompt

*string

Shift + Return to add a new line

flat color 2d animation of a portrait of woman with white hair and green eyes, dynamic sceneflat color 2d animation of a portrait of woman with white hair and green eyes, dynamic scene

Text prompt for video generation

negative_prompt

string

Shift + Return to add a new line

Things you do not want to see in your video

Default: ""

image

file

Image to use as a starting frame for image to video generation.

aspect_ratio

string

The aspect ratio of the video. 16:9, 9:16, 1:1, etc.

Default: "16:9"

frames

integer

The number of frames to generate (1 to 5 seconds)

Default: 81

model

string

The model to use. 1.3b is faster, but 14b is better quality. A LORA either works with 1.3b or 14b, depending on the version it was trained on.

Default: "14b"

resolution

string

The resolution of the video. 720p is not supported for 1.3b.

Default: "480p"

lora_url

string

Shift + Return to add a new line

https://huggingface.co/motimalu/wan-flat-color-v2/resolve/main/wan_flat_color_v2.safetensorshttps://huggingface.co/motimalu/wan-flat-color-v2/resolve/main/wan_flat_color_v2.safetensors

Optional: The URL of a LORA to use

lora_strength_model

number

Strength of the LORA applied to the model. 0.0 is no LORA.

Default: 1

lora_strength_clip

number

Strength of the LORA applied to the CLIP model. 0.0 is no LORA.

Default: 1

fast_mode

string

Speed up generation with different levels of acceleration. Faster modes may degrade quality somewhat. The speedup is dependent on the content, so different videos may see different speedups.

Default: "Balanced"

sample_steps

integer

(minimum: 1, maximum: 60)

Number of generation steps. Fewer steps means faster generation, at the expensive of output quality. 30 steps is sufficient for most prompts

Default: 30

sample_guide_scale

number

(minimum: 0, maximum: 10)

Higher guide scale makes prompt adherence better, but can reduce variation

Default: 5

sample_shift

number

(minimum: 0, maximum: 10)

Sample shift factor

Default: 8

seed

integer

Set a seed for reproducibility. Random by default.

Run this model in Node.js with one line of code:

npx create-replicate --model=fofr/wan2.1-with-lora

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/wan2.1-with-lora using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/wan2.1-with-lora:c48fa8ec65b13143cb552ab98ea17984eab9d70e9fe99479117de40a2a7f9ed0",
  {
    input: {
      model: "14b",
      frames: 81,
      prompt: "flat color 2d animation of a portrait of woman with white hair and green eyes, dynamic scene",
      lora_url: "https://huggingface.co/motimalu/wan-flat-color-v2/resolve/main/wan_flat_color_v2.safetensors",
      fast_mode: "Balanced",
      resolution: "480p",
      aspect_ratio: "16:9",
      sample_shift: 8,
      sample_steps: 30,
      negative_prompt: "",
      lora_strength_clip: 1,
      sample_guide_scale: 5,
      lora_strength_model: 1
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run fofr/wan2.1-with-lora using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/wan2.1-with-lora:c48fa8ec65b13143cb552ab98ea17984eab9d70e9fe99479117de40a2a7f9ed0",
    input={
        "model": "14b",
        "frames": 81,
        "prompt": "flat color 2d animation of a portrait of woman with white hair and green eyes, dynamic scene",
        "lora_url": "https://huggingface.co/motimalu/wan-flat-color-v2/resolve/main/wan_flat_color_v2.safetensors",
        "fast_mode": "Balanced",
        "resolution": "480p",
        "aspect_ratio": "16:9",
        "sample_shift": 8,
        "sample_steps": 30,
        "negative_prompt": "",
        "lora_strength_clip": 1,
        "sample_guide_scale": 5,
        "lora_strength_model": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run fofr/wan2.1-with-lora using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "c48fa8ec65b13143cb552ab98ea17984eab9d70e9fe99479117de40a2a7f9ed0",
    "input": {
      "model": "14b",
      "frames": 81,
      "prompt": "flat color 2d animation of a portrait of woman with white hair and green eyes, dynamic scene",
      "lora_url": "https://huggingface.co/motimalu/wan-flat-color-v2/resolve/main/wan_flat_color_v2.safetensors",
      "fast_mode": "Balanced",
      "resolution": "480p",
      "aspect_ratio": "16:9",
      "sample_shift": 8,
      "sample_steps": 30,
      "negative_prompt": "",
      "lora_strength_clip": 1,
      "sample_guide_scale": 5,
      "lora_strength_model": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2025-03-10T14:52:27.712452Z",
  "created_at": "2025-03-10T14:45:04.239000Z",
  "data_removed": false,
  "error": null,
  "id": "swdwvq9tdxrme0cng2281bmcwc",
  "input": {
    "model": "14b",
    "frames": 81,
    "prompt": "flat color 2d animation of a portrait of woman with white hair and green eyes, dynamic scene",
    "lora_url": "https://huggingface.co/motimalu/wan-flat-color-v2/resolve/main/wan_flat_color_v2.safetensors",
    "aspect_ratio": "16:9",
    "sample_shift": 8,
    "sample_steps": 30,
    "lora_strength_clip": 1,
    "sample_guide_scale": 5,
    "lora_strength_model": 1
  },
  "logs": "Random seed set to: 2381169083\nChecking inputs\n====================================\nChecking weights\nConverting LoraLoader node 49 to LoraLoaderFromURL\n✅ umt5_xxl_fp16.safetensors exists in ComfyUI/models/text_encoders\n✅ wan_2.1_vae.safetensors exists in ComfyUI/models/vae\n✅ wan2.1_t2v_14B_bf16.safetensors exists in ComfyUI/models/diffusion_models\n====================================\nRunning workflow\n[ComfyUI] got prompt\nExecuting node 37, title: Load Diffusion Model, class type: UNETLoader\n[ComfyUI] model weight dtype torch.bfloat16, manual cast: None\n[ComfyUI] model_type FLOW\nExecuting node 49, title: Load LoRA, class type: LoraLoaderFromURL\n[ComfyUI] Requested to load WanTEModel\nExecuting node 6, title: CLIP Text Encode (Positive Prompt), class type: CLIPTextEncode\n[ComfyUI] loaded completely 140527.45920448302 10835.4765625 True\nExecuting node 48, title: ModelSamplingSD3, class type: ModelSamplingSD3\nExecuting node 3, title: KSampler, class type: KSampler\n[ComfyUI] Requested to load WAN21\n[ComfyUI] loaded completely 123801.93451991271 27251.406372070312 True\n[ComfyUI]\n[ComfyUI] 0%|          | 0/30 [00:00<?, ?it/s]\n[ComfyUI] 3%|▎         | 1/30 [00:06<03:00,  6.21s/it]\n[ComfyUI] 7%|▋         | 2/30 [00:15<03:37,  7.76s/it]\n[ComfyUI] 10%|█         | 3/30 [00:23<03:42,  8.26s/it]\n[ComfyUI] 13%|█▎        | 4/30 [00:32<03:40,  8.50s/it]\n[ComfyUI] 17%|█▋        | 5/30 [00:41<03:36,  8.64s/it]\n[ComfyUI] 20%|██        | 6/30 [00:50<03:29,  8.73s/it]\n[ComfyUI] 23%|██▎       | 7/30 [00:59<03:22,  8.78s/it]\n[ComfyUI] 27%|██▋       | 8/30 [01:08<03:14,  8.82s/it]\n[ComfyUI] 30%|███       | 9/30 [01:17<03:05,  8.85s/it]\n[ComfyUI] 33%|███▎      | 10/30 [01:26<02:57,  8.86s/it]\n[ComfyUI] 37%|███▋      | 11/30 [01:35<02:48,  8.88s/it]\n[ComfyUI] 40%|████      | 12/30 [01:43<02:40,  8.89s/it]\n[ComfyUI] 43%|████▎     | 13/30 [01:52<02:31,  8.90s/it]\n[ComfyUI] 47%|████▋     | 14/30 [02:01<02:22,  8.90s/it]\n[ComfyUI] 50%|█████     | 15/30 [02:10<02:13,  8.90s/it]\n[ComfyUI] 53%|█████▎    | 16/30 [02:19<02:04,  8.90s/it]\n[ComfyUI] 57%|█████▋    | 17/30 [02:28<01:55,  8.91s/it]\n[ComfyUI] 60%|██████    | 18/30 [02:37<01:46,  8.90s/it]\n[ComfyUI] 63%|██████▎   | 19/30 [02:46<01:37,  8.91s/it]\n[ComfyUI] 67%|██████▋   | 20/30 [02:55<01:29,  8.91s/it]\n[ComfyUI] 70%|███████   | 21/30 [03:04<01:20,  8.91s/it]\n[ComfyUI] 73%|███████▎  | 22/30 [03:13<01:11,  8.91s/it]\n[ComfyUI] 77%|███████▋  | 23/30 [03:21<01:02,  8.91s/it]\n[ComfyUI] 80%|████████  | 24/30 [03:30<00:53,  8.91s/it]\n[ComfyUI] 83%|████████▎ | 25/30 [03:39<00:44,  8.91s/it]\n[ComfyUI] 87%|████████▋ | 26/30 [03:48<00:35,  8.91s/it]\n[ComfyUI] 90%|█████████ | 27/30 [03:57<00:26,  8.91s/it]\n[ComfyUI] 93%|█████████▎| 28/30 [04:06<00:17,  8.91s/it]\n[ComfyUI] 97%|█████████▋| 29/30 [04:15<00:08,  8.91s/it]\n[ComfyUI] 100%|██████████| 30/30 [04:27<00:00,  9.71s/it]\nExecuting node 8, title: VAE Decode, class type: VAEDecode\nExecuting node 50, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine\n[ComfyUI] 100%|██████████| 30/30 [04:27<00:00,  8.90s/it]\n[ComfyUI] Prompt executed in 392.13 seconds\noutputs:  {'50': {'gifs': [{'filename': 'R8_Wan_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4', 'frame_rate': 16.0, 'workflow': 'R8_Wan_00001.png', 'fullpath': '/tmp/outputs/R8_Wan_00001.mp4'}]}}\n====================================\nR8_Wan_00001.png\nR8_Wan_00001.mp4",
  "metrics": {
    "predict_time": 392.32474699,
    "total_time": 443.473452
  },
  "output": [
    "https://replicate.delivery/xezq/EVxdGGJo5m6hDxI2Saz3pOeLwUOEegyijKf2ToWHaBQX1VuoA/R8_Wan_00001.mp4"
  ],
  "started_at": "2025-03-10T14:45:55.387705Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://stream.replicate.com/v1/files/bcwr-ljynnsvgwvxa3r5b53cd3ydxoajxlphdh3vfyefzpmfalhxancoq",
    "get": "https://api.replicate.com/v1/predictions/swdwvq9tdxrme0cng2281bmcwc",
    "cancel": "https://api.replicate.com/v1/predictions/swdwvq9tdxrme0cng2281bmcwc/cancel"
  },
  "version": "0615656dcf90c621b2fe05bddc3c87db422dda0aad7b0a681fa641d8749e9fd3"
}

Generated in

6 minutes 32 seconds

Tweak it ShareReport

Random seed set to: 2381169083
Checking inputs
====================================
Checking weights
Converting LoraLoader node 49 to LoraLoaderFromURL
✅ umt5_xxl_fp16.safetensors exists in ComfyUI/models/text_encoders
✅ wan_2.1_vae.safetensors exists in ComfyUI/models/vae
✅ wan2.1_t2v_14B_bf16.safetensors exists in ComfyUI/models/diffusion_models
====================================
Running workflow
[ComfyUI] got prompt
Executing node 37, title: Load Diffusion Model, class type: UNETLoader
[ComfyUI] model weight dtype torch.bfloat16, manual cast: None
[ComfyUI] model_type FLOW
Executing node 49, title: Load LoRA, class type: LoraLoaderFromURL
[ComfyUI] Requested to load WanTEModel
Executing node 6, title: CLIP Text Encode (Positive Prompt), class type: CLIPTextEncode
[ComfyUI] loaded completely 140527.45920448302 10835.4765625 True
Executing node 48, title: ModelSamplingSD3, class type: ModelSamplingSD3
Executing node 3, title: KSampler, class type: KSampler
[ComfyUI] Requested to load WAN21
[ComfyUI] loaded completely 123801.93451991271 27251.406372070312 True
[ComfyUI]
[ComfyUI] 0%|          | 0/30 [00:00<?, ?it/s]
[ComfyUI] 3%|▎         | 1/30 [00:06<03:00,  6.21s/it]
[ComfyUI] 7%|▋         | 2/30 [00:15<03:37,  7.76s/it]
[ComfyUI] 10%|█         | 3/30 [00:23<03:42,  8.26s/it]
[ComfyUI] 13%|█▎        | 4/30 [00:32<03:40,  8.50s/it]
[ComfyUI] 17%|█▋        | 5/30 [00:41<03:36,  8.64s/it]
[ComfyUI] 20%|██        | 6/30 [00:50<03:29,  8.73s/it]
[ComfyUI] 23%|██▎       | 7/30 [00:59<03:22,  8.78s/it]
[ComfyUI] 27%|██▋       | 8/30 [01:08<03:14,  8.82s/it]
[ComfyUI] 30%|███       | 9/30 [01:17<03:05,  8.85s/it]
[ComfyUI] 33%|███▎      | 10/30 [01:26<02:57,  8.86s/it]
[ComfyUI] 37%|███▋      | 11/30 [01:35<02:48,  8.88s/it]
[ComfyUI] 40%|████      | 12/30 [01:43<02:40,  8.89s/it]
[ComfyUI] 43%|████▎     | 13/30 [01:52<02:31,  8.90s/it]
[ComfyUI] 47%|████▋     | 14/30 [02:01<02:22,  8.90s/it]
[ComfyUI] 50%|█████     | 15/30 [02:10<02:13,  8.90s/it]
[ComfyUI] 53%|█████▎    | 16/30 [02:19<02:04,  8.90s/it]
[ComfyUI] 57%|█████▋    | 17/30 [02:28<01:55,  8.91s/it]
[ComfyUI] 60%|██████    | 18/30 [02:37<01:46,  8.90s/it]
[ComfyUI] 63%|██████▎   | 19/30 [02:46<01:37,  8.91s/it]
[ComfyUI] 67%|██████▋   | 20/30 [02:55<01:29,  8.91s/it]
[ComfyUI] 70%|███████   | 21/30 [03:04<01:20,  8.91s/it]
[ComfyUI] 73%|███████▎  | 22/30 [03:13<01:11,  8.91s/it]
[ComfyUI] 77%|███████▋  | 23/30 [03:21<01:02,  8.91s/it]
[ComfyUI] 80%|████████  | 24/30 [03:30<00:53,  8.91s/it]
[ComfyUI] 83%|████████▎ | 25/30 [03:39<00:44,  8.91s/it]
[ComfyUI] 87%|████████▋ | 26/30 [03:48<00:35,  8.91s/it]
[ComfyUI] 90%|█████████ | 27/30 [03:57<00:26,  8.91s/it]
[ComfyUI] 93%|█████████▎| 28/30 [04:06<00:17,  8.91s/it]
[ComfyUI] 97%|█████████▋| 29/30 [04:15<00:08,  8.91s/it]
[ComfyUI] 100%|██████████| 30/30 [04:27<00:00,  9.71s/it]
Executing node 8, title: VAE Decode, class type: VAEDecode
Executing node 50, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
[ComfyUI] 100%|██████████| 30/30 [04:27<00:00,  8.90s/it]
[ComfyUI] Prompt executed in 392.13 seconds
outputs:  {'50': {'gifs': [{'filename': 'R8_Wan_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4', 'frame_rate': 16.0, 'workflow': 'R8_Wan_00001.png', 'fullpath': '/tmp/outputs/R8_Wan_00001.mp4'}]}}
====================================
R8_Wan_00001.png
R8_Wan_00001.mp4

This output was created using a different version of the model, fofr/wan2.1-with-lora:0615656d.

Examples

View more examples

Run time and cost

This model costs approximately $5.14 to run on Replicate, or 0 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia H100 GPU hardware. Predictions typically complete within 57 minutes. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.