cjwbw / resshift

Efficient Diffusion Model for Image Super-resolution by Residual Shifting

Public
3.1K runs
T4
Paper
License

Iterate in playground

Run with an API

Playground API Examples README Versions

Input

Run this model in Node.js with one line of code:

npx create-replicate --model=cjwbw/resshift

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run cjwbw/resshift using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "cjwbw/resshift:3592f9923723132ea2da2209ca8762f030c52b06b9c2d8f709cd4fa97187d201",
  {
    input: {
      task: "realsrx4",
      image: "https://replicate.delivery/pbxt/JHFWRZMpgOu7jvS2Ag6pj7vXOdjJx2zYkKcqDpiVwcOp7gcx/comic1.png",
      scale: 4,
      chop_size: 512
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run cjwbw/resshift using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "cjwbw/resshift:3592f9923723132ea2da2209ca8762f030c52b06b9c2d8f709cd4fa97187d201",
    input={
        "task": "realsrx4",
        "image": "https://replicate.delivery/pbxt/JHFWRZMpgOu7jvS2Ag6pj7vXOdjJx2zYkKcqDpiVwcOp7gcx/comic1.png",
        "scale": 4,
        "chop_size": 512
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run cjwbw/resshift using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "cjwbw/resshift:3592f9923723132ea2da2209ca8762f030c52b06b9c2d8f709cd4fa97187d201",
    "input": {
      "task": "realsrx4",
      "image": "https://replicate.delivery/pbxt/JHFWRZMpgOu7jvS2Ag6pj7vXOdjJx2zYkKcqDpiVwcOp7gcx/comic1.png",
      "scale": 4,
      "chop_size": 512
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

output

Generated in

19.8 seconds

Tweak it Report

Examples

View more examples

Run time and cost

This model costs approximately $0.016 to run on Replicate, or 62 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 70 seconds. The predict time for this model varies significantly based on the inputs.

Readme

ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting

Diffusion-based image super-resolution (SR) methods are mainly limited by the low inference speed due to the requirements of hundreds or even thousands of sampling steps. Existing acceleration sampling techniques inevitably sacrifice performance to some extent, leading to over-blurry SR results. To address this issue, we propose a novel and efficient diffusion model for SR that significantly reduces the number of diffusion steps, thereby eliminating the need for post-acceleration during inference and its associated performance deterioration. Our method constructs a Markov chain that transfers between the high-resolution image and the low-resolution image by shifting the residual between them, substantially improving the transition efficiency. Additionally, an elaborate noise schedule is developed to flexibly control the shifting speed and the noise strength during the diffusion process. Extensive experiments demonstrate that the proposed method obtains superior or at least comparable performance to current state-of-the-art methods on both synthetic and real-world datasets, even only with 15 sampling steps.

License

This project is licensed under NTU S-Lab License 1.0. Redistribution and use should follow this license.

Acknowledgement

This project is based on Improved Diffusion Model, LDM, and BasicSR. We also adopt Real-ESRGAN to synthesize the training data for real-world super-resolution. Thanks for their awesome works.