✋ This model is not published yet.

You can claim this model if you're @lhoyer on GitHub. Contact us.

lhoyer / hrda

image semantic segmentation

Cold

Public
35 runs
T4
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

Run this model in Node.js with one line of code:

npx create-replicate --model=lhoyer/hrda

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run lhoyer/hrda using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "lhoyer/hrda:5a260c83723c8e79a87a9cba90f5c00e84193bb1c251e327f9beb467e4fa5b64",
  {
    input: {
      opacity: 0.75,
      input_image: "https://replicate.delivery/mgxm/29eefb43-84e0-4f82-b75d-9f07cf90ba8b/16th-Street-Mall-CO-1.jpg"
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run lhoyer/hrda using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "lhoyer/hrda:5a260c83723c8e79a87a9cba90f5c00e84193bb1c251e327f9beb467e4fa5b64",
    input={
        "opacity": 0.75,
        "input_image": "https://replicate.delivery/mgxm/29eefb43-84e0-4f82-b75d-9f07cf90ba8b/16th-Street-Mall-CO-1.jpg"
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run lhoyer/hrda using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "5a260c83723c8e79a87a9cba90f5c00e84193bb1c251e327f9beb467e4fa5b64",
    "input": {
      "opacity": 0.75,
      "input_image": "https://replicate.delivery/mgxm/29eefb43-84e0-4f82-b75d-9f07cf90ba8b/16th-Street-Mall-CO-1.jpg"
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

You can run this model locally using Cog. First, install Cog:

brew install cog

If you don’t have Homebrew, there are other installation options available.

Run this to download the model and run it in your local environment:

cog predict r8.im/lhoyer/hrda@sha256:5a260c83723c8e79a87a9cba90f5c00e84193bb1c251e327f9beb467e4fa5b64 \
  -i 'opacity=0.75' \
  -i 'input_image="https://replicate.delivery/mgxm/29eefb43-84e0-4f82-b75d-9f07cf90ba8b/16th-Street-Mall-CO-1.jpg"'

To learn more, take a look at the Cog documentation.

Run this to download the model and run it in your local environment:

docker run -d -p 5000:5000 --gpus=all r8.im/lhoyer/hrda@sha256:5a260c83723c8e79a87a9cba90f5c00e84193bb1c251e327f9beb467e4fa5b64
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "opacity": 0.75,
      "input_image": "https://replicate.delivery/mgxm/29eefb43-84e0-4f82-b75d-9f07cf90ba8b/16th-Street-Mall-CO-1.jpg"
    }
  }' \
  http://localhost:5000/predictions

To learn more, take a look at the Cog documentation.

Output

Generated in

14.4 seconds

Tweak itReport

Examples

View more examples

Run time and cost

This model runs on Nvidia T4 GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation

by Lukas Hoyer, Dengxin Dai, and Luc Van Gool

[Arxiv] [Paper]

Overview

Unsupervised domain adaptation (UDA) aims to adapt a model trained on synthetic data to real-world data without requiring expensive annotations of real-world images. As UDA methods for semantic segmentation are usually GPU memory intensive, most previous methods operate only on downscaled images. We question this design as low-resolution predictions often fail to preserve fine details. The alternative of training with random crops of high-resolution images alleviates this problem but falls short in capturing long-range, domain-robust context information.

Therefore, we propose HRDA, a multi-resolution training approach for UDA, that combines the strengths of small high-resolution crops to preserve fine segmentation details and large low-resolution crops to capture long-range context dependencies with a learned scale attention, while maintaining a manageable GPU memory footprint.

HRDA Overview

HRDA enables adapting small objects and preserving fine segmentation details. It significantly improves the state-of-the-art performance by 5.5 mIoU for GTA→Cityscapes and by 4.9 mIoU for Synthia→Cityscapes, resulting in an unprecedented performance of 73.8 and 65.8 mIoU, respectively.

UDA over time

The more detailed domain-adaptive semantic segmentation of HRDA, compared to the previous state-of-the-art UDA method DAFormer, can also be observed in example predictions from the Cityscapes validation set.

Demo Color Palette

For more information on HRDA, please check our [Paper].

If you find HRDA useful in your research, please consider citing:

@Article{hoyer2022hrda,
  title={{HRDA}: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation},
  author={Hoyer, Lukas and Dai, Dengxin and Van Gool, Luc},
  journal={arXiv preprint arXiv:2204.13132},
  year={2022}
}