✋ This model is not published yet.

You can claim this model if you're @google-research on GitHub. Contact us.

google-research / maskgit

Masked Generative Image Transformer

Cold

Public
136 runs
T4
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

task_type

string

Choose task type.

Default: "Class-conditional Image Synthesis"

category

*string

Choose the ImageNet label, which determines what type of object to synthesize or edit to.

image_size

integer

Choose the size of the generated image. Output will generate 8 images.

Default: 256

seed

integer

set random seed

Default: 42

image

file

Provide input image for Class-conditional Image Editing. The image will be resized to image_size.

bbox_top_left_height_width

string

Shift + Return to add a new line

For Class-conditional Image Editing, provide the area for image editing in the format of top_left_height_width, e.g. input image 128_64_256_288. Output will show the resized image with the box highlighting the edited area and 8 edited images.

Run this model in Node.js with one line of code:

npx create-replicate --model=google-research/maskgit

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run google-research/maskgit using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "google-research/maskgit:16325bd5e326690813672502341daab987a292560bf7a4fb9295e796785557b5",
  {
    input: {
      seed: 42,
      category: "90) lorikeet",
      task_type: "Class-conditional Image Synthesis",
      image_size: 256
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run google-research/maskgit using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "google-research/maskgit:16325bd5e326690813672502341daab987a292560bf7a4fb9295e796785557b5",
    input={
        "seed": 42,
        "category": "90) lorikeet",
        "task_type": "Class-conditional Image Synthesis",
        "image_size": 256
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run google-research/maskgit using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "16325bd5e326690813672502341daab987a292560bf7a4fb9295e796785557b5",
    "input": {
      "seed": 42,
      "category": "90) lorikeet",
      "task_type": "Class-conditional Image Synthesis",
      "image_size": 256
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Generated in

25.4 seconds

Tweak itReport

Examples

View more examples

Run time and cost

This model runs on Nvidia T4 GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

MaskGIT: Masked Generative Image Transformer

Official Jax Implementation of the CVPR 2022 Paper

Summary

MaskGIT is a novel image synthesis paradigm using a bidirectional transformer decoder. During training, MaskGIT learns to predict randomly masked tokens by attending to tokens in all directions. At inference time, the model begins with generating all tokens of an image simultaneously, and then refines the image iteratively conditioned on the previous generation.

BibTeX

@InProceedings{chang2022maskgit,
  title = {MaskGIT: Masked Generative Image Transformer},
  author={Huiwen Chang and Han Zhang and Lu Jiang and Ce Liu and William T. Freeman},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2022}
}