paper11667 / clipstyler

Image Style Transfer with Text Condition

Cold

Public
25.9K runs
T4
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

Run this model in Node.js with one line of code:

npx create-replicate --model=paper11667/clipstyler

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run paper11667/clipstyler using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "paper11667/clipstyler:f2d6b24e6002f25f77ae89c2b0a5987daa6d0bf751b858b94b8416e8542434d1",
  {
    input: {
      text: "sketch with crayon",
      image: "https://replicate.delivery/mgxm/e4500aa0-f71b-42ff-a540-aadb44c8d1b2/face.jpg",
      iterations: 100
    }
  }
);

// To access the file URL:
console.log(output[0].url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output[0]);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run paper11667/clipstyler using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "paper11667/clipstyler:f2d6b24e6002f25f77ae89c2b0a5987daa6d0bf751b858b94b8416e8542434d1",
    input={
        "text": "sketch with crayon",
        "image": "https://replicate.delivery/mgxm/e4500aa0-f71b-42ff-a540-aadb44c8d1b2/face.jpg",
        "iterations": 100
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run paper11667/clipstyler using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "paper11667/clipstyler:f2d6b24e6002f25f77ae89c2b0a5987daa6d0bf751b858b94b8416e8542434d1",
    "input": {
      "text": "sketch with crayon",
      "image": "https://replicate.delivery/mgxm/e4500aa0-f71b-42ff-a540-aadb44c8d1b2/face.jpg",
      "iterations": 100
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

{
  "completed_at": "2021-12-05T15:00:38.792184Z",
  "created_at": "2021-12-05T14:59:30.323839Z",
  "data_removed": false,
  "error": null,
  "id": "qckvikgch5go7mhnqymnheheki",
  "input": {
    "text": "sketch with crayon",
    "image": "https://replicate.delivery/mgxm/e4500aa0-f71b-42ff-a540-aadb44c8d1b2/face.jpg",
    "iterations": "100"
  },
  "logs": "/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/functional.py:3060: UserWarning: Default upsampling behavior when mode=bicubic is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.\n  warnings.warn(\"Default upsampling behavior when mode={} is changed \"\n/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:131: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate\n  warnings.warn(\"Detected call of `lr_scheduler.step()` before `optimizer.step()`. \"\n/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/functional.py:3060: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.\n  warnings.warn(\"Default upsampling behavior when mode={} is changed \"\n/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torchvision/transforms/functional_tensor.py:876: UserWarning: Argument fill/fillcolor is not supported for Tensor input. Fill value is zero\n  warnings.warn(\"Argument fill/fillcolor is not supported for Tensor input. Fill value is zero\")\nAfter 0 iterations\nTotal loss: 9802.1005859375\nContent loss: 5.383391380310059\npatch loss: 0.9482421875\ndir loss: 0.91650390625\nTV loss: 0.3422693610191345\nAfter 20 iterations\nTotal loss: 8772.9951171875\nContent loss: 3.439800500869751\npatch loss: 0.87939453125\ndir loss: 0.681640625\nTV loss: 0.2753937542438507\nAfter 40 iterations\nTotal loss: 8309.794921875\nContent loss: 2.8816094398498535\npatch loss: 0.84228515625\ndir loss: 0.5947265625\nTV loss: 0.30363643169403076\nAfter 60 iterations\nTotal loss: 8068.50048828125\nContent loss: 2.774502754211426\npatch loss: 0.8193359375\ndir loss: 0.5517578125\nTV loss: 0.3253156840801239\nAfter 80 iterations\nTotal loss: 7695.32177734375\nContent loss: 2.5266687870025635\npatch loss: 0.78466796875\ndir loss: 0.50390625\nTV loss: 0.321286678314209\nAfter 100 iterations\nTotal loss: 7482.86083984375\nContent loss: 2.315229892730713\npatch loss: 0.7685546875\ndir loss: 0.4384765625\nTV loss: 0.32614612579345703",
  "metrics": {
    "predict_time": 42.086328,
    "total_time": 68.468345
  },
  "output": [
    {
      "file": "https://replicate.delivery/mgxm/bdcd9c0c-09df-4ea7-8ad8-0efa3fa48eda/out.png"
    },
    {
      "file": "https://replicate.delivery/mgxm/ea95e311-13b1-41e3-bfe1-0d2c9e6dfc5d/out.png"
    },
    {
      "file": "https://replicate.delivery/mgxm/d4e354f6-e22b-4b70-abc8-63e92b0de85d/out.png"
    },
    {
      "file": "https://replicate.delivery/mgxm/9f274601-c587-4d54-b841-f7f3af3f9636/out.png"
    },
    {
      "file": "https://replicate.delivery/mgxm/59d6234b-3662-429d-93ba-637d849f1383/out.png"
    }
  ],
  "started_at": "2021-12-05T14:59:56.705856Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/qckvikgch5go7mhnqymnheheki",
    "cancel": "https://api.replicate.com/v1/predictions/qckvikgch5go7mhnqymnheheki/cancel"
  },
  "version": "847f902b2e3b7d34dc7e500ec7b03eb574a28610fdddf87d527e26e7c4286016"
}

Generated in

42.1 seconds

Tweak itReport

/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/functional.py:3060: UserWarning: Default upsampling behavior when mode=bicubic is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn("Default upsampling behavior when mode={} is changed "
/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:131: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/functional.py:3060: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn("Default upsampling behavior when mode={} is changed "
/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torchvision/transforms/functional_tensor.py:876: UserWarning: Argument fill/fillcolor is not supported for Tensor input. Fill value is zero
  warnings.warn("Argument fill/fillcolor is not supported for Tensor input. Fill value is zero")
After 0 iterations
Total loss: 9802.1005859375
Content loss: 5.383391380310059
patch loss: 0.9482421875
dir loss: 0.91650390625
TV loss: 0.3422693610191345
After 20 iterations
Total loss: 8772.9951171875
Content loss: 3.439800500869751
patch loss: 0.87939453125
dir loss: 0.681640625
TV loss: 0.2753937542438507
After 40 iterations
Total loss: 8309.794921875
Content loss: 2.8816094398498535
patch loss: 0.84228515625
dir loss: 0.5947265625
TV loss: 0.30363643169403076
After 60 iterations
Total loss: 8068.50048828125
Content loss: 2.774502754211426
patch loss: 0.8193359375
dir loss: 0.5517578125
TV loss: 0.3253156840801239
After 80 iterations
Total loss: 7695.32177734375
Content loss: 2.5266687870025635
patch loss: 0.78466796875
dir loss: 0.50390625
TV loss: 0.321286678314209
After 100 iterations
Total loss: 7482.86083984375
Content loss: 2.315229892730713
patch loss: 0.7685546875
dir loss: 0.4384765625
TV loss: 0.32614612579345703

This output was created using a different version of the model, paper11667/clipstyler:847f902b.

Examples

View more examples

Run time and cost

This model costs approximately $0.037 to run on Replicate, or 27 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 3 minutes. The predict time for this model varies significantly based on the inputs.

paper11667 / clipstyler

Input

Output

Examples

Run time and cost

Readme

CLIPstyler

Official Pytorch implementation of “CLIPstyler:Image Style Transfer with a Single Text Condition”

Logs (qckvikgch5go7mhnqymnheheki)