iordcalin/material-transfer – Run with an API on Replicate

iordcalin / material-transfer

Transfer a material from an image to a subject

Cold

Public
8.8K runs
L40S
GitHub
License

Run with an API

Playground API Examples README Versions

Input

material_image

*file

Preview

Material to transfer to the input image

subject_image

*file

Preview

Subject image to transfer the material to

prompt

string

Shift + Return to add a new line

Use a prompt that describe the image when the material is applied

Default: "marble sculpture"

negative_prompt

string

Shift + Return to add a new line

What you do not want to see in the image

Default: ""

guidance_scale

number

(minimum: 1, maximum: 10)

Guidance scale for the diffusion process

Default: 2

steps

integer

Number of steps. 6 steps gives good results, but try increasing to 15 or 20 if you need more detail

Default: 6

max_width

integer

Max width of the output image

Default: 1920

max_height

integer

Max height of the output image

Default: 1920

material_strength

string

Strength of the material

Default: "medium"

return_intermediate_images

boolean

Return intermediate images like mask, and annotated images. Useful for debugging.

Default: false

seed

integer

Set a seed for reproducibility. Random by default.

output_format

string

Format of the output images

Default: "webp"

output_quality

integer

(minimum: 0, maximum: 100)

Quality of the output images, from 0 to 100. 100 is best quality, 0 is lowest quality.

Default: 80

Run this model in Node.js with one line of code:

npx create-replicate --model=iordcalin/material-transfer

or set up a project from scratch

Install Replicate’s Node.js client library

npm install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run iordcalin/material-transfer using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "iordcalin/material-transfer:2f640c887323bae445f8b5154d010b2d31c336dd5f3a1d85acd5ae04d215dea0",
  {
    input: {
      steps: 12,
      prompt: "delicate ruby model of a bird",
      max_width: 1920,
      max_height: 1920,
      output_format: "webp",
      subject_image: "https://replicate.delivery/pbxt/KqS0LyQoJuCpq0D0eQEmcKVuFtT7WJfuxb1pxmaLtk7kpwSt/image.png",
      guidance_scale: 2,
      material_image: "https://replicate.delivery/pbxt/KqS0L9BvaN0MsGQ5v9X1AIuzCbeb5jXYWDVRqmQelQB2qPGP/image.png",
      output_quality: 80,
      negative_prompt: "",
      material_strength: "medium",
      return_intermediate_images: false
    }
  }
);
console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library

pip install replicate

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client

import replicate

Run iordcalin/material-transfer using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "iordcalin/material-transfer:2f640c887323bae445f8b5154d010b2d31c336dd5f3a1d85acd5ae04d215dea0",
    input={
        "steps": 12,
        "prompt": "delicate ruby model of a bird",
        "max_width": 1920,
        "max_height": 1920,
        "output_format": "webp",
        "subject_image": "https://replicate.delivery/pbxt/KqS0LyQoJuCpq0D0eQEmcKVuFtT7WJfuxb1pxmaLtk7kpwSt/image.png",
        "guidance_scale": 2,
        "material_image": "https://replicate.delivery/pbxt/KqS0L9BvaN0MsGQ5v9X1AIuzCbeb5jXYWDVRqmQelQB2qPGP/image.png",
        "output_quality": 80,
        "negative_prompt": "",
        "material_strength": "medium",
        "return_intermediate_images": False
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run iordcalin/material-transfer using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "2f640c887323bae445f8b5154d010b2d31c336dd5f3a1d85acd5ae04d215dea0",
    "input": {
      "steps": 12,
      "prompt": "delicate ruby model of a bird",
      "max_width": 1920,
      "max_height": 1920,
      "output_format": "webp",
      "subject_image": "https://replicate.delivery/pbxt/KqS0LyQoJuCpq0D0eQEmcKVuFtT7WJfuxb1pxmaLtk7kpwSt/image.png",
      "guidance_scale": 2,
      "material_image": "https://replicate.delivery/pbxt/KqS0L9BvaN0MsGQ5v9X1AIuzCbeb5jXYWDVRqmQelQB2qPGP/image.png",
      "output_quality": 80,
      "negative_prompt": "",
      "material_strength": "medium",
      "return_intermediate_images": false
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Install Cog

brew install cog

If you don’t have Homebrew, there are other installation options available.

Pull and run iordcalin/material-transfer using Cog (this will download the full model and run it in your local environment):

cog predict r8.im/fofr/material-transfer@sha256:2f640c887323bae445f8b5154d010b2d31c336dd5f3a1d85acd5ae04d215dea0 \
  -i 'steps=12' \
  -i 'prompt="delicate ruby model of a bird"' \
  -i 'max_width=1920' \
  -i 'max_height=1920' \
  -i 'output_format="webp"' \
  -i 'subject_image="https://replicate.delivery/pbxt/KqS0LyQoJuCpq0D0eQEmcKVuFtT7WJfuxb1pxmaLtk7kpwSt/image.png"' \
  -i 'guidance_scale=2' \
  -i 'material_image="https://replicate.delivery/pbxt/KqS0L9BvaN0MsGQ5v9X1AIuzCbeb5jXYWDVRqmQelQB2qPGP/image.png"' \
  -i 'output_quality=80' \
  -i 'negative_prompt=""' \
  -i 'material_strength="medium"' \
  -i 'return_intermediate_images=false'

To learn more, take a look at the Cog documentation.

Pull and run iordcalin/material-transfer using Docker (this will download the full model and run it in your local environment):

docker run -d -p 5000:5000 --gpus=all r8.im/fofr/material-transfer@sha256:2f640c887323bae445f8b5154d010b2d31c336dd5f3a1d85acd5ae04d215dea0
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d $'{
    "input": {
      "steps": 12,
      "prompt": "delicate ruby model of a bird",
      "max_width": 1920,
      "max_height": 1920,
      "output_format": "webp",
      "subject_image": "https://replicate.delivery/pbxt/KqS0LyQoJuCpq0D0eQEmcKVuFtT7WJfuxb1pxmaLtk7kpwSt/image.png",
      "guidance_scale": 2,
      "material_image": "https://replicate.delivery/pbxt/KqS0L9BvaN0MsGQ5v9X1AIuzCbeb5jXYWDVRqmQelQB2qPGP/image.png",
      "output_quality": 80,
      "negative_prompt": "",
      "material_strength": "medium",
      "return_intermediate_images": false
    }
  }' \
  http://localhost:5000/predictions

Output

{
  "completed_at": "2024-05-01T20:39:39.473008Z",
  "created_at": "2024-05-01T20:39:27.707000Z",
  "data_removed": false,
  "error": null,
  "id": "ph0g7a9jvdrgj0cf6qha7b74w4",
  "input": {
    "steps": 12,
    "prompt": "delicate ruby model of a bird",
    "max_width": 1920,
    "max_height": 1920,
    "output_format": "webp",
    "subject_image": "https://replicate.delivery/pbxt/KqS0LyQoJuCpq0D0eQEmcKVuFtT7WJfuxb1pxmaLtk7kpwSt/image.png",
    "guidance_scale": 2,
    "material_image": "https://replicate.delivery/pbxt/KqS0L9BvaN0MsGQ5v9X1AIuzCbeb5jXYWDVRqmQelQB2qPGP/image.png",
    "output_quality": 80,
    "material_strength": "medium",
    "return_intermediate_images": false
  },
  "logs": "Random seed set to: 1637856312\nChecking inputs\n✅ /tmp/inputs/subject.png\n✅ /tmp/inputs/material.png\n====================================\nChecking weights\nIncluding weights for IPAdapter preset: STANDARD (medium strength)\n✅ sk_model2.pth\n✅ ip-adapter_sd15.safetensors\n✅ ip-adapter_sdxl_vit-h.safetensors\n✅ t2i-adapter_diffusers_xl_lineart.safetensors\n✅ depth_anything_vitb14.pth\n✅ depth_anything_vitl14.pth\n✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors\n✅ depth_anything_vits14.pth\n✅ t2i-adapter_diffusers_xl_depth_midas.safetensors\n✅ juggernautXL_juggernautX.safetensors\n✅ sk_model.pth\n====================================\nRunning workflow\ngot prompt\nExecuting node 46, title: Load Image, class type: LoadImage\nExecuting node 47, title: 🔧 Image Crop, class type: ImageCrop+\nExecuting node 58, title: Preview Image, class type: PreviewImage\nExecuting node 45, title: IPAdapter Advanced, class type: IPAdapterAdvanced\nExecuting node 48, title: Differential Diffusion, class type: DifferentialDiffusion\nExecuting node 49, title: RescaleCFG, class type: RescaleCFG\nExecuting node 6, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode\nExecuting node 41, title: InpaintModelConditioning, class type: InpaintModelConditioning\nExecuting node 50, title: Apply ControlNet (Advanced), class type: ControlNetApplyAdvanced\nExecuting node 52, title: Apply ControlNet (Advanced), class type: ControlNetApplyAdvanced\nExecuting node 10, title: KSampler (Efficient), class type: KSampler (Efficient)\nRequested to load SDXL\nLoading 1 new model\n  0%|          | 0/12 [00:00<?, ?it/s]\n  8%|▊         | 1/12 [00:00<00:08,  1.33it/s]\n 17%|█▋        | 2/12 [00:01<00:05,  1.73it/s]\n 25%|██▌       | 3/12 [00:01<00:04,  1.91it/s]\n 33%|███▎      | 4/12 [00:02<00:03,  2.01it/s]\n 42%|████▏     | 5/12 [00:02<00:03,  2.08it/s]\n 50%|█████     | 6/12 [00:03<00:02,  2.12it/s]\n 58%|█████▊    | 7/12 [00:03<00:02,  2.15it/s]\n 67%|██████▋   | 8/12 [00:03<00:01,  2.17it/s]\n 75%|███████▌  | 9/12 [00:04<00:01,  2.18it/s]\n 83%|████████▎ | 10/12 [00:04<00:00,  2.21it/s]\n 92%|█████████▏| 11/12 [00:05<00:00,  2.27it/s]\n100%|██████████| 12/12 [00:05<00:00,  2.72it/s]\n100%|██████████| 12/12 [00:05<00:00,  2.21it/s]\nExecuting node 9, title: Save Image, class type: SaveImage\nPrompt executed in 8.05 seconds\noutputs:  {'56': {'images': [{'filename': 'ComfyUI_temp_sxdyu_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '57': {'images': [{'filename': 'ComfyUI_temp_mvubh_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '39': {'images': [{'filename': 'ComfyUI_temp_oessj_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '40': {'images': [{'filename': 'ComfyUI_temp_pecmx_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '58': {'images': [{'filename': 'ComfyUI_temp_qjpvz_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '10': {'images': [{'filename': 'ComfyUI_temp_ygbim_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '9': {'images': [{'filename': 'ComfyUI_00001_.png', 'subfolder': '', 'type': 'output'}]}}\n====================================\nContents of /tmp/outputs:\nComfyUI_00001_.png",
  "metrics": {
    "predict_time": 11.726818,
    "total_time": 11.766008
  },
  "output": [
    "https://replicate.delivery/pbxt/j9VxP7h07NYkMJlLgg9CeZm06FIei6IYuG9AGlBzAarKqFwSA/ComfyUI_00001_.webp"
  ],
  "started_at": "2024-05-01T20:39:27.746190Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/ph0g7a9jvdrgj0cf6qha7b74w4",
    "cancel": "https://api.replicate.com/v1/predictions/ph0g7a9jvdrgj0cf6qha7b74w4/cancel"
  },
  "version": "2f640c887323bae445f8b5154d010b2d31c336dd5f3a1d85acd5ae04d215dea0"
}

Generated in

11.7 seconds

Tweak it ShareReport

Random seed set to: 1637856312
Checking inputs
✅ /tmp/inputs/subject.png
✅ /tmp/inputs/material.png
====================================
Checking weights
Including weights for IPAdapter preset: STANDARD (medium strength)
✅ sk_model2.pth
✅ ip-adapter_sd15.safetensors
✅ ip-adapter_sdxl_vit-h.safetensors
✅ t2i-adapter_diffusers_xl_lineart.safetensors
✅ depth_anything_vitb14.pth
✅ depth_anything_vitl14.pth
✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
✅ depth_anything_vits14.pth
✅ t2i-adapter_diffusers_xl_depth_midas.safetensors
✅ juggernautXL_juggernautX.safetensors
✅ sk_model.pth
====================================
Running workflow
got prompt
Executing node 46, title: Load Image, class type: LoadImage
Executing node 47, title: 🔧 Image Crop, class type: ImageCrop+
Executing node 58, title: Preview Image, class type: PreviewImage
Executing node 45, title: IPAdapter Advanced, class type: IPAdapterAdvanced
Executing node 48, title: Differential Diffusion, class type: DifferentialDiffusion
Executing node 49, title: RescaleCFG, class type: RescaleCFG
Executing node 6, title: CLIP Text Encode (Prompt), class type: CLIPTextEncode
Executing node 41, title: InpaintModelConditioning, class type: InpaintModelConditioning
Executing node 50, title: Apply ControlNet (Advanced), class type: ControlNetApplyAdvanced
Executing node 52, title: Apply ControlNet (Advanced), class type: ControlNetApplyAdvanced
Executing node 10, title: KSampler (Efficient), class type: KSampler (Efficient)
Requested to load SDXL
Loading 1 new model
  0%|          | 0/12 [00:00<?, ?it/s]
  8%|▊         | 1/12 [00:00<00:08,  1.33it/s]
 17%|█▋        | 2/12 [00:01<00:05,  1.73it/s]
 25%|██▌       | 3/12 [00:01<00:04,  1.91it/s]
 33%|███▎      | 4/12 [00:02<00:03,  2.01it/s]
 42%|████▏     | 5/12 [00:02<00:03,  2.08it/s]
 50%|█████     | 6/12 [00:03<00:02,  2.12it/s]
 58%|█████▊    | 7/12 [00:03<00:02,  2.15it/s]
 67%|██████▋   | 8/12 [00:03<00:01,  2.17it/s]
 75%|███████▌  | 9/12 [00:04<00:01,  2.18it/s]
 83%|████████▎ | 10/12 [00:04<00:00,  2.21it/s]
 92%|█████████▏| 11/12 [00:05<00:00,  2.27it/s]
100%|██████████| 12/12 [00:05<00:00,  2.72it/s]
100%|██████████| 12/12 [00:05<00:00,  2.21it/s]
Executing node 9, title: Save Image, class type: SaveImage
Prompt executed in 8.05 seconds
outputs:  {'56': {'images': [{'filename': 'ComfyUI_temp_sxdyu_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '57': {'images': [{'filename': 'ComfyUI_temp_mvubh_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '39': {'images': [{'filename': 'ComfyUI_temp_oessj_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '40': {'images': [{'filename': 'ComfyUI_temp_pecmx_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '58': {'images': [{'filename': 'ComfyUI_temp_qjpvz_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '10': {'images': [{'filename': 'ComfyUI_temp_ygbim_00001_.png', 'subfolder': '', 'type': 'temp'}]}, '9': {'images': [{'filename': 'ComfyUI_00001_.png', 'subfolder': '', 'type': 'output'}]}}
====================================
Contents of /tmp/outputs:
ComfyUI_00001_.png

Examples

View more examples

Run time and cost

This model costs approximately $0.12 to run on Replicate, or 8 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 119 seconds. The predict time for this model varies significantly based on the inputs.

Readme

This model doesn't have a readme.