dribnet
/
pixray-text2image
Uses pixray to generate an image from text prompt
Input
Run this model in Node.js with one line of code:
npx create-replicate --model=dribnet/pixray-text2image
or set up a project from scratch
Install Replicate’s Node.js client library:
npm install replicateSet the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Import and set up the client:
import Replicate from "replicate";
const replicate = new Replicate({
auth: process.env.REPLICATE_API_TOKEN,
});Run dribnet/pixray-text2image using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
const output = await replicate.run(
"dribnet/pixray-text2image:50f96fcd1980e7dcaba18e757acbac05e7f2ad4fbcb4a75f86a13c4086df26d0",
{
input: {
drawer: "vqgan",
prompts: "Robots skydiving high above the city",
settings: "\n"
}
}
);
console.log(output);To learn more, take a look at the guide on getting started with Node.js.
Install Replicate’s Python client library:
pip install replicateSet the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Import the client:
import replicateRun dribnet/pixray-text2image using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
output = replicate.run(
"dribnet/pixray-text2image:50f96fcd1980e7dcaba18e757acbac05e7f2ad4fbcb4a75f86a13c4086df26d0",
input={
"drawer": "vqgan",
"prompts": "Robots skydiving high above the city",
"settings": "\n"
}
)
# The dribnet/pixray-text2image model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
# https://replicate.com/dribnet/pixray-text2image/api#output-schema
print(item)To learn more, take a look at the guide on getting started with Python.
Set the
REPLICATE_API_TOKEN environment variable:export REPLICATE_API_TOKEN=<paste-your-token-here>Find your API token in your account settings.
Run dribnet/pixray-text2image using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.
curl -s -X POST \
-H "Authorization: Bearer $REPLICATE_API_TOKEN" \
-H "Content-Type: application/json" \
-H "Prefer: wait" \
-d $'{
"version": "50f96fcd1980e7dcaba18e757acbac05e7f2ad4fbcb4a75f86a13c4086df26d0",
"input": {
"drawer": "vqgan",
"prompts": "Robots skydiving high above the city",
"settings": "\\n"
}
}' \
https://api.replicate.com/v1/predictionsTo learn more, take a look at Replicate’s HTTP API reference docs.
You can run this model locally using Cog. First, install Cog:
brew install cogIf you don’t have Homebrew, there are other installation options available.
Run this to download the model and run it in your local environment:
cog predict r8.im/dribnet/pixray-text2image@sha256:50f96fcd1980e7dcaba18e757acbac05e7f2ad4fbcb4a75f86a13c4086df26d0 \
-i 'drawer="vqgan"' \
-i 'prompts="Robots skydiving high above the city"' \
-i $'settings="\\n"'To learn more, take a look at the Cog documentation.
Run this to download the model and run it in your local environment:
docker run -d -p 5000:5000 --gpus=all r8.im/dribnet/pixray-text2image@sha256:50f96fcd1980e7dcaba18e757acbac05e7f2ad4fbcb4a75f86a13c4086df26d0curl -s -X POST \ -H "Content-Type: application/json" \ -d $'{ "input": { "drawer": "vqgan", "prompts": "Robots skydiving high above the city", "settings": "\\n" } }' \ http://localhost:5000/predictions
To learn more, take a look at the Cog documentation.
Add a payment method to run this model.
Each run costs approximately $0.080. Alternatively, try out our featured models for free.
Sign in with GitHub
By signing in, you agree to our
terms of service and privacy policy
Output
{
"completed_at": "2021-12-01T01:31:24.192336Z",
"created_at": "2021-12-01T01:27:12.367827Z",
"data_removed": false,
"error": null,
"id": "2bce3gyk7ja7bc6el56h7marn4",
"input": {
"aspect": "widescreen",
"prompts": "Robots skydiving high above the city",
"quality": "normal"
},
"logs": "---> BasePixrayPredictor Predict\nUsing seed:\n7848817447306549499\nWorking with z of shape (1, 256, 16, 16) = 65536 dimensions.\nloaded pretrained LPIPS loss from taming/modules/autoencoder/lpips/vgg.pth\nVQLPIPSWithDiscriminator running with hinge loss.\nRestored from models/vqgan_imagenet_f16_16384.ckpt\nUsing device:\ncuda:0\nOptimising using:\nAdam\nUsing text prompts:\n['Robots skydiving high above the city']\n\n0it [00:00, ?it/s]\n/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/functional.py:3609: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.\n warnings.warn(\niter: 0, loss: 2, losses: 0.965, 0.0457, 0.943, 0.0472 (-0=>2.001)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 10, loss: 1.81, losses: 0.862, 0.0465, 0.852, 0.046 (-0=>1.807)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 20, loss: 1.69, losses: 0.806, 0.052, 0.785, 0.0511 (-0=>1.694)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 30, loss: 1.66, losses: 0.782, 0.0555, 0.77, 0.0534 (-6=>1.655)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 40, loss: 1.62, losses: 0.764, 0.056, 0.748, 0.0536 (-2=>1.586)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 50, loss: 1.64, losses: 0.779, 0.0543, 0.755, 0.053 (-8=>1.583)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 60, loss: 1.6, losses: 0.759, 0.056, 0.733, 0.0546 (-4=>1.558)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 70, loss: 1.55, losses: 0.729, 0.0566, 0.706, 0.0551 (-0=>1.547)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 80, loss: 1.55, losses: 0.731, 0.0557, 0.705, 0.0539 (-0=>1.546)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 90, loss: 1.59, losses: 0.754, 0.0568, 0.723, 0.0555 (-6=>1.535)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 100, loss: 1.57, losses: 0.749, 0.057, 0.709, 0.0565 (-2=>1.532)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 110, loss: 1.57, losses: 0.744, 0.0581, 0.708, 0.057 (-2=>1.522)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 120, loss: 1.57, losses: 0.741, 0.0591, 0.709, 0.0576 (-12=>1.522)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 130, loss: 1.52, losses: 0.716, 0.0608, 0.682, 0.0593 (-4=>1.515)\n\n0it [00:00, ?it/s]\n\n0it [00:08, ?it/s]\n\n0it [00:00, ?it/s]\niter: 140, loss: 1.57, losses: 0.744, 0.0588, 0.71, 0.0593 (-14=>1.515)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 150, loss: 1.55, losses: 0.733, 0.0606, 0.698, 0.0588 (-24=>1.515)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 160, loss: 1.49, losses: 0.7, 0.0611, 0.668, 0.061 (-0=>1.491)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 170, loss: 1.53, losses: 0.719, 0.0616, 0.691, 0.0592 (-10=>1.491)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 180, loss: 1.55, losses: 0.728, 0.0591, 0.702, 0.0591 (-20=>1.491)\n\n0it [00:00, ?it/s]\nDropping learning rate\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 190, loss: 1.55, losses: 0.725, 0.062, 0.7, 0.0606 (-2=>1.503)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 200, loss: 1.53, losses: 0.721, 0.0601, 0.694, 0.0592 (-3=>1.495)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 210, loss: 1.48, losses: 0.695, 0.0616, 0.666, 0.0614 (-0=>1.484)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 220, loss: 1.54, losses: 0.723, 0.0609, 0.693, 0.0603 (-6=>1.482)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 230, loss: 1.49, losses: 0.697, 0.062, 0.665, 0.0624 (-16=>1.482)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 240, loss: 1.53, losses: 0.717, 0.0617, 0.692, 0.0604 (-26=>1.482)\n\n0it [00:00, ?it/s]\n\n0it [00:09, ?it/s]\n\n0it [00:00, ?it/s]\niter: 250, finished (-2=>1.481)\n\n0it [00:00, ?it/s]\n\n0it [00:00, ?it/s]",
"metrics": {
"predict_time": 250.596219,
"total_time": 251.824509
},
"output": [
{
"file": "https://replicate.delivery/mgxm/4992351b-d1db-4415-b25a-a51a3fe8f8fe/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/469032f7-7a0b-46c8-8c4c-36f70bb37b37/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/2a82daf7-731b-4641-b7de-661a31a49fae/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/a4ecc63b-222f-474f-8ade-a2d7d4464fd4/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/e5fa5aad-ca17-4451-8ed1-9ba5a6cfc6ab/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/d018ef02-8764-449c-baed-02db65187f2e/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/054e7679-b7a8-4fed-88c3-56fd42665954/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/e1f0b3f2-407a-43bf-a735-1e969fd4befd/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/c38e2952-bf2d-4faa-9a03-2ec4de3f3683/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/9f848682-6373-415e-9b6c-26f393f85c91/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/1df4cd14-a0c2-4a97-a5fa-bf22c07f18ea/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/17f4718e-0f3c-482d-8985-2f0338cf2468/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/9f360e10-d2f0-44c9-b413-6dbc157a15b8/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/8b3ca6b3-99a3-4a65-a3ef-e142b0980692/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/a5319666-d2d0-46a8-aebf-f015addbe856/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/772a808b-2d3c-41a0-acd1-1c30106c67ab/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/f0e788fd-cee9-4ba5-8216-e5274035551d/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/2338f84b-d5af-4afe-93ce-a888b97cda7c/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/b7f65950-c1d4-408f-bd53-ebeb0d0c7e72/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/9b500b52-622a-4678-bdd6-0a26f9f9033d/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/431af114-5799-4d87-bae3-92f87cdcd801/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/11f1bce4-9d95-4f5b-ade4-dd24916a6032/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/edf782f5-2fa2-43e9-8e34-bdf4c9933b85/tempfile.png"
},
{
"file": "https://replicate.delivery/mgxm/cbd88190-7873-4fd9-8bfb-9a89451c0451/tempfile.png"
}
],
"started_at": "2021-12-01T01:27:13.596117Z",
"status": "succeeded",
"urls": {
"get": "https://api.replicate.com/v1/predictions/2bce3gyk7ja7bc6el56h7marn4",
"cancel": "https://api.replicate.com/v1/predictions/2bce3gyk7ja7bc6el56h7marn4/cancel"
},
"version": "7eeecc6f78234419555288d44cc7a5bc3eee80519da21c24efef6fbffe4cc3fe"
}Generated in
---> BasePixrayPredictor Predict
Using seed:
7848817447306549499
Working with z of shape (1, 256, 16, 16) = 65536 dimensions.
loaded pretrained LPIPS loss from taming/modules/autoencoder/lpips/vgg.pth
VQLPIPSWithDiscriminator running with hinge loss.
Restored from models/vqgan_imagenet_f16_16384.ckpt
Using device:
cuda:0
Optimising using:
Adam
Using text prompts:
['Robots skydiving high above the city']
0it [00:00, ?it/s]
/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/nn/functional.py:3609: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
warnings.warn(
iter: 0, loss: 2, losses: 0.965, 0.0457, 0.943, 0.0472 (-0=>2.001)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 10, loss: 1.81, losses: 0.862, 0.0465, 0.852, 0.046 (-0=>1.807)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 20, loss: 1.69, losses: 0.806, 0.052, 0.785, 0.0511 (-0=>1.694)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 30, loss: 1.66, losses: 0.782, 0.0555, 0.77, 0.0534 (-6=>1.655)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 40, loss: 1.62, losses: 0.764, 0.056, 0.748, 0.0536 (-2=>1.586)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 50, loss: 1.64, losses: 0.779, 0.0543, 0.755, 0.053 (-8=>1.583)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 60, loss: 1.6, losses: 0.759, 0.056, 0.733, 0.0546 (-4=>1.558)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 70, loss: 1.55, losses: 0.729, 0.0566, 0.706, 0.0551 (-0=>1.547)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 80, loss: 1.55, losses: 0.731, 0.0557, 0.705, 0.0539 (-0=>1.546)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 90, loss: 1.59, losses: 0.754, 0.0568, 0.723, 0.0555 (-6=>1.535)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 100, loss: 1.57, losses: 0.749, 0.057, 0.709, 0.0565 (-2=>1.532)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 110, loss: 1.57, losses: 0.744, 0.0581, 0.708, 0.057 (-2=>1.522)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 120, loss: 1.57, losses: 0.741, 0.0591, 0.709, 0.0576 (-12=>1.522)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 130, loss: 1.52, losses: 0.716, 0.0608, 0.682, 0.0593 (-4=>1.515)
0it [00:00, ?it/s]
0it [00:08, ?it/s]
0it [00:00, ?it/s]
iter: 140, loss: 1.57, losses: 0.744, 0.0588, 0.71, 0.0593 (-14=>1.515)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 150, loss: 1.55, losses: 0.733, 0.0606, 0.698, 0.0588 (-24=>1.515)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 160, loss: 1.49, losses: 0.7, 0.0611, 0.668, 0.061 (-0=>1.491)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 170, loss: 1.53, losses: 0.719, 0.0616, 0.691, 0.0592 (-10=>1.491)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 180, loss: 1.55, losses: 0.728, 0.0591, 0.702, 0.0591 (-20=>1.491)
0it [00:00, ?it/s]
Dropping learning rate
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 190, loss: 1.55, losses: 0.725, 0.062, 0.7, 0.0606 (-2=>1.503)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 200, loss: 1.53, losses: 0.721, 0.0601, 0.694, 0.0592 (-3=>1.495)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 210, loss: 1.48, losses: 0.695, 0.0616, 0.666, 0.0614 (-0=>1.484)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 220, loss: 1.54, losses: 0.723, 0.0609, 0.693, 0.0603 (-6=>1.482)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 230, loss: 1.49, losses: 0.697, 0.062, 0.665, 0.0624 (-16=>1.482)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 240, loss: 1.53, losses: 0.717, 0.0617, 0.692, 0.0604 (-26=>1.482)
0it [00:00, ?it/s]
0it [00:09, ?it/s]
0it [00:00, ?it/s]
iter: 250, finished (-2=>1.481)
0it [00:00, ?it/s]
0it [00:00, ?it/s]This example was created by a different version, dribnet/pixray-text2image:7eeecc6f.
Examples
Run time and cost
This model costs approximately $0.080 to run on Replicate, or 12 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.
This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 6 minutes. The predict time for this model varies significantly based on the inputs.