fofr/batch-image-captioning:3adde40e – Run with an API on Replicate

Version

You're looking at a specific version of this model. Jump to the model overview.

fofr /batch-image-captioning:3adde40e

Playground API

Input

image_zip_archive

*file

ZIP archive containing images to process

caption_prefix

string

Shift + Return to add a new line

Optional prefix for image captions

Default: ""

caption_suffix

string

Shift + Return to add a new line

Optional suffix for image captions

Default: ""

resize_images_for_captioning

boolean

Whether to resize images for captioning. This makes captioning cheaper

Default: true

max_dimension

integer

Maximum dimension (width or height) for resized images

Default: 1024

model

string

OpenAI model to use. Your OpenAI account will be charged for usage, see pricing: https://openai.com/api/pricing/

Default: "gpt-4o-2024-08-06"

openai_api_key

secret

OpenAI API key

system_prompt

string

Shift + Return to add a new line

Write a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using "or". Precise composition is important. Avoid phrases like "conveys a sense of" and "capturing the", just use the terms themselves.

Good examples are:

"Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine."

"A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series."

"An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed."
Write a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using "or". Precise composition is important. Avoid phrases like "conveys a sense of" and "capturing the", just use the terms themselves.

Good examples are:

"Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine."

"A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series."

"An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed."

System prompt for image analysis

Default: "\nWrite a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using \"or\". Precise composition is important. Avoid phrases like \"conveys a sense of\" and \"capturing the\", just use the terms themselves.\n\nGood examples are:\n\n\"Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine.\"\n\n\"A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series.\"\n\n\"An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed.\"\n"

message_prompt

string

Shift + Return to add a new line

Message prompt for image captioning

Default: "Caption this image please"

Run this model in Node.js with one line of code:

npx create-replicate --model=fofr/batch-image-captioning

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run fofr/batch-image-captioning using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "fofr/batch-image-captioning:3adde40e56d70b1ff1a6f1300da81b8af9a0f7983163f83022ebdb2c911fdc49",
  {
    input: {
      model: "gpt-4o-2024-08-06",
      max_dimension: 1024,
      system_prompt: "Write a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using \"or\". Precise composition is important. Avoid phrases like \"conveys a sense of\" and \"capturing the\", just use the terms themselves.\n\nGood examples are:\n\n\"Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine.\"\n\n\"A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series.\"\n\n\"An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed.\"\n",
      caption_prefix: "a photo of a phone in a toaster",
      caption_suffix: "",
      message_prompt: "Caption this image please",
      openai_api_key: "",
      image_zip_archive: "https://replicate.delivery/pbxt/LRETkJ5qT5ghj6DfjAhUYUNdufuX47IKlFfUV8QDNyS6uwMy/Archive.zip",
      resize_images_for_captioning: true
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run fofr/batch-image-captioning using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "fofr/batch-image-captioning:3adde40e56d70b1ff1a6f1300da81b8af9a0f7983163f83022ebdb2c911fdc49",
    input={
        "model": "gpt-4o-2024-08-06",
        "max_dimension": 1024,
        "system_prompt": "Write a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using \"or\". Precise composition is important. Avoid phrases like \"conveys a sense of\" and \"capturing the\", just use the terms themselves.\n\nGood examples are:\n\n\"Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine.\"\n\n\"A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series.\"\n\n\"An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed.\"\n",
        "caption_prefix": "a photo of a phone in a toaster",
        "caption_suffix": "",
        "message_prompt": "Caption this image please",
        "openai_api_key": "",
        "image_zip_archive": "https://replicate.delivery/pbxt/LRETkJ5qT5ghj6DfjAhUYUNdufuX47IKlFfUV8QDNyS6uwMy/Archive.zip",
        "resize_images_for_captioning": True
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run fofr/batch-image-captioning using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "fofr/batch-image-captioning:3adde40e56d70b1ff1a6f1300da81b8af9a0f7983163f83022ebdb2c911fdc49",
    "input": {
      "model": "gpt-4o-2024-08-06",
      "max_dimension": 1024,
      "system_prompt": "Write a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using \\"or\\". Precise composition is important. Avoid phrases like \\"conveys a sense of\\" and \\"capturing the\\", just use the terms themselves.\\n\\nGood examples are:\\n\\n\\"Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine.\\"\\n\\n\\"A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series.\\"\\n\\n\\"An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed.\\"\\n",
      "caption_prefix": "a photo of a phone in a toaster",
      "caption_suffix": "",
      "message_prompt": "Caption this image please",
      "openai_api_key": "",
      "image_zip_archive": "https://replicate.delivery/pbxt/LRETkJ5qT5ghj6DfjAhUYUNdufuX47IKlFfUV8QDNyS6uwMy/Archive.zip",
      "resize_images_for_captioning": true
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

captions_and_csv.zip

{
  "completed_at": "2024-08-13T11:37:45.452413Z",
  "created_at": "2024-08-13T11:37:25.208000Z",
  "data_removed": false,
  "error": null,
  "id": "h6t1p9ahv1rgc0ch9e3v9h2d2m",
  "input": {
    "model": "gpt-4o-2024-08-06",
    "max_dimension": 1024,
    "system_prompt": "Write a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using \"or\". Precise composition is important. Avoid phrases like \"conveys a sense of\" and \"capturing the\", just use the terms themselves.\n\nGood examples are:\n\n\"Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine.\"\n\n\"A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series.\"\n\n\"An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed.\"\n",
    "caption_prefix": "a photo of a phone in a toaster",
    "caption_suffix": "",
    "message_prompt": "Caption this image please",
    "openai_api_key": "[REDACTED]",
    "image_zip_archive": "https://replicate.delivery/pbxt/LRETkJ5qT5ghj6DfjAhUYUNdufuX47IKlFfUV8QDNyS6uwMy/Archive.zip",
    "resize_images_for_captioning": true
  },
  "logs": "Files extracted:\n/tmp/outputs/2024-05-14--09-10-07-u-q3-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png\n/tmp/outputs/2024-05-14--09-10-07-u-q2-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png\n/tmp/outputs/2024-05-14--09-10-07-u-q1-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png\n/tmp/outputs/2024-05-14--09-10-07-u-q4-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png\nNumber of images to be captioned: 4\n===================================================\nProcessing 2024-05-14--09-10-07-u-q3-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png\nResized from 928x1232 to 771x1024\nCaption: A photo of a phone in a toaster, on a sunny kitchen countertop. The toaster holds a piece of toast and a phone displaying a full battery icon. A glass mug filled with a beige drink sits in the background. Soft shadows add warmth to the scene, creating a playful juxtaposition.\n===================================================\nProcessing 2024-05-14--09-10-07-u-q2-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png\nResized from 928x1232 to 771x1024\nCaption: A photo of a phone in a toaster, featuring a modern smartphone inserted into the slot of a vibrant orange toaster. The screen displays an image of an orange slice. The scene is set on a wooden countertop with natural light filtering through a nearby window. A clear glass tumbler and wooden kitchen items are faintly visible in the background.\n===================================================\nProcessing 2024-05-14--09-10-07-u-q1-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png\nResized from 928x1232 to 771x1024\nCaption: A photo of a phone in a toaster shows a modern smartphone sitting vertically inside the slot of a retro-style toaster. The phone screen displays the time as 9:17 with a simple background. The toaster has a cream and silver finish with three dials on the front. The setting is a cozy kitchen with a wooden countertop and soft lighting.\n===================================================\nProcessing 2024-05-14--09-10-07-u-q4-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png\nResized from 928x1232 to 771x1024\nCaption: a photo of a phone in a toaster, with a green smartphone inserted into a teal vintage toaster. The toaster is placed on a kitchen countertop with a loaf of bread surrounding the device. In the background, there’s a blurred silver kettle and ceramic cookware. The scene is warmly lit, creating a cozy kitchen atmosphere.\n===================================================",
  "metrics": {
    "predict_time": 20.228402371,
    "total_time": 20.244413
  },
  "output": "https://replicate.delivery/czjl/NBDLQFY9K4rPMNhLTegSWUU230XvRzrLxvickadzVruEvHpJA/captions_and_csv.zip",
  "started_at": "2024-08-13T11:37:25.224010Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/h6t1p9ahv1rgc0ch9e3v9h2d2m",
    "cancel": "https://api.replicate.com/v1/predictions/h6t1p9ahv1rgc0ch9e3v9h2d2m/cancel"
  },
  "version": "3adde40e56d70b1ff1a6f1300da81b8af9a0f7983163f83022ebdb2c911fdc49"
}

Generated in

20.2 seconds

Tweak it ShareReport

Files extracted:
/tmp/outputs/2024-05-14--09-10-07-u-q3-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png
/tmp/outputs/2024-05-14--09-10-07-u-q2-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png
/tmp/outputs/2024-05-14--09-10-07-u-q1-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png
/tmp/outputs/2024-05-14--09-10-07-u-q4-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png
Number of images to be captioned: 4
===================================================
Processing 2024-05-14--09-10-07-u-q3-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png
Resized from 928x1232 to 771x1024
Caption: A photo of a phone in a toaster, on a sunny kitchen countertop. The toaster holds a piece of toast and a phone displaying a full battery icon. A glass mug filled with a beige drink sits in the background. Soft shadows add warmth to the scene, creating a playful juxtaposition.
===================================================
Processing 2024-05-14--09-10-07-u-q2-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png
Resized from 928x1232 to 771x1024
Caption: A photo of a phone in a toaster, featuring a modern smartphone inserted into the slot of a vibrant orange toaster. The screen displays an image of an orange slice. The scene is set on a wooden countertop with natural light filtering through a nearby window. A clear glass tumbler and wooden kitchen items are faintly visible in the background.
===================================================
Processing 2024-05-14--09-10-07-u-q1-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png
Resized from 928x1232 to 771x1024
Caption: A photo of a phone in a toaster shows a modern smartphone sitting vertically inside the slot of a retro-style toaster. The phone screen displays the time as 9:17 with a simple background. The toaster has a cream and silver finish with three dials on the front. The setting is a cozy kitchen with a wooden countertop and soft lighting.
===================================================
Processing 2024-05-14--09-10-07-u-q4-fofr_a_photo_of_a_phone_in_a_toaster_cb8e7a47-edf9-4b11-af8d-cbd799ccc983.png
Resized from 928x1232 to 771x1024
Caption: a photo of a phone in a toaster, with a green smartphone inserted into a teal vintage toaster. The toaster is placed on a kitchen countertop with a loaf of bread surrounding the device. In the background, there’s a blurred silver kettle and ceramic cookware. The scene is warmly lit, creating a cozy kitchen atmosphere.
===================================================