Prediction

ID

6gx56twc39rme0cm2ccsdrrmcw

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

3m 45s

Created

5 months ago

Input

text: Describe each stage of this image in detail
top_p: 0.9
image1
temperature: 0.1
max_new_tokens: 2048
repetition_penalty: 1.1

{
  "text": "Describe each stage of this image in detail",
  "top_p": 0.9,
  "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/vqa_1.jpg",
  "temperature": 0.1,
  "max_new_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
  {
    input: {
      text: "Describe each stage of this image in detail",
      top_p: 0.9,
      image1: "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/vqa_1.jpg",
      temperature: 0.1,
      max_new_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    input={
        "text": "Describe each stage of this image in detail",
        "top_p": 0.9,
        "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/vqa_1.jpg",
        "temperature": 0.1,
        "max_new_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    "input": {
      "text": "Describe each stage of this image in detail",
      "top_p": 0.9,
      "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/vqa_1.jpg",
      "temperature": 0.1,
      "max_new_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

The figure illustrates the three-stage training process for a Vision-Language (VL) model using the DeepSeek LLM. Here's a detailed description of each stage: ### Stage 1: Training VL Adaptor - **Components Involved:** - **DeepSeek LLM:** This is the large language model that serves as the backbone for the entire process. - **Vision-Language Adaptor:** This component is responsible for adapting the vision and language models to work together effectively. - **Hybrid Vision Encoder:** This encoder processes visual data and converts it into a format suitable for the model. It uses two specific models, SAM-B and SigLIP-L, which are pre-trained on image-text pairs. - **Image-Text Pairs:** These pairs are used to train the hybrid vision encoder. The encoder learns to understand and generate text based on visual inputs. - **Process:** - The hybrid vision encoder is trained using image-text pairs. This training helps the encoder learn how to map images to corresponding text descriptions. ### Stage 2: Joint VL Pre-training - **Components Involved:** - **DeepSeek LLM:** Continues to be the central model. - **Vision-Language Adaptor:** Continues to adapt the vision and language components. - **Hybrid Vision Encoder:** Continues to process visual data. - **Interleaved VL + Pure Language Sequences:** In this stage, both vision-language sequences and pure language sequences are interleaved. This means that the model is exposed to both types of data during training, allowing it to learn from a more diverse set of examples. - **Process:** - The model is pre-trained using a combination of vision-language sequences and pure language sequences. This joint training helps the model develop a better understanding of both visual and textual information. ### Stage 3: Supervised Finetuning - **Components Involved:** - **DeepSeek LLM:** Continues to be the central model. - **Vision-Language Adaptor:** Continues to refine the adaptation between vision and language. - **Hybrid Vision Encoder:** Continues to process visual data. - **VL Chat Data + Pure Language Chat Data:** In this final stage, the model is fine-tuned using both vision-language chat data and pure language chat data. This step focuses on improving the model's performance specifically in generating responses to chat queries. - **Process:** - The model undergoes supervised finetuning with a focus on chat tasks. The finetuning process involves adjusting the model parameters to optimize its performance in generating coherent and contextually relevant responses to user queries. ### Summary The figure outlines a comprehensive approach to training a Vision-Language model using the DeepSeek LLM. Starting with training the VL adaptor and hybrid vision encoder on image-text pairs, followed by joint pre-training with interleaved sequences, and finally, supervised finetuning with chat data, the model is progressively refined to enhance its ability to handle complex vision-language tasks.<｜end▁of▁sentence｜>

{
  "completed_at": "2024-12-29T15:50:53.493097Z",
  "created_at": "2024-12-29T15:47:08.954000Z",
  "data_removed": false,
  "error": null,
  "id": "6gx56twc39rme0cm2ccsdrrmcw",
  "input": {
    "text": "Describe each stage of this image in detail",
    "top_p": 0.9,
    "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/vqa_1.jpg",
    "temperature": 0.1,
    "max_new_tokens": 2048,
    "repetition_penalty": 1.1
  },
  "logs": "2024-12-29 15:48:34,853 - gradio_logger - INFO - ================================================================================\n2024-12-29 15:48:34,853 - gradio_logger - INFO - <|User|>: <image>\nDescribe each stage of this image in detail\n<|Assistant|>:\n[{'role': '<|User|>', 'content': '<image>\\nDescribe each stage of this image in detail', 'images': [<PIL.Image.Image image mode=RGB size=5014x2004 at 0x7251DA80DFD0>]}, {'role': '<|Assistant|>', 'content': ''}]\nYou're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.",
  "metrics": {
    "predict_time": 139.019578712,
    "total_time": 224.539097
  },
  "output": {
    "img_out": null,
    "text_out": "The figure illustrates the three-stage training process for a Vision-Language (VL) model using the DeepSeek LLM. Here's a detailed description of each stage:\n\n### Stage 1: Training VL Adaptor\n- **Components Involved:**\n  - **DeepSeek LLM:** This is the large language model that serves as the backbone for the entire process.\n  - **Vision-Language Adaptor:** This component is responsible for adapting the vision and language models to work together effectively.\n  - **Hybrid Vision Encoder:** This encoder processes visual data and converts it into a format suitable for the model. It uses two specific models, SAM-B and SigLIP-L, which are pre-trained on image-text pairs.\n  - **Image-Text Pairs:** These pairs are used to train the hybrid vision encoder. The encoder learns to understand and generate text based on visual inputs.\n\n- **Process:**\n  - The hybrid vision encoder is trained using image-text pairs. This training helps the encoder learn how to map images to corresponding text descriptions.\n\n### Stage 2: Joint VL Pre-training\n- **Components Involved:**\n  - **DeepSeek LLM:** Continues to be the central model.\n  - **Vision-Language Adaptor:** Continues to adapt the vision and language components.\n  - **Hybrid Vision Encoder:** Continues to process visual data.\n  - **Interleaved VL + Pure Language Sequences:** In this stage, both vision-language sequences and pure language sequences are interleaved. This means that the model is exposed to both types of data during training, allowing it to learn from a more diverse set of examples.\n\n- **Process:**\n  - The model is pre-trained using a combination of vision-language sequences and pure language sequences. This joint training helps the model develop a better understanding of both visual and textual information.\n\n### Stage 3: Supervised Finetuning\n- **Components Involved:**\n  - **DeepSeek LLM:** Continues to be the central model.\n  - **Vision-Language Adaptor:** Continues to refine the adaptation between vision and language.\n  - **Hybrid Vision Encoder:** Continues to process visual data.\n  - **VL Chat Data + Pure Language Chat Data:** In this final stage, the model is fine-tuned using both vision-language chat data and pure language chat data. This step focuses on improving the model's performance specifically in generating responses to chat queries.\n\n- **Process:**\n  - The model undergoes supervised finetuning with a focus on chat tasks. The finetuning process involves adjusting the model parameters to optimize its performance in generating coherent and contextually relevant responses to user queries.\n\n### Summary\nThe figure outlines a comprehensive approach to training a Vision-Language model using the DeepSeek LLM. Starting with training the VL adaptor and hybrid vision encoder on image-text pairs, followed by joint pre-training with interleaved sequences, and finally, supervised finetuning with chat data, the model is progressively refined to enhance its ability to handle complex vision-language tasks.<｜end▁of▁sentence｜>"
  },
  "started_at": "2024-12-29T15:48:34.473518Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/6gx56twc39rme0cm2ccsdrrmcw",
    "cancel": "https://api.replicate.com/v1/predictions/6gx56twc39rme0cm2ccsdrrmcw/cancel"
  },
  "version": "8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb"
}

Generated in

2 minutes 19 seconds

chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb

Prediction

Model

ID

8knq41b1zxrme0cm2cfb8jxneg

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

10.1s

Created

5 months ago

Input

text: <|ref|>The giraffe at the back.<|/ref|>
top_p: 0.9
image1
temperature: 0.1
max_new_tokens: 2048
repetition_penalty: 1.1

{
  "text": "<|ref|>The giraffe at the back.<|/ref|>",
  "top_p": 0.9,
  "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/visual_grounding_1.jpeg",
  "temperature": 0.1,
  "max_new_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
  {
    input: {
      text: "<|ref|>The giraffe at the back.<|/ref|>",
      top_p: 0.9,
      image1: "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/visual_grounding_1.jpeg",
      temperature: 0.1,
      max_new_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    input={
        "text": "<|ref|>The giraffe at the back.<|/ref|>",
        "top_p": 0.9,
        "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/visual_grounding_1.jpeg",
        "temperature": 0.1,
        "max_new_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    "input": {
      "text": "<|ref|>The giraffe at the back.<|/ref|>",
      "top_p": 0.9,
      "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/visual_grounding_1.jpeg",
      "temperature": 0.1,
      "max_new_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

img_out

text_out

<|ref|>The giraffe at the back.<|/ref|><|det|>[[583, 271, 942, 888]]<|/det|><｜end▁of▁sentence｜>

Generated in

8.5 seconds

chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb

Prediction

Model

ID

g7g5qnefrhrma0cm2cft94nwvw

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

37.9s

Created

5 months ago

Input

text: <|grounding|>In the first image, an object within the red rectangle is marked. Locate the object of the same category in the second image.
top_p: 0.9
image1
image2
temperature: 0.1
max_new_tokens: 2048
repetition_penalty: 1.1

{
  "text": "<|grounding|>In the first image, an object within the red rectangle is marked. Locate the object of the same category in the second image.",
  "top_p": 0.9,
  "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/incontext_visual_grounding_1.jpeg",
  "image2": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/icl_vg_2.jpeg",
  "temperature": 0.1,
  "max_new_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
  {
    input: {
      text: "<|grounding|>In the first image, an object within the red rectangle is marked. Locate the object of the same category in the second image.",
      top_p: 0.9,
      image1: "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/incontext_visual_grounding_1.jpeg",
      image2: "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/icl_vg_2.jpeg",
      temperature: 0.1,
      max_new_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    input={
        "text": "<|grounding|>In the first image, an object within the red rectangle is marked. Locate the object of the same category in the second image.",
        "top_p": 0.9,
        "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/incontext_visual_grounding_1.jpeg",
        "image2": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/icl_vg_2.jpeg",
        "temperature": 0.1,
        "max_new_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    "input": {
      "text": "<|grounding|>In the first image, an object within the red rectangle is marked. Locate the object of the same category in the second image.",
      "top_p": 0.9,
      "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/incontext_visual_grounding_1.jpeg",
      "image2": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/icl_vg_2.jpeg",
      "temperature": 0.1,
      "max_new_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

img_out

text_out

<|ref|>bicolor cat<|/ref|><|det|>[[81, 273, 906, 820]]<|/det|><｜end▁of▁sentence｜>

{
  "completed_at": "2024-12-29T15:54:37.402571Z",
  "created_at": "2024-12-29T15:53:59.492000Z",
  "data_removed": false,
  "error": null,
  "id": "g7g5qnefrhrma0cm2cft94nwvw",
  "input": {
    "text": "<|grounding|>In the first image, an object within the red rectangle is marked. Locate the object of the same category in the second image.",
    "top_p": 0.9,
    "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/incontext_visual_grounding_1.jpeg",
    "image2": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/icl_vg_2.jpeg",
    "temperature": 0.1,
    "max_new_tokens": 2048,
    "repetition_penalty": 1.1
  },
  "logs": "2024-12-29 15:54:29,706 - gradio_logger - INFO - ================================================================================\n2024-12-29 15:54:29,706 - gradio_logger - INFO - <|User|>: <image>\n<image>\n<|grounding|>In the first image, an object within the red rectangle is marked. Locate the object of the same category in the second image.\n<|Assistant|>:\n[{'role': '<|User|>', 'content': '<image>\\n<image>\\n<|grounding|>In the first image, an object within the red rectangle is marked. Locate the object of the same category in the second image.', 'images': [<PIL.Image.Image image mode=RGB size=1738x1146 at 0x7250C05E17D0>, <PIL.Image.Image image mode=RGB size=868x1160 at 0x7250C0592D10>]}, {'role': '<|Assistant|>', 'content': ''}]",
  "metrics": {
    "predict_time": 8.492753054,
    "total_time": 37.910571
  },
  "output": {
    "img_out": "https://replicate.delivery/xezq/zrbPK8uiALqaOdMRCRocFTElQZonyP82Ta1pC2pgHcbvi8fJA/out.png",
    "text_out": "<|ref|>bicolor cat<|/ref|><|det|>[[81, 273, 906, 820]]<|/det|><｜end▁of▁sentence｜>"
  },
  "started_at": "2024-12-29T15:54:28.909818Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/g7g5qnefrhrma0cm2cft94nwvw",
    "cancel": "https://api.replicate.com/v1/predictions/g7g5qnefrhrma0cm2cft94nwvw/cancel"
  },
  "version": "8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb"
}

Generated in

8.5 seconds

chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb

Prediction

Model

ID

54dbyeaknxrme0cm2cgt4eyzgg

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

1m 38s

Created

5 months ago

Input

text: <|grounding|>I want to throw out the trash now, what should I do?
top_p: 0.9
image1
temperature: 0.1
max_new_tokens: 2048
repetition_penalty: 1.1

{
  "text": "<|grounding|>I want to throw out the trash now, what should I do?",
  "top_p": 0.9,
  "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/grounding_conversation_1.jpeg",
  "temperature": 0.1,
  "max_new_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
  {
    input: {
      text: "<|grounding|>I want to throw out the trash now, what should I do?",
      top_p: 0.9,
      image1: "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/grounding_conversation_1.jpeg",
      temperature: 0.1,
      max_new_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    input={
        "text": "<|grounding|>I want to throw out the trash now, what should I do?",
        "top_p": 0.9,
        "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/grounding_conversation_1.jpeg",
        "temperature": 0.1,
        "max_new_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    "input": {
      "text": "<|grounding|>I want to throw out the trash now, what should I do?",
      "top_p": 0.9,
      "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/grounding_conversation_1.jpeg",
      "temperature": 0.1,
      "max_new_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

img_out

text_out

You can use <|ref|>the trash can<|/ref|><|det|>[[589, 643, 807, 999]]<|/det|> that's located on the sidewalk. It has a lid and is designed for public use.<｜end▁of▁sentence｜>

Generated in

12.3 seconds

chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb

Prediction

Model

ID

wksm0fwe7hrmc0cm2cg81fvfbw

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

2m 17s

Created

5 months ago

Input

text: Give me some suggestions for recipes using all the materials in the three images
top_p: 0.9
image1
image2
image3
temperature: 0.1
max_new_tokens: 2048
repetition_penalty: 1.1

{
  "text": "Give me some suggestions for recipes using all the materials in the three images",
  "top_p": 0.9,
  "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_1.jpeg",
  "image2": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_2.jpeg",
  "image3": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_3.jpeg",
  "temperature": 0.1,
  "max_new_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
  {
    input: {
      text: "Give me some suggestions for recipes using all the materials in the three images",
      top_p: 0.9,
      image1: "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_1.jpeg",
      image2: "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_2.jpeg",
      image3: "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_3.jpeg",
      temperature: 0.1,
      max_new_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    input={
        "text": "Give me some suggestions for recipes using all the materials in the three images",
        "top_p": 0.9,
        "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_1.jpeg",
        "image2": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_2.jpeg",
        "image3": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_3.jpeg",
        "temperature": 0.1,
        "max_new_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    "input": {
      "text": "Give me some suggestions for recipes using all the materials in the three images",
      "top_p": 0.9,
      "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_1.jpeg",
      "image2": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_2.jpeg",
      "image3": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_3.jpeg",
      "temperature": 0.1,
      "max_new_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Certainly! Here are a few recipe ideas that incorporate carrots, corn, and raw meat (likely pork ribs): ### Carrot Corn Pork Ribs Soup **Ingredients:** - 4 large carrots - 2 ears of corn - 1 pound pork ribs - 2 tablespoons olive oil - 1 onion, chopped - 3 cloves garlic, minced - 8 cups chicken broth - 1 teaspoon dried thyme - Salt and pepper to taste **Instructions:** 1. **Preparation:** - Peel and chop the carrots into bite-sized pieces. - Remove the husks from the corn cobs and cut off the ends. - Cut the pork ribs into smaller pieces. 2. **Cooking:** - Heat the olive oil in a large pot over medium heat. - Add the chopped onion and sauté until translucent. - Add the minced garlic and cook for another minute. - Add the pork ribs and brown them on all sides. - Pour in the chicken broth and add the thyme. - Bring to a boil, then reduce the heat and let it simmer for about an hour or until the meat is tender. - Add the carrots and corn cobs, cover, and continue cooking for another 30 minutes. - Season with salt and pepper to taste. ### Grilled Pork Ribs with Roasted Carrots and Corn Salsa **Ingredients:** - 4 pork ribs - 4 large carrots - 2 ears of corn - 1/4 cup olive oil - 2 tablespoons honey - 1 tablespoon soy sauce - 1 teaspoon ground cumin - 1 teaspoon smoked paprika - 1/4 cup fresh cilantro, chopped - Salt and pepper to taste **Instructions:** 1. **Marinating:** - In a bowl, mix together olive oil, honey, soy sauce, cumin, smoked paprika, and salt and pepper. - Place the pork ribs in the marinade, turning to coat evenly. Let it marinate for at least 2 hours, preferably overnight. 2. **Grilling:** - Preheat your grill to medium-high heat. - Remove the ribs from the marinade, letting any excess liquid drip off. - Grill the ribs for about 5-7 minutes per side, or until they reach your desired level of doneness. 3. **Roasting:** - Preheat your oven to 400°F (200°C). - Toss the carrots and corn with olive oil, salt, and pepper. Spread them out on a baking sheet. - Roast in the oven for about 20-25 minutes, or until the vegetables are tender and slightly caramelized. 4. **Assembly:** - Serve the grilled pork ribs hot, accompanied by roasted carrots and corn. - Garnish with fresh cilantro and serve with a side of salsa made from the remaining marinade ingredients. These recipes should provide a delicious and hearty meal using the ingredients shown in the images. Enjoy cooking!<｜end▁of▁sentence｜>

{
  "completed_at": "2024-12-29T15:57:04.808972Z",
  "created_at": "2024-12-29T15:54:48.252000Z",
  "data_removed": false,
  "error": null,
  "id": "wksm0fwe7hrmc0cm2cg81fvfbw",
  "input": {
    "text": "Give me some suggestions for recipes using all the materials in the three images",
    "top_p": 0.9,
    "image1": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_1.jpeg",
    "image2": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_2.jpeg",
    "image3": "https://raw.githubusercontent.com/deepseek-ai/DeepSeek-VL2/refs/heads/main/images/multi_image_3.jpeg",
    "temperature": 0.1,
    "max_new_tokens": 2048,
    "repetition_penalty": 1.1
  },
  "logs": "2024-12-29 15:54:48,661 - gradio_logger - INFO - ================================================================================\n2024-12-29 15:54:48,661 - gradio_logger - INFO - <|User|>: <image>\n<image>\n<image>\nGive me some suggestions for recipes using all the materials in the three images\n<|Assistant|>:\n[{'role': '<|User|>', 'content': '<image>\\n<image>\\n<image>\\nGive me some suggestions for recipes using all the materials in the three images', 'images': [<PIL.Image.Image image mode=RGB size=1024x701 at 0x7250C672E010>, <PIL.Image.Image image mode=RGB size=450x308 at 0x7250ACC8E490>, <PIL.Image.Image image mode=RGB size=1280x720 at 0x7250EBAC3150>]}, {'role': '<|Assistant|>', 'content': ''}]",
  "metrics": {
    "predict_time": 136.549170906,
    "total_time": 136.556972
  },
  "output": {
    "img_out": null,
    "text_out": "Certainly! Here are a few recipe ideas that incorporate carrots, corn, and raw meat (likely pork ribs):\n\n### Carrot Corn Pork Ribs Soup\n**Ingredients:**\n- 4 large carrots\n- 2 ears of corn\n- 1 pound pork ribs\n- 2 tablespoons olive oil\n- 1 onion, chopped\n- 3 cloves garlic, minced\n- 8 cups chicken broth\n- 1 teaspoon dried thyme\n- Salt and pepper to taste\n\n**Instructions:**\n1. **Preparation:**\n   - Peel and chop the carrots into bite-sized pieces.\n   - Remove the husks from the corn cobs and cut off the ends.\n   - Cut the pork ribs into smaller pieces.\n\n2. **Cooking:**\n   - Heat the olive oil in a large pot over medium heat.\n   - Add the chopped onion and sauté until translucent.\n   - Add the minced garlic and cook for another minute.\n   - Add the pork ribs and brown them on all sides.\n   - Pour in the chicken broth and add the thyme.\n   - Bring to a boil, then reduce the heat and let it simmer for about an hour or until the meat is tender.\n   - Add the carrots and corn cobs, cover, and continue cooking for another 30 minutes.\n   - Season with salt and pepper to taste.\n\n### Grilled Pork Ribs with Roasted Carrots and Corn Salsa\n**Ingredients:**\n- 4 pork ribs\n- 4 large carrots\n- 2 ears of corn\n- 1/4 cup olive oil\n- 2 tablespoons honey\n- 1 tablespoon soy sauce\n- 1 teaspoon ground cumin\n- 1 teaspoon smoked paprika\n- 1/4 cup fresh cilantro, chopped\n- Salt and pepper to taste\n\n**Instructions:**\n1. **Marinating:**\n   - In a bowl, mix together olive oil, honey, soy sauce, cumin, smoked paprika, and salt and pepper.\n   - Place the pork ribs in the marinade, turning to coat evenly. Let it marinate for at least 2 hours, preferably overnight.\n\n2. **Grilling:**\n   - Preheat your grill to medium-high heat.\n   - Remove the ribs from the marinade, letting any excess liquid drip off.\n   - Grill the ribs for about 5-7 minutes per side, or until they reach your desired level of doneness.\n\n3. **Roasting:**\n   - Preheat your oven to 400°F (200°C).\n   - Toss the carrots and corn with olive oil, salt, and pepper. Spread them out on a baking sheet.\n   - Roast in the oven for about 20-25 minutes, or until the vegetables are tender and slightly caramelized.\n\n4. **Assembly:**\n   - Serve the grilled pork ribs hot, accompanied by roasted carrots and corn.\n   - Garnish with fresh cilantro and serve with a side of salsa made from the remaining marinade ingredients.\n\nThese recipes should provide a delicious and hearty meal using the ingredients shown in the images. Enjoy cooking!<｜end▁of▁sentence｜>"
  },
  "started_at": "2024-12-29T15:54:48.259801Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/wksm0fwe7hrmc0cm2cg81fvfbw",
    "cancel": "https://api.replicate.com/v1/predictions/wksm0fwe7hrmc0cm2cg81fvfbw/cancel"
  },
  "version": "8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb"
}

Generated in

2 minutes 17 seconds

chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb

Prediction

Model

ID

00e9h1m96xrmc0cm2cjv7rj8fc

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

1m 46s

Created

5 months ago

Input

text: are the images showing the same animal? describe each of them in detail
top_p: 0.9
image1
image2
temperature: 0.1
max_new_tokens: 2048
repetition_penalty: 1.1

{
  "text": "are the images showing the same animal? describe each of them in detail",
  "top_p": 0.9,
  "image1": "https://replicate.delivery/pbxt/MEHVBzZ5sWQvUBTfaXnVRDSXf4oENRJjh0mr85MPCAR4QYtH/IMG_4702.jpg",
  "image2": "https://replicate.delivery/pbxt/MEHVBHadjDWUh70sDEundgZ5rcFTPdBkUnSceTjam0SZOKi9/pic.jpg",
  "temperature": 0.1,
  "max_new_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
  {
    input: {
      text: "are the images showing the same animal? describe each of them in detail",
      top_p: 0.9,
      image1: "https://replicate.delivery/pbxt/MEHVBzZ5sWQvUBTfaXnVRDSXf4oENRJjh0mr85MPCAR4QYtH/IMG_4702.jpg",
      image2: "https://replicate.delivery/pbxt/MEHVBHadjDWUh70sDEundgZ5rcFTPdBkUnSceTjam0SZOKi9/pic.jpg",
      temperature: 0.1,
      max_new_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    input={
        "text": "are the images showing the same animal? describe each of them in detail",
        "top_p": 0.9,
        "image1": "https://replicate.delivery/pbxt/MEHVBzZ5sWQvUBTfaXnVRDSXf4oENRJjh0mr85MPCAR4QYtH/IMG_4702.jpg",
        "image2": "https://replicate.delivery/pbxt/MEHVBHadjDWUh70sDEundgZ5rcFTPdBkUnSceTjam0SZOKi9/pic.jpg",
        "temperature": 0.1,
        "max_new_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run chenxwh/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "chenxwh/deepseek-vl2:8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb",
    "input": {
      "text": "are the images showing the same animal? describe each of them in detail",
      "top_p": 0.9,
      "image1": "https://replicate.delivery/pbxt/MEHVBzZ5sWQvUBTfaXnVRDSXf4oENRJjh0mr85MPCAR4QYtH/IMG_4702.jpg",
      "image2": "https://replicate.delivery/pbxt/MEHVBHadjDWUh70sDEundgZ5rcFTPdBkUnSceTjam0SZOKi9/pic.jpg",
      "temperature": 0.1,
      "max_new_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

No, the two images are not showing the same animal. The first image features a raccoon with distinctive black "mask" markings around its eyes and a bushy tail. It has a grayish-brown fur coat and is looking directly at the camera. The second image shows a red panda with reddish-brown fur, white markings on its face, and a fluffy tail. It is perched on a branch against a clear blue sky background.<｜end▁of▁sentence｜>

{
  "completed_at": "2024-12-29T16:02:00.965183Z",
  "created_at": "2024-12-29T16:00:14.647000Z",
  "data_removed": false,
  "error": null,
  "id": "00e9h1m96xrmc0cm2cjv7rj8fc",
  "input": {
    "text": "are the images showing the same animal? describe each of them in detail",
    "top_p": 0.9,
    "image1": "https://replicate.delivery/pbxt/MEHVBzZ5sWQvUBTfaXnVRDSXf4oENRJjh0mr85MPCAR4QYtH/IMG_4702.jpg",
    "image2": "https://replicate.delivery/pbxt/MEHVBHadjDWUh70sDEundgZ5rcFTPdBkUnSceTjam0SZOKi9/pic.jpg",
    "temperature": 0.1,
    "max_new_tokens": 2048,
    "repetition_penalty": 1.1
  },
  "logs": "2024-12-29 16:01:40,343 - gradio_logger - INFO - ================================================================================\n2024-12-29 16:01:40,343 - gradio_logger - INFO - <|User|>: <image>\n<image>\nare the images showing the same animal? describe each of them in detail\n<|Assistant|>:\n[{'role': '<|User|>', 'content': '<image>\\n<image>\\nare the images showing the same animal? describe each of them in detail', 'images': [<PIL.Image.Image image mode=RGB size=707x707 at 0x704779EE2950>, <PIL.Image.Image image mode=RGB size=1573x1050 at 0x704749A768D0>]}, {'role': '<|Assistant|>', 'content': ''}]\nYou're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.",
  "metrics": {
    "predict_time": 20.864705923,
    "total_time": 106.318183
  },
  "output": {
    "img_out": null,
    "text_out": "No, the two images are not showing the same animal. The first image features a raccoon with distinctive black \"mask\" markings around its eyes and a bushy tail. It has a grayish-brown fur coat and is looking directly at the camera. The second image shows a red panda with reddish-brown fur, white markings on its face, and a fluffy tail. It is perched on a branch against a clear blue sky background.<｜end▁of▁sentence｜>"
  },
  "started_at": "2024-12-29T16:01:40.100477Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/00e9h1m96xrmc0cm2cjv7rj8fc",
    "cancel": "https://api.replicate.com/v1/predictions/00e9h1m96xrmc0cm2cjv7rj8fc/cancel"
  },
  "version": "8ea887897e772107ce53f3a7fa4850e78ae88b2b73ff854b4700db9f0d59c7cb"
}

Generated in

20.9 seconds