Prediction

ID

ecz06tf7m5rme0cmypw9te2crm

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

1.3s

Created

3 months ago by @lucataco

Input

image
top_p: 0.9
prompt: Find all the <|ref|>watermelon slices<|/ref|>
temperature: 0.1
max_length_tokens: 2048
repetition_penalty: 1.1

{
  "image": "https://replicate.delivery/pbxt/MTtkULhkuPEYWa83RRWL29jyJlTgpTr9uhY5JUxVVoRLE3Vs/watermelon.png",
  "top_p": 0.9,
  "prompt": "Find all the <|ref|>watermelon slices<|/ref|>",
  "temperature": 0.1,
  "max_length_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
  {
    input: {
      image: "https://replicate.delivery/pbxt/MTtkULhkuPEYWa83RRWL29jyJlTgpTr9uhY5JUxVVoRLE3Vs/watermelon.png",
      top_p: 0.9,
      prompt: "Find all the <|ref|>watermelon slices<|/ref|>",
      temperature: 0.1,
      max_length_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
    input={
        "image": "https://replicate.delivery/pbxt/MTtkULhkuPEYWa83RRWL29jyJlTgpTr9uhY5JUxVVoRLE3Vs/watermelon.png",
        "top_p": 0.9,
        "prompt": "Find all the <|ref|>watermelon slices<|/ref|>",
        "temperature": 0.1,
        "max_length_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
    "input": {
      "image": "https://replicate.delivery/pbxt/MTtkULhkuPEYWa83RRWL29jyJlTgpTr9uhY5JUxVVoRLE3Vs/watermelon.png",
      "top_p": 0.9,
      "prompt": "Find all the <|ref|>watermelon slices<|/ref|>",
      "temperature": 0.1,
      "max_length_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

<|ref|>watermelon slices<|/ref|><|det|>[[159, 327, 408, 615]]<|/det|>

Generated in

1.3 seconds

deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4

Prediction

Model

ID

ebt05v9rt5rmc0cmypwvmwe5d4

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

1.5s

Created

3 months ago

Input

image
top_p: 0.9
prompt: <|ref|>The giraffe at the back.<|/ref|>
temperature: 0.1
max_length_tokens: 2048
repetition_penalty: 1.1

{
  "image": "https://replicate.delivery/pbxt/MTtkoHqGPiRHcMS9hSxkHSnY02EBnSNLwuR0WZuxBo322l4v/giraffe.png",
  "top_p": 0.9,
  "prompt": "<|ref|>The giraffe at the back.<|/ref|>",
  "temperature": 0.1,
  "max_length_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
  {
    input: {
      image: "https://replicate.delivery/pbxt/MTtkoHqGPiRHcMS9hSxkHSnY02EBnSNLwuR0WZuxBo322l4v/giraffe.png",
      top_p: 0.9,
      prompt: "<|ref|>The giraffe at the back.<|/ref|>",
      temperature: 0.1,
      max_length_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
    input={
        "image": "https://replicate.delivery/pbxt/MTtkoHqGPiRHcMS9hSxkHSnY02EBnSNLwuR0WZuxBo322l4v/giraffe.png",
        "top_p": 0.9,
        "prompt": "<|ref|>The giraffe at the back.<|/ref|>",
        "temperature": 0.1,
        "max_length_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
    "input": {
      "image": "https://replicate.delivery/pbxt/MTtkoHqGPiRHcMS9hSxkHSnY02EBnSNLwuR0WZuxBo322l4v/giraffe.png",
      "top_p": 0.9,
      "prompt": "<|ref|>The giraffe at the back.<|/ref|>",
      "temperature": 0.1,
      "max_length_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

<|ref|>The giraffe at the back.<|/ref|><|det|>[[104, 289, 357, 613]]<|/det|>

Generated in

1.5 seconds

deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4

Prediction

Model

ID

1h2sgpbrr9rm80cmypwvz1efd8

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

1.9s

Created

3 months ago

Input

image
top_p: 0.9
prompt: <|grounding|>I want to throw out the trash now, what should I do?
temperature: 0.1
max_length_tokens: 2048
repetition_penalty: 1.1

{
  "image": "https://replicate.delivery/pbxt/MTtl5BDPSz6kkcvuz8ZnHkI3Ix16KEe1Jrgodgpgwz5tEeNZ/trash.png",
  "top_p": 0.9,
  "prompt": "<|grounding|>I want to throw out the trash now, what should I do?",
  "temperature": 0.1,
  "max_length_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
  {
    input: {
      image: "https://replicate.delivery/pbxt/MTtl5BDPSz6kkcvuz8ZnHkI3Ix16KEe1Jrgodgpgwz5tEeNZ/trash.png",
      top_p: 0.9,
      prompt: "<|grounding|>I want to throw out the trash now, what should I do?",
      temperature: 0.1,
      max_length_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
    input={
        "image": "https://replicate.delivery/pbxt/MTtl5BDPSz6kkcvuz8ZnHkI3Ix16KEe1Jrgodgpgwz5tEeNZ/trash.png",
        "top_p": 0.9,
        "prompt": "<|grounding|>I want to throw out the trash now, what should I do?",
        "temperature": 0.1,
        "max_length_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
    "input": {
      "image": "https://replicate.delivery/pbxt/MTtl5BDPSz6kkcvuz8ZnHkI3Ix16KEe1Jrgodgpgwz5tEeNZ/trash.png",
      "top_p": 0.9,
      "prompt": "<|grounding|>I want to throw out the trash now, what should I do?",
      "temperature": 0.1,
      "max_length_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Well, you can just take it out of your house and place it in the <|ref|>garbage<|/ref|><|det|>[[135, 207, 869, 945]]<|/det|> .

Generated in

1.9 seconds

deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4

Prediction

Model

ID

9g5fzk28qdrme0cmypz9z43sfc

Status

Succeeded

Source

Web

Hardware

L40S

Total duration

1m 46s

Created

3 months ago

Input

image
top_p: 0.9
prompt: Describe each stage of this <image> in detail
temperature: 0.1
max_length_tokens: 2048
repetition_penalty: 1.1

{
  "image": "https://replicate.delivery/pbxt/MTtq4AbrRWL05upjmLYEI1JNyjVYYZv7CDuZ0PgzDtfMegYO/workflow.png",
  "top_p": 0.9,
  "prompt": "Describe each stage of this <image> in detail",
  "temperature": 0.1,
  "max_length_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
  {
    input: {
      image: "https://replicate.delivery/pbxt/MTtq4AbrRWL05upjmLYEI1JNyjVYYZv7CDuZ0PgzDtfMegYO/workflow.png",
      top_p: 0.9,
      prompt: "Describe each stage of this <image> in detail",
      temperature: 0.1,
      max_length_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
    input={
        "image": "https://replicate.delivery/pbxt/MTtq4AbrRWL05upjmLYEI1JNyjVYYZv7CDuZ0PgzDtfMegYO/workflow.png",
        "top_p": 0.9,
        "prompt": "Describe each stage of this <image> in detail",
        "temperature": 0.1,
        "max_length_tokens": 2048,
        "repetition_penalty": 1.1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Run deepseek-ai/deepseek-vl2-small using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "deepseek-ai/deepseek-vl2-small:965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4",
    "input": {
      "image": "https://replicate.delivery/pbxt/MTtq4AbrRWL05upjmLYEI1JNyjVYYZv7CDuZ0PgzDtfMegYO/workflow.png",
      "top_p": 0.9,
      "prompt": "Describe each stage of this <image> in detail",
      "temperature": 0.1,
      "max_length_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

The diagram illustrates a three-stage process for training and fine-tuning a model using a combination of vision-language (VL) adaptation, pre-training, and supervised finetuning. Here's a detailed breakdown of each stage: ### Stage 1: Training VL Adaptor - **Components Involved:** - **DeepSeek LLM:** This is the large language model that will be adapted to work with visual data. - **Vision-Language Adaptor:** A module designed to bridge the gap between visual inputs and the language model. - **Hybrid Vision Encoder:** This encoder processes visual information from images and text pairs. It consists of two components: - **SAM-B:** Likely a component related to spatial attention mechanisms. - **SigLIP-L:** Possibly a component related to signal processing or lip reading. - **Process:** - The hybrid vision encoder takes image-text pairs as input. - The vision-language adaptor then uses these processed visual features to interact with the DeepSeek LLM. ### Stage 2: Joint VL Pre-training - **Components Involved:** - Same as Stage 1, but now the system is being trained on interleaved VL + pure language sequences. - **Process:** - The hybrid vision encoder continues to process visual information. - The vision-language adaptor integrates these visual features with the DeepSeek LLM. - The model is pre-trained on a dataset that includes both visual-linguistic and purely linguistic sequences. ### Stage 3: Supervised Finetuning - **Components Involved:** - Same as Stage 1 and Stage 2, but now the system is being fine-tuned on VL chat data + pure language chat data. - **Process:** - The hybrid vision encoder processes the new dataset. - The vision-language adaptor combines the visual features with the DeepSeek LLM. - The model undergoes supervised finetuning to improve its performance on specific tasks, such as understanding and generating responses in VL chat contexts. ### Summary The overall process involves: 1. **Training the VL Adaptor:** Using image-text pairs to train the vision-language adaptor and hybrid vision encoder. 2. **Joint Pre-training:** Integrating visual and linguistic information to pre-train the model on a mixed dataset. 3. **Supervised Finetuning:** Fine-tuning the model on specialized VL chat data to enhance its capabilities in handling conversational tasks.

{
  "completed_at": "2025-02-11T16:03:02.398453Z",
  "created_at": "2025-02-11T16:01:15.963000Z",
  "data_removed": false,
  "error": null,
  "id": "9g5fzk28qdrme0cmypz9z43sfc",
  "input": {
    "image": "https://replicate.delivery/pbxt/MTtq4AbrRWL05upjmLYEI1JNyjVYYZv7CDuZ0PgzDtfMegYO/workflow.png",
    "top_p": 0.9,
    "prompt": "Describe each stage of this <image> in detail",
    "temperature": 0.1,
    "max_length_tokens": 2048,
    "repetition_penalty": 1.1
  },
  "logs": "You're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.",
  "metrics": {
    "predict_time": 20.591099016,
    "total_time": 106.435453
  },
  "output": "The diagram illustrates a three-stage process for training and fine-tuning a model using a combination of vision-language (VL) adaptation, pre-training, and supervised finetuning. Here's a detailed breakdown of each stage:\n\n### Stage 1: Training VL Adaptor\n- **Components Involved:**\n  - **DeepSeek LLM:** This is the large language model that will be adapted to work with visual data.\n  - **Vision-Language Adaptor:** A module designed to bridge the gap between visual inputs and the language model.\n  - **Hybrid Vision Encoder:** This encoder processes visual information from images and text pairs. It consists of two components:\n    - **SAM-B:** Likely a component related to spatial attention mechanisms.\n    - **SigLIP-L:** Possibly a component related to signal processing or lip reading.\n\n- **Process:**\n  - The hybrid vision encoder takes image-text pairs as input.\n  - The vision-language adaptor then uses these processed visual features to interact with the DeepSeek LLM.\n\n### Stage 2: Joint VL Pre-training\n- **Components Involved:**\n  - Same as Stage 1, but now the system is being trained on interleaved VL + pure language sequences.\n\n- **Process:**\n  - The hybrid vision encoder continues to process visual information.\n  - The vision-language adaptor integrates these visual features with the DeepSeek LLM.\n  - The model is pre-trained on a dataset that includes both visual-linguistic and purely linguistic sequences.\n\n### Stage 3: Supervised Finetuning\n- **Components Involved:**\n  - Same as Stage 1 and Stage 2, but now the system is being fine-tuned on VL chat data + pure language chat data.\n\n- **Process:**\n  - The hybrid vision encoder processes the new dataset.\n  - The vision-language adaptor combines the visual features with the DeepSeek LLM.\n  - The model undergoes supervised finetuning to improve its performance on specific tasks, such as understanding and generating responses in VL chat contexts.\n\n### Summary\nThe overall process involves:\n1. **Training the VL Adaptor:** Using image-text pairs to train the vision-language adaptor and hybrid vision encoder.\n2. **Joint Pre-training:** Integrating visual and linguistic information to pre-train the model on a mixed dataset.\n3. **Supervised Finetuning:** Fine-tuning the model on specialized VL chat data to enhance its capabilities in handling conversational tasks.",
  "started_at": "2025-02-11T16:02:41.807354Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/9g5fzk28qdrme0cmypz9z43sfc",
    "cancel": "https://api.replicate.com/v1/predictions/9g5fzk28qdrme0cmypz9z43sfc/cancel"
  },
  "version": "965f877546f4f0e3f5ae59847f27079d66da81fff2d87d753d5495e862dcb3a4"
}

Generated in

20.6 seconds