Prediction

ID

k3ss3jc7pdrj60cmypzsr8za48

Status

Succeeded

Source

Web

Hardware

A100 (80GB)

Total duration

2.0s

Created

7 months ago

Input

image
top_p: 0.9
prompt: <|ref|>Watermelon slices.<|/ref|>
temperature: 0.1
max_length_tokens: 2048
repetition_penalty: 1.1

{
  "image": "https://replicate.delivery/pbxt/MTtrMAC9CsWmzzCXqkPwiix6NHm1mrAMV0egMB9vt3NTPZu0/watermelon.png",
  "top_p": 0.9,
  "prompt": " <|ref|>Watermelon slices.<|/ref|>",
  "temperature": 0.1,
  "max_length_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
  {
    input: {
      image: "https://replicate.delivery/pbxt/MTtrMAC9CsWmzzCXqkPwiix6NHm1mrAMV0egMB9vt3NTPZu0/watermelon.png",
      top_p: 0.9,
      prompt: " <|ref|>Watermelon slices.<|/ref|>",
      temperature: 0.1,
      max_length_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
    input={
        "image": "https://replicate.delivery/pbxt/MTtrMAC9CsWmzzCXqkPwiix6NHm1mrAMV0egMB9vt3NTPZu0/watermelon.png",
        "top_p": 0.9,
        "prompt": " <|ref|>Watermelon slices.<|/ref|>",
        "temperature": 0.1,
        "max_length_tokens": 2048,
        "repetition_penalty": 1.1
    }
)

print(output)

To learn more, take a look at the guide on getting started with Python.

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
    "input": {
      "image": "https://replicate.delivery/pbxt/MTtrMAC9CsWmzzCXqkPwiix6NHm1mrAMV0egMB9vt3NTPZu0/watermelon.png",
      "top_p": 0.9,
      "prompt": " <|ref|>Watermelon slices.<|/ref|>",
      "temperature": 0.1,
      "max_length_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

<|ref|>Watermelon slices.<|/ref|><|det|>[[0, 0, 998, 999]]<|/det|>

Generated in

2.0 seconds

deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200

Prediction

Model

ID

exksbr65d9rj60cmypztam6hnr

Status

Succeeded

Source

Web

Hardware

A100 (80GB)

Total duration

2.3s

Created

7 months ago

Input

image
top_p: 0.9
prompt: <|ref|>The giraffe at the back.<|/ref|>
temperature: 0.1
max_length_tokens: 2048
repetition_penalty: 1.1

{
  "image": "https://replicate.delivery/pbxt/MTtraUkxR3pRH1V04hU6HxqhYRsDMMW9OU7fhwbb9w6i28aV/giraffe.png",
  "top_p": 0.9,
  "prompt": "<|ref|>The giraffe at the back.<|/ref|>",
  "temperature": 0.1,
  "max_length_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
  {
    input: {
      image: "https://replicate.delivery/pbxt/MTtraUkxR3pRH1V04hU6HxqhYRsDMMW9OU7fhwbb9w6i28aV/giraffe.png",
      top_p: 0.9,
      prompt: "<|ref|>The giraffe at the back.<|/ref|>",
      temperature: 0.1,
      max_length_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
    input={
        "image": "https://replicate.delivery/pbxt/MTtraUkxR3pRH1V04hU6HxqhYRsDMMW9OU7fhwbb9w6i28aV/giraffe.png",
        "top_p": 0.9,
        "prompt": "<|ref|>The giraffe at the back.<|/ref|>",
        "temperature": 0.1,
        "max_length_tokens": 2048,
        "repetition_penalty": 1.1
    }
)

print(output)

To learn more, take a look at the guide on getting started with Python.

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
    "input": {
      "image": "https://replicate.delivery/pbxt/MTtraUkxR3pRH1V04hU6HxqhYRsDMMW9OU7fhwbb9w6i28aV/giraffe.png",
      "top_p": 0.9,
      "prompt": "<|ref|>The giraffe at the back.<|/ref|>",
      "temperature": 0.1,
      "max_length_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

<|ref|>The giraffe at the back.<|/ref|><|det|>[[0, 0, 998, 999]]<|/det|>

Generated in

2.3 seconds

deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200

Prediction

Model

ID

ad7s5gashdrj00cmyq083sgkp8

Status

Succeeded

Source

Web

Hardware

A100 (80GB)

Total duration

40.1s

Created

7 months ago by @lucataco

Input

image
top_p: 0.9
prompt: Describe each stage of this <image> in detail
temperature: 0.1
max_length_tokens: 2048
repetition_penalty: 1.1

{
  "image": "https://replicate.delivery/pbxt/MTtsBStHRqLDgNZMkt0J7PptoJ3lseSUNcGaDkG230ttNJlT/workflow.png",
  "top_p": 0.9,
  "prompt": "Describe each stage of this <image> in detail",
  "temperature": 0.1,
  "max_length_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
  {
    input: {
      image: "https://replicate.delivery/pbxt/MTtsBStHRqLDgNZMkt0J7PptoJ3lseSUNcGaDkG230ttNJlT/workflow.png",
      top_p: 0.9,
      prompt: "Describe each stage of this <image> in detail",
      temperature: 0.1,
      max_length_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
    input={
        "image": "https://replicate.delivery/pbxt/MTtsBStHRqLDgNZMkt0J7PptoJ3lseSUNcGaDkG230ttNJlT/workflow.png",
        "top_p": 0.9,
        "prompt": "Describe each stage of this <image> in detail",
        "temperature": 0.1,
        "max_length_tokens": 2048,
        "repetition_penalty": 1.1
    }
)

print(output)

To learn more, take a look at the guide on getting started with Python.

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
    "input": {
      "image": "https://replicate.delivery/pbxt/MTtsBStHRqLDgNZMkt0J7PptoJ3lseSUNcGaDkG230ttNJlT/workflow.png",
      "top_p": 0.9,
      "prompt": "Describe each stage of this <image> in detail",
      "temperature": 0.1,
      "max_length_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

The figure illustrates a three-stage process for training and fine-tuning a Vision-Language (VL) model using the DeepSeek LLM framework. Here's a detailed description of each stage: ### Stage 1: Training VL Adaptor - **Components Involved**: - **DeepSeek LLM**: This is the foundational language model used throughout the stages. - **Vision-Language Adaptor**: This component is responsible for adapting the language model to handle vision-language tasks. - **Hybrid Vision Encoder**: This encoder processes visual data and combines it with textual information. - **SAM-B and SigLIP-L**: These are specific models or components within the hybrid vision encoder that contribute to processing image-text pairs. - **Process**: - The DeepSeek LLM is used as the base model. - The Vision-Language Adaptor is trained on image-text pairs, which involves aligning visual and textual information. - The Hybrid Vision Encoder processes these image-text pairs, integrating both visual and textual features. ### Stage 2: Joint VL Pre-training - **Components Involved**: - **DeepSeek LLM**: Continues to be the core language model. - **Vision-Language Adaptor**: Further refined through joint pre-training. - **Hybrid Vision Encoder**: Enhanced to better handle interleaved vision and language sequences. - **SAM-B and SigLIP-L**: Continue to play roles in encoding visual and textual data. - **Process**: - The model undergoes joint pre-training using interleaved vision and language sequences. - This step helps the model learn to effectively combine and process both types of data simultaneously. - The Vision-Language Adaptor and Hybrid Vision Encoder are further optimized during this phase. ### Stage 3: Supervised Finetuning - **Components Involved**: - **DeepSeek LLM**: Now fully integrated into the VL system. - **Vision-Language Adaptor**: Fully adapted and ready for specific tasks. - **Hybrid Vision Encoder**: Finalized and capable of handling complex VL tasks. - **SAM-B and SigLIP-L**: Continue their roles in encoding and processing data. - **Process**: - The model is fine-tuned using VL chat data and pure language chat data. - This supervised finetuning phase refines the model's performance on specific VL tasks, such as conversational understanding and generation. - The Vision-Language Adaptor and Hybrid Vision Encoder are fine-tuned to ensure they work seamlessly together for the desired outcomes. Overall, this three-stage process leverages the strengths of the DeepSeek LLM and specialized adaptors to create a robust Vision-Language model capable of handling various tasks involving both visual and textual data.

{
  "completed_at": "2025-02-11T16:04:11.390146Z",
  "created_at": "2025-02-11T16:03:31.339000Z",
  "data_removed": false,
  "error": null,
  "id": "ad7s5gashdrj00cmyq083sgkp8",
  "input": {
    "image": "https://replicate.delivery/pbxt/MTtsBStHRqLDgNZMkt0J7PptoJ3lseSUNcGaDkG230ttNJlT/workflow.png",
    "top_p": 0.9,
    "prompt": "Describe each stage of this <image> in detail",
    "temperature": 0.1,
    "max_length_tokens": 2048,
    "repetition_penalty": 1.1
  },
  "logs": null,
  "metrics": {
    "predict_time": 40.035444169,
    "total_time": 40.051146
  },
  "output": "The figure illustrates a three-stage process for training and fine-tuning a Vision-Language (VL) model using the DeepSeek LLM framework. Here's a detailed description of each stage:\n\n### Stage 1: Training VL Adaptor\n- **Components Involved**: \n  - **DeepSeek LLM**: This is the foundational language model used throughout the stages.\n  - **Vision-Language Adaptor**: This component is responsible for adapting the language model to handle vision-language tasks.\n  - **Hybrid Vision Encoder**: This encoder processes visual data and combines it with textual information.\n  - **SAM-B and SigLIP-L**: These are specific models or components within the hybrid vision encoder that contribute to processing image-text pairs.\n\n- **Process**:\n  - The DeepSeek LLM is used as the base model.\n  - The Vision-Language Adaptor is trained on image-text pairs, which involves aligning visual and textual information.\n  - The Hybrid Vision Encoder processes these image-text pairs, integrating both visual and textual features.\n\n### Stage 2: Joint VL Pre-training\n- **Components Involved**:\n  - **DeepSeek LLM**: Continues to be the core language model.\n  - **Vision-Language Adaptor**: Further refined through joint pre-training.\n  - **Hybrid Vision Encoder**: Enhanced to better handle interleaved vision and language sequences.\n  - **SAM-B and SigLIP-L**: Continue to play roles in encoding visual and textual data.\n\n- **Process**:\n  - The model undergoes joint pre-training using interleaved vision and language sequences.\n  - This step helps the model learn to effectively combine and process both types of data simultaneously.\n  - The Vision-Language Adaptor and Hybrid Vision Encoder are further optimized during this phase.\n\n### Stage 3: Supervised Finetuning\n- **Components Involved**:\n  - **DeepSeek LLM**: Now fully integrated into the VL system.\n  - **Vision-Language Adaptor**: Fully adapted and ready for specific tasks.\n  - **Hybrid Vision Encoder**: Finalized and capable of handling complex VL tasks.\n  - **SAM-B and SigLIP-L**: Continue their roles in encoding and processing data.\n\n- **Process**:\n  - The model is fine-tuned using VL chat data and pure language chat data.\n  - This supervised finetuning phase refines the model's performance on specific VL tasks, such as conversational understanding and generation.\n  - The Vision-Language Adaptor and Hybrid Vision Encoder are fine-tuned to ensure they work seamlessly together for the desired outcomes.\n\nOverall, this three-stage process leverages the strengths of the DeepSeek LLM and specialized adaptors to create a robust Vision-Language model capable of handling various tasks involving both visual and textual data.",
  "started_at": "2025-02-11T16:03:31.354702Z",
  "status": "succeeded",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/ad7s5gashdrj00cmyq083sgkp8",
    "cancel": "https://api.replicate.com/v1/predictions/ad7s5gashdrj00cmyq083sgkp8/cancel"
  },
  "version": "e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200"
}

Generated in

40.0 seconds

deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200

Prediction

Model

ID

0rj52k0asxrj00cmyq095jsbr8

Status

Succeeded

Source

Web

Hardware

A100 (80GB)

Total duration

4.4s

Created

7 months ago

Input

image
top_p: 0.9
prompt: <|grounding|>I want to throw out the trash now, what should I do?
temperature: 0.1
max_length_tokens: 2048
repetition_penalty: 1.1

{
  "image": "https://replicate.delivery/pbxt/MTtrs2poZLXitl4i6GOfEAQ00BuA26T71pJsYSxgP3r6cNCs/trash.png",
  "top_p": 0.9,
  "prompt": "<|grounding|>I want to throw out the trash now, what should I do?",
  "temperature": 0.1,
  "max_length_tokens": 2048,
  "repetition_penalty": 1.1
}

npm install replicate

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
  {
    input: {
      image: "https://replicate.delivery/pbxt/MTtrs2poZLXitl4i6GOfEAQ00BuA26T71pJsYSxgP3r6cNCs/trash.png",
      top_p: 0.9,
      prompt: "<|grounding|>I want to throw out the trash now, what should I do?",
      temperature: 0.1,
      max_length_tokens: 2048,
      repetition_penalty: 1.1
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

pip install replicate

Import the client:

import replicate

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
    input={
        "image": "https://replicate.delivery/pbxt/MTtrs2poZLXitl4i6GOfEAQ00BuA26T71pJsYSxgP3r6cNCs/trash.png",
        "top_p": 0.9,
        "prompt": "<|grounding|>I want to throw out the trash now, what should I do?",
        "temperature": 0.1,
        "max_length_tokens": 2048,
        "repetition_penalty": 1.1
    }
)

print(output)

To learn more, take a look at the guide on getting started with Python.

Run deepseek-ai/deepseek-vl2 using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "deepseek-ai/deepseek-vl2:e5caf557dd9e5dcee46442e1315291ef1867f027991ede8ff95e304d4f734200",
    "input": {
      "image": "https://replicate.delivery/pbxt/MTtrs2poZLXitl4i6GOfEAQ00BuA26T71pJsYSxgP3r6cNCs/trash.png",
      "top_p": 0.9,
      "prompt": "<|grounding|>I want to throw out the trash now, what should I do?",
      "temperature": 0.1,
      "max_length_tokens": 2048,
      "repetition_penalty": 1.1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

You can simply take <|ref|>the garbage<|/ref|><|det|>[[0, 0, 995, 993]]<|/det|> and put it in <|ref|>the trashcan<|/ref|><|det|>[[0, 0, 995, 993]]<|/det|> . It's right there for you to dispose of.

Generated in

4.4 seconds