deepseek-ai / deepseek-coder-v2-lite-instruct

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

Cold

Public
426 runs
L40S
GitHub
Paper
License

Run with an API

Playground API Examples README Versions

Input

prompt

string

Shift + Return to add a new line

Prompt

Default: ""

system_prompt

string

Shift + Return to add a new line

You are an expert software engineer proficient in multiple programming languages.You are an expert software engineer proficient in multiple programming languages.

System prompt to send to the model. This is prepended to the prompt and helps guide system behavior. Ignored for non-chat models.

Default: "You are an expert software engineer proficient in multiple programming languages."

min_tokens

integer

The minimum number of tokens the model should generate as output.

Default: 0

max_tokens

integer

The maximum number of tokens the model should generate as output.

Default: 512

temperature

number

The value used to modulate the next token probabilities.

Default: 0.6

top_p

number

A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).

Default: 0.9

top_k

integer

The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).

Default: 50

presence_penalty

number

Presence penalty

Default: 0

frequency_penalty

number

Frequency penalty

Default: 0

stop_sequences

string

Shift + Return to add a new line

A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.

Run this model in Node.js with one line of code:

npx create-replicate --model=deepseek-ai/deepseek-coder-v2-lite-instruct

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run deepseek-ai/deepseek-coder-v2-lite-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "deepseek-ai/deepseek-coder-v2-lite-instruct:a182159595ee209e25e673a5ab6f0293111dd6733df973f77efcd1f851123330",
  {
    input: {
      top_k: 50,
      top_p: 0.9,
      prompt: "write a quick sort algorithm in python.",
      max_tokens: 512,
      min_tokens: 0,
      temperature: 0.6,
      system_prompt: "You are an expert software engineer proficient in multiple programming languages.",
      presence_penalty: 0,
      frequency_penalty: 0
    }
  }
);

console.log(output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run deepseek-ai/deepseek-coder-v2-lite-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "deepseek-ai/deepseek-coder-v2-lite-instruct:a182159595ee209e25e673a5ab6f0293111dd6733df973f77efcd1f851123330",
    input={
        "top_k": 50,
        "top_p": 0.9,
        "prompt": "write a quick sort algorithm in python.",
        "max_tokens": 512,
        "min_tokens": 0,
        "temperature": 0.6,
        "system_prompt": "You are an expert software engineer proficient in multiple programming languages.",
        "presence_penalty": 0,
        "frequency_penalty": 0
    }
)

# The deepseek-ai/deepseek-coder-v2-lite-instruct model can stream output as it's running.
# The predict method returns an iterator, and you can iterate over that output.
for item in output:
    # https://replicate.com/deepseek-ai/deepseek-coder-v2-lite-instruct/api#output-schema
    print(item, end="")

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run deepseek-ai/deepseek-coder-v2-lite-instruct using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "a182159595ee209e25e673a5ab6f0293111dd6733df973f77efcd1f851123330",
    "input": {
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "write a quick sort algorithm in python.",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "system_prompt": "You are an expert software engineer proficient in multiple programming languages.",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

def quick_sort(arr): if len(arr) <= 1: return arr pivot = arr[len(arr) // 2] left = [x for x in arr if x < pivot] middle = [x for x in arr if x == pivot] right = [x for x in arr if x > pivot] return quick_sort(left) + middle + quick_sort(right) print(quick_sort([3,6,8,10,1,2,1]))

{
  "completed_at": "2024-07-08T19:19:24.930258Z",
  "created_at": "2024-07-08T19:19:23.259000Z",
  "data_removed": false,
  "error": null,
  "id": "ha35gkakfdrgm0cgjf4rm0gfm8",
  "input": {
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "write a quick sort algorithm in python.",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "system_prompt": "You are an expert software engineer proficient in multiple programming languages.",
    "presence_penalty": 0,
    "frequency_penalty": 0
  },
  "logs": "INFO 07-08 19:19:23 async_llm_engine.py:584] Received request 3bea0322126645299a822ea89c4e4778: prompt: 'write a quick sort algorithm in python.', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=50, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[100001], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.\n stdoutINFO 07-08 19:19:23 metrics.py:341] Avg prompt throughput: 1.5 tokens/s, Avg generation throughput: 21.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.\n stdoutGeneration took 1720466049.90sINFO 07-08 19:19:24 async_llm_engine.py:134] Finished request 3bea0322126645299a822ea89c4e4778.\n stdoutFormatted prompt: {system_prompt}\n\nUser: {prompt}\n\nAssistant:",
  "metrics": {
    "predict_time": 1.637787113,
    "total_time": 1.671258
  },
  "output": [
    "\n",
    "def",
    " quick",
    "_",
    "sort",
    "(",
    "arr",
    "):",
    "\n",
    "   ",
    " if",
    " len",
    "(",
    "arr",
    ")",
    " <=",
    " ",
    "1",
    ":",
    "\n",
    "       ",
    " return",
    " arr",
    "\n",
    "   ",
    " pivot",
    " =",
    " arr",
    "[",
    "len",
    "(",
    "arr",
    ")",
    " //",
    " ",
    "2",
    "]",
    "\n",
    "   ",
    " left",
    " =",
    " [",
    "x",
    " for",
    " x",
    " in",
    " arr",
    " if",
    " x",
    " <",
    " pivot",
    "]",
    "\n",
    "   ",
    " middle",
    " =",
    " [",
    "x",
    " for",
    " x",
    " in",
    " arr",
    " if",
    " x",
    " ==",
    " pivot",
    "]",
    "\n",
    "   ",
    " right",
    " =",
    " [",
    "x",
    " for",
    " x",
    " in",
    " arr",
    " if",
    " x",
    " >",
    " pivot",
    "]",
    "\n",
    "   ",
    " return",
    " quick",
    "_",
    "sort",
    "(",
    "left",
    ")",
    " +",
    " middle",
    " +",
    " quick",
    "_",
    "sort",
    "(",
    "right",
    ")",
    "\n",
    "\n",
    "print",
    "(",
    "quick",
    "_",
    "sort",
    "([",
    "3",
    ",",
    "6",
    ",",
    "8",
    ",",
    "1",
    "0",
    ",",
    "1",
    ",",
    "2",
    ",",
    "1",
    "]))",
    "\n",
    ""
  ],
  "started_at": "2024-07-08T19:19:23.292470Z",
  "status": "succeeded",
  "urls": {
    "stream": "https://streaming-api.svc.us.c.replicate.net/v1/streams/rp3y6oobige4rodjk2vl4uz5q5undz3tagqvqjwcph5jmlp7cyha",
    "get": "https://api.replicate.com/v1/predictions/ha35gkakfdrgm0cgjf4rm0gfm8",
    "cancel": "https://api.replicate.com/v1/predictions/ha35gkakfdrgm0cgjf4rm0gfm8/cancel"
  },
  "version": "a182159595ee209e25e673a5ab6f0293111dd6733df973f77efcd1f851123330"
}

Generated in

1.6 seconds

Tweak it Share Report

Run time and cost

This model costs approximately $0.0073 to run on Replicate, or 136 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia L40S GPU hardware. Predictions typically complete within 8 seconds. The predict time for this model varies significantly based on the inputs.

Readme

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

1. Introduction

We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks. Compared to DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in various aspects of code-related tasks, as well as reasoning and general capabilities. Additionally, DeepSeek-Coder-V2 expands its support for programming languages from 86 to 338, while extending the context length from 16K to 128K.

In standard benchmark evaluations, DeepSeek-Coder-V2 achieves superior performance compared to closed-source models such as GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks. The list of supported programming languages can be found here.

2. Model Downloads

We release the DeepSeek-Coder-V2 with 16B and 236B parameters based on the DeepSeekMoE framework, which has actived parameters of only 2.4B and 21B , including base and instruct models, to the public.

<div align="center"> | **Model** | **#Total Params** | **#Active Params** | **Context Length** | **Download** | | :-----------------------------: | :---------------: | :----------------: | :----------------: | :----------------------------------------------------------: | | DeepSeek-Coder-V2-Lite-Base | 16B | 2.4B | 128k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Base) | | DeepSeek-Coder-V2-Lite-Instruct | 16B | 2.4B | 128k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct) | | DeepSeek-Coder-V2-Base | 236B | 21B | 128k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Base) | | DeepSeek-Coder-V2-Instruct | 236B | 21B | 128k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct) | </div>

3. Chat Website

You can chat with the DeepSeek-Coder-V2 on DeepSeek’s official website: coder.deepseek.com

4. API Platform

We also provide OpenAI-Compatible API at DeepSeek Platform: platform.deepseek.com, and you can also pay-as-you-go at an unbeatable price.

5. License

This code repository is licensed under the MIT License. The use of DeepSeek-Coder-V2 Base/Instruct models is subject to the Model License. DeepSeek-Coder-V2 series (including Base and Instruct) supports commercial use.

6. Contact

If you have any questions, please raise an issue or contact us at service@deepseek.com.