yorickvp / cog-vllm-build

test image please ignore

  • Public
  • 0 runs
  • A100 (80GB)

Input

Install Cog
brew install cog

If you don’t have Homebrew, there are other installation options available.

Pull and run yorickvp/cog-vllm-build using Cog (this will download the full model and run it in your local environment):

cog predict r8.im/yorickvp/cog-vllm-build@sha256:2110a4e63c0c8d06088696dbda48813635204719185fac492df1e8717d5e1e07 \
  -i 'top_k=50' \
  -i 'top_p=0.9' \
  -i 'prompt=""' \
  -i 'max_tokens=512' \
  -i 'min_tokens=0' \
  -i 'temperature=0.6' \
  -i 'system_prompt="You are a helpful assistant."' \
  -i 'presence_penalty=0' \
  -i 'frequency_penalty=0'

To learn more, take a look at the Cog documentation.

Output

No output yet! Press "Submit" to start a prediction.

Run time and cost

This model runs on Nvidia A100 (80GB) GPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

This model doesn't have a readme.