Qwen2-0.5B-Instruct on Replicate

This Replicate model provides access to the Qwen2-0.5B-Instruct model, part of the Qwen2 language model series. It offers three variants:

• Qwen/Qwen2-0.5B-Instruct: Full precision model
• Qwen/Qwen2-0.5B-Instruct-GPTQ-Int8: 8-bit quantized model
• Qwen/Qwen2-0.5B-Instruct-GPTQ-Int4: 4-bit quantized model

Introduction

Qwen2 is the latest series of Qwen large language models, offering both pretrained and instruction-tuned models in five sizes: 0.5B, 1.5B, 7B, 57B-A14B, and 72B. This Replicate implementation focuses on the instruction-tuned 0.5B Qwen2 model.

Qwen2 demonstrates competitive performance against state-of-the-art open-source and proprietary models across various benchmarks, including language understanding, generation, multilingual capability, coding, mathematics, and reasoning.

For more details about Qwen2, visit:

• Qwen Blog
• Qwen GitHub Repository
• Qwen Documentation

Model Details

Qwen2 is based on the Transformer architecture and incorporates: - SwiGLU activation - Attention QKV bias - Group query attention - Improved tokenizer for multiple natural languages and code

Training Details

The model underwent pretraining with a large dataset, followed by post-training using both supervised fine-tuning and direct preference optimization.

Quickstart

To use this Replicate implementation:

1. Visit the Replicate model page.

2. Use the web interface or API to run a prediction with your desired parameters.

For local testing or development:

1. Clone the repository: git clone -b Qwen2-0.5B-Instruct https://github.com/zsxkib/cog-qwen-2.git cd cog-qwen-2

2. Run a prediction using Cog: cog predict \ -i 'top_k=1' \ -i 'top_p=1' \ -i 'prompt="Tell me a funny joke about cowboys in the style of Yoda from star wars"' \ -i 'model_type="Qwen2-0.5B-Instruct"' \ -i 'temperature=1' \ -i 'system_prompt="You are a funny and helpful assistant."' \ -i 'max_new_tokens=512' \ -i 'repetition_penalty=1'

Evaluation

Performance comparison between Qwen2-0.5B-Instruct and Qwen1.5-0.5B-Chat:

Citation

If you find the Qwen2 model helpful in your work, please cite:

@article{qwen2,
  title={Qwen2 Technical Report},
  year={2024}
}

License

The Qwen2 model is licensed under the Apache 2.0 License.

Credits and Support

• The Qwen2 model was developed by the Qwen team.
• This Replicate implementation was created by @zsakib_.
• For issues related to the Replicate implementation, please use the GitHub issue tracker.
• For questions about the underlying Qwen2 model, refer to the official Qwen repository.