vLLM Recipes

This repo intends to host community maintained common recipes to run vLLM answering the question: How do I run model Y on hardware Y for task Z?

Guides

DeepSeek

• DeepSeek-V3, DeepSeek-R1
• DeepSeek-V3.1

GLM

• GLM-4.5, GLM-4.5-Air
• GLM-4.5V

InternLM

• Intern-S1

Llama

• Llama3.3-70B
• Llama4-Scout

OpenAI

• gpt-oss

Qwen

• Qwen2.5-VL
• Qwen3-Coder-480B-A35B

Seed

• Seed-OSS-36B

Contributing

Please feel free to contribute by adding a new recipe or improving an existing one, just send us a PR!

While the repo is designed to be directly viewable in GitHub (Markdown files as first citizen), you can build the docs as web pages locally.

uv venv
source .venv/bin/activate
uv pip install -r requirements.txt
uv run mkdocs serve

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.