dsv3.cuda

Fast inference for DeepSeek-V3 LLMs written in CUDA.

Quick Start

Requirements

1. git lfs -- weight installation
2. bazel -- build tooling

Steps

1. Install Weights

# Weights is a symbolic link so you will have to change it
cd weights

# Clone weights from huggingface
git clone https://huggingface.co/deepseek-ai/DeepSeek-V3

2. Build Inference Engine

bazel build

3. Run

Resources

• DeepSeek-V3
• Hugging Face Weights