[Doc]: Add Kubernetes (K8s) Deployment Documentation for vLLM with GPUs #8438
Description
📚 The doc issue
Context:
I am currently managing H100 GPUs using Kubernetes (K8s). However, I’ve noticed that the vLLM documentation only provides deployment instructions for Docker, which is quite different from K8s. This creates a gap for users like me who rely on K8s for managing our infrastructure.
Issue:
Lack of Kubernetes (K8s) deployment documentation for vLLM.
Existing documentation focuses solely on Docker, leaving out crucial information for K8s users.
The differences between Docker and K8s make it challenging for users to adapt the Docker-based instructions for a K8s environment.
Proposal:
I suggest expanding the current vLLM documentation to include detailed instructions for deploying vLLM in a Kubernetes (K8s) environment. This should cover:
Setting up the environment to manage GPUs using K8s.
Step-by-step deployment instructions tailored for K8s.
Configuration and optimization tips specific to K8s.
Additional Details:
As a K8s user managing H100 GPUs, I am willing to contribute by sharing more details on how to handle these tasks. This could include:
Sample YAML files for deployment.
Configuration tips and best practices.
Troubleshooting common issues encountered during deployment in K8s.
Suggest a potential alternative/fix
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.