Skip to content

llm-d/llm-d-pd-utils

Repository files navigation

PD utils

Build NIXL Image

docker build -t nixl:latest .
docker tag nixl:latest ghcr.io/<>/nixl:latest
docker push ghcr.io/<>/nixl:latest

Deploy & Test

cd benchmarks
./benchmark_deployment.sh nixl -rdma

Use -rdma for the deployment in cluster with RoCE enabled for performance. This will deploy nixl-client-roce and nixl-server-roce, and run a benchmarking script to measure the transfer throughput (GB/s). Refer to benchmarking for more details.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors 4

  •  
  •  
  •  
  •