-
-
Notifications
You must be signed in to change notification settings - Fork 10.6k
Closed as not planned
Labels
documentationImprovements or additions to documentationImprovements or additions to documentationstaleOver 90 days of inactivityOver 90 days of inactivity
Description
📚 The doc issue
i want to know Why NGramWorker does not support cache operations. this code https://github.com/vllm-project/vllm/blob/main/vllm/spec_decode/ngram_worker.py#L155 . this pr (#8824) seems can support cache operations when run in NGramWorker .
Suggest a potential alternative/fix
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
Metadata
Metadata
Assignees
Labels
documentationImprovements or additions to documentationImprovements or additions to documentationstaleOver 90 days of inactivityOver 90 days of inactivity