Skip to content

Conversation

simon-mo
Copy link
Collaborator

Summary

  • inline the LoRA model manager's adapter bookkeeping so it no longer relies on generic helper wrappers
  • simplify the worker-side LoRA manager to operate directly on the requested adapter set when applying changes

Testing

  • pytest tests/lora/test_lora_manager.py (fails: ModuleNotFoundError: No module named 'tblib')
  • python -m compileall vllm/lora/models.py vllm/lora/worker_manager.py vllm/lora/utils.py

https://chatgpt.com/codex/tasks/task_e_68ca49ea72a08329a069e9249f17eb78

@simon-mo simon-mo requested a review from jeejeelee as a code owner September 17, 2025 06:27
@simon-mo simon-mo changed the title Simplify LoRA manager helpers cleanup: remove adapter commons Sep 17, 2025
@simon-mo simon-mo requested a review from WoosukKwon September 17, 2025 06:28
@simon-mo
Copy link
Collaborator Author

We no longer need this abstraction after deleting prompt adapters.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a solid refactoring that simplifies the LoRA management logic by removing the generic adapter_commons helper layer and inlining its functionality directly into LoRAModelManager and WorkerLoRAManager. This change significantly improves code clarity and maintainability by removing a layer of abstraction that was specific to LoRA anyway. The logic from the helper functions has been correctly preserved in the new inlined implementations, and the code is now more straightforward and easier to follow. I've reviewed the changes and found no issues; this is a great improvement.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex Review: Here are some suggestions.

Reply with @codex fix comments to fix any unresolved comments.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

Comment on lines 160 to +164
def set_active_adapters(self, requests: set[Any],
mapping: Optional[Any]) -> None:
set_active_adapters_worker(requests, mapping, self._apply_adapters,
self._adapter_manager.set_adapter_mapping)
self._apply_adapters(requests)
if mapping is not None:
self._adapter_manager.set_adapter_mapping(mapping)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Failing to clear adapter mapping when disabled

The new worker manager now calls _adapter_manager.set_adapter_mapping only when mapping is not None. Previously the helper always invoked the setter even for None, which resets any stale mapping in the model. With the current logic, calling set_active_adapters(requests, None) leaves whatever mapping was previously set, so a request that intends to disable or clear LoRA mappings will continue to use the old mapping and potentially apply outdated adapters. This can cause incorrect adapter application across requests or after removing adapters. Consider always delegating the call, including when mapping is None, so the manager can reset its internal state.

Useful? React with 👍 / 👎.

@simon-mo
Copy link
Collaborator Author

@codex fix comments

Copy link

Summary

  • Ensured the worker LoRA manager always delegates adapter mapping updates so stale mappings are cleared even when None is provided. vllm/lora/worker_manager.pyL160-L164
  • Normalized None mappings inside LoRAModelManager by converting them to an empty LoRAMapping, allowing the manager to reset its internal state safely. vllm/lora/models.pyL703-L708

Testing

  • python -m compileall vllm/lora/models.py vllm/lora/worker_manager.py

View task →

@jeejeelee jeejeelee added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 17, 2025
Copy link
Collaborator

@jeejeelee jeejeelee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for cleanup, tested on my locall device:

===================================================================== test session starts =====================================================================
platform linux -- Python 3.10.15, pytest-8.3.3, pluggy-1.5.0
rootdir: /root/Code/vllm_main/vllm
configfile: pyproject.toml
plugins: shard-0.1.2, rerunfailures-14.0, mock-3.14.0, buildkite-test-collector-0.1.9, forked-1.6.0, anyio-4.6.2.post1, asyncio-0.24.0
asyncio: mode=strict, default_loop_scope=None
collected 15 items                                                                                                                                            
Running 15 items in this shard

test_lora_manager.py ...............                                                                                                                    [100%]

@simon-mo simon-mo enabled auto-merge (squash) September 17, 2025 16:38
@simon-mo simon-mo merged commit 4aa8c7b into main Sep 17, 2025
49 checks passed
@simon-mo simon-mo deleted the codex/remove-adapter_commons-directory-and-references branch September 17, 2025 16:46
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
charlifu pushed a commit to ROCm/vllm that referenced this pull request Sep 25, 2025
Signed-off-by: Jee Jee Li <[email protected]>
Co-authored-by: Jee Jee Li <[email protected]>
Signed-off-by: charlifu <[email protected]>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
Signed-off-by: Jee Jee Li <[email protected]>
Co-authored-by: Jee Jee Li <[email protected]>
Signed-off-by: xuebwang-amd <[email protected]>
choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants