-
-
Notifications
You must be signed in to change notification settings - Fork 10.5k
[SupportsQuant] Chameleon, Chatglm, Commandr #15952
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SupportsQuant] Chameleon, Chatglm, Commandr #15952
Conversation
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
This pull request has merge conflicts that must be resolved before it can be |
009eaba
to
a79cc68
Compare
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]> Signed-off-by: xinyuxiao <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]> Signed-off-by: Mu Huai <[email protected]>
Purpose
SupportsQuant
mixin to models in order to uniformity support quantization across all modelspacked_modules_mapping
is correctly updated across nested modelsignored modules
are correctly updated according tohf_to_vllm_mapper
and across nested modelsRelated Issues
Changes
SupportsQuant
andpacked_modules_mapping
attribute to Chameleon, Chatglm, and Commandr