[Examples] Create qwen_2_5_vl_example.py

[Zhao-Dongyu]

SUMMARY:
Add Qwen2.5-VL support for W8A8 FP8 quantization example
This PR adds support for Qwen2.5-VL model in the W8A8 FP8 quantization examples. While Qwen2-VL is still available, Qwen2.5-VL has gained more popularity among users. This addition provides users with an example for FP8 quantization of Qwen's newer vision-language model alongside the existing Qwen2-VL support.

TEST PLAN:
This is a low-risk change that simply adapts the existing Qwen2-VL example for Qwen2.5-VL:
Simple Adaptation: Only changed the model ID from Qwen2-VL to Qwen2.5-VL - no structural changes to the quantization logic
Proven Feasibility: The code has been tested and verified to work correctly with Qwen2.5-VL
Consistency: Follows the exact same pattern as other Qwen2.5-VL examples already in the repository
No Risk: This is a straightforward model ID update that maintains all existing functionality

[gemini-code-assist]

Summary of Changes

Hello @Zhao-Dongyu, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new example for performing W8A8 FP8 quantization on the Qwen2.5-VL model. It provides a dedicated script, qwen_2_5_vl_example.py, demonstrating how to apply dynamic FP8 quantization to the Qwen2.5-VL-7B-Instruct model using the llmcompressor library, thereby expanding the available quantization examples to include this popular, newer vision-language model.

Highlights

• New Example for Qwen2.5-VL Quantization: A new Python script, qwen_2_5_vl_example.py, has been added to demonstrate W8A8 FP8 quantization for the Qwen2.5-VL-7B-Instruct model.
• FP8 Dynamic Quantization: The example configures a QuantizationModifier to apply FP8_DYNAMIC scheme targeting Linear layers, while ignoring lm_head and visual layers.
• Model Loading and Processing: The script loads the Qwen2.5-VLForConditionalGeneration model and its corresponding processor from Hugging Face.
• One-Shot Quantization and Saving: It utilizes the llmcompressor.oneshot function to apply the defined quantization recipe and then saves the quantized model and processor in compressed-tensors format.
• Sample Generation Verification: A sample text generation is included to verify the functionality and sanity of the quantized model.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds a new example for quantizing Qwen2.5-VL, which is a valuable addition. The code is clear and follows the existing structure. My main feedback is to improve device handling for better robustness and to add a test case for this new example to ensure its continued correctness.

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[kylesayrs]

Thanks for the example!

[rahul-tuli]

Thanks!