[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files #20560

bigPYJ1151 · 2025-07-07T09:39:44Z

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

During the merging of #20412 , there were some conflicts and dropped the commit of #20437 accidently.

This PR resolves the conflicts and re-commits the commit to fix CPU CI tests.

Test Plan

Test Result

(Optional) Documentation Update

Signed-off-by: jiang1.li <[email protected]>

github-actions · 2025-07-07T09:39:52Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

gemini-code-assist

Summary of Changes

Hello @bigPYJ1151, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request primarily focuses on stabilizing and advancing the CPU continuous integration (CI) pipeline by resolving prior merge conflicts and re-integrating critical fixes. It marks a significant step in the transition to the V1 CPU backend by adjusting the test suite to reflect V1 compatibility and limitations, and by consolidating the entire CPU attention backend implementation within a dedicated V1 file. This ensures that CPU-related development and testing are aligned with the new V1 architecture.

Highlights

CPU CI Fixes & Test Adjustments: This PR addresses previous merge conflicts and re-applies a dropped commit to fix CPU CI tests. It updates the CPU test suite by disabling certain tests (e.g., test_cache.py, test_mla_decode_cpu.py, test_bart.py, test_chunked_prefill.py) and modifying others to align with the V1 CPU backend's current capabilities and limitations.
V1 CPU Backend Consolidation: The core CPU attention backend (TorchSDPABackendImpl and its associated TorchSDPAMetadata) has been fully implemented and consolidated within vllm/v1/attention/backends/cpu_attn.py. This includes the detailed logic for PagedAttention operations, with conditional support for Intel Extension for PyTorch (IPEX) for optimized performance.
Transition to V1 for CPU: Several test configurations have been updated to explicitly enforce the use of the V1 CPU backend, marking a clear deprecation of V0 for CPU-specific tests where applicable. This ensures that CPU-related tests are run against the intended V1 architecture.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request fixes CPU CI tests by resolving merge conflicts and re-committing changes. It disables certain tests and introduces a new V1 CPU attention backend. There are a couple of high-severity issues in the new backend related to correctness and performance that should be addressed.

gemini-code-assist · 2025-07-07T09:42:03Z

vllm/v1/attention/backends/cpu_attn.py

    def get_supported_head_sizes(cls) -> list[int]:
-        return PagedAttention.get_supported_head_sizes()
+        if _use_ipex:
+            return []
+        else:
+            return _get_paged_attn_impl()\
+                .get_supported_head_sizes() # type: ignore


When _use_ipex is true, this function returns an empty list, indicating no supported head sizes. However, the TorchSDPABackendImpl constructor still validates against a hardcoded list, potentially leading to a ValueError at runtime if the head size isn't in that list. To avoid this inconsistency, the supported head sizes should be retrieved from _get_paged_attn_impl() even when _use_ipex is true.

Suggested change

def get_supported_head_sizes(cls) -> list[int]:

return PagedAttention.get_supported_head_sizes()

if _use_ipex:

return []

else:

return _get_paged_attn_impl()\

.get_supported_head_sizes() # type: ignore

@classmethod

def get_supported_head_sizes(cls) -> list[int]:

if _use_ipex:

return _get_paged_attn_impl().get_supported_head_sizes()

else:

return _get_paged_attn_impl().get_supported_head_sizes() # type: ignore

gemini-code-assist · 2025-07-07T09:42:03Z

vllm/v1/attention/backends/cpu_attn.py

+        for seq_len_q, seq_len_kv, mask in zip(seq_lens_q, seq_lens_kv,
+                                               attn_masks):
+            end_q = start_q + seq_len_q
+            end_kv = start_kv + seq_len_kv
+            sub_out = scaled_dot_product_attention(
+                query[None, :, start_q:end_q, :],
+                key[None, :, start_kv:end_kv, :],
+                value[None, :, start_kv:end_kv, :],
+                attn_mask=mask,
+                dropout_p=0.0,
+                is_causal=causal_attn and mask is None,
+                scale=self.scale).squeeze(0).movedim(query.dim() - 2, 0)
+            output[start_q:end_q, :, :] = sub_out
+            start_q, start_kv = end_q, end_kv


The loop iterates through sequences one by one, which can be inefficient due to Python loop overhead and tensor slicing. scaled_dot_product_attention is optimized for batched operations. Vectorizing this operation by constructing a single batch-level attention mask and calling scaled_dot_product_attention once can improve performance.

gemini-code-assist · 2025-07-07T09:42:03Z

tests/models/language/pooling/test_reward.py

+    if current_platform.is_cpu() and os.environ.get("VLLM_USE_V1", "0") == "0":
+        pytest.skip("CPU only supports V1")


Consider adding a comment explaining why the test is skipped when current_platform.is_cpu() and os.environ.get("VLLM_USE_V1", "0") == "0".

DarkLight1337 · 2025-07-07T09:45:47Z

vllm/v1/attention/backends/cpu_attn.py

    @classmethod
    def get_supported_head_sizes(cls) -> list[int]:
-        return PagedAttention.get_supported_head_sizes()
+        if _use_ipex:


I suggest merging these two functions together. We only need validate_head_size in V1 - get_supported_head_sizes is just for convenience and only used by validate_head_size

Similarly, the Attention classes in this file can implement validate_head_size instead of get_supported_head_sizes

Signed-off-by: jiang1.li <[email protected]>

vllm/v1/attention/backends/cpu_attn.py

Signed-off-by: jiang1.li <[email protected]>

DarkLight1337 · 2025-07-07T11:31:50Z

Going to wait until the release is out before merging this

mgoin · 2025-07-07T14:41:07Z

.buildkite/scripts/hardware_ci/run-cpu-test.sh

-  docker exec cpu-test-"$NUMA_NODE" bash -c "
-    set -e
-    pytest -s -v -k cpu_model \
-    tests/basic_correctness/test_chunked_prefill.py"  


Can we remove cpu_model from this test?

Signed-off-by: jiang1.li <[email protected]>

…20560) Signed-off-by: jiang1.li <[email protected]>

…20560) Signed-off-by: jiang1.li <[email protected]> Signed-off-by: Jinzhen Lin <[email protected]>

…20560) Signed-off-by: jiang1.li <[email protected]>

bigPYJ1151 added 2 commits July 7, 2025 09:23

refactor cpu attn

15f758c

Signed-off-by: jiang1.li <[email protected]>

fix tests

feafcc7

Signed-off-by: jiang1.li <[email protected]>

bigPYJ1151 marked this pull request as ready for review July 7, 2025 09:40

bigPYJ1151 requested review from DarkLight1337, WoosukKwon, alexm-redhat, comaniac, mgoin, njhill, robertgshaw2-redhat and ywang96 as code owners July 7, 2025 09:40

mergify bot added the ci/build label Jul 7, 2025

gemini-code-assist bot reviewed Jul 7, 2025

View reviewed changes

mergify bot added the v1 label Jul 7, 2025

gemini-code-assist bot reviewed Jul 7, 2025

View reviewed changes

DarkLight1337 reviewed Jul 7, 2025

View reviewed changes

bigPYJ1151 added 2 commits July 7, 2025 09:59

fix comment

831683e

Signed-off-by: jiang1.li <[email protected]>

fix

f9ea54d

Signed-off-by: jiang1.li <[email protected]>

DarkLight1337 reviewed Jul 7, 2025

View reviewed changes

vllm/v1/attention/backends/cpu_attn.py Outdated Show resolved Hide resolved

fix

57df062

Signed-off-by: jiang1.li <[email protected]>

DarkLight1337 approved these changes Jul 7, 2025

View reviewed changes

mgoin reviewed Jul 7, 2025

View reviewed changes

fix comment

5a9407d

Signed-off-by: jiang1.li <[email protected]>

vllm-bot merged commit 7721ef1 into vllm-project:main Jul 8, 2025
14 checks passed

Pradyun92 pushed a commit to Pradyun92/vllm that referenced this pull request Aug 6, 2025

[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files (vllm-project#…

ae2d73a

…20560) Signed-off-by: jiang1.li <[email protected]>

npanpaliya pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Aug 6, 2025

[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files (vllm-project#…

96cdcfe

…20560) Signed-off-by: jiang1.li <[email protected]>

jinzhen-lin pushed a commit to jinzhen-lin/vllm that referenced this pull request Aug 9, 2025

[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files (vllm-project#…

18fb8c1

…20560) Signed-off-by: jiang1.li <[email protected]> Signed-off-by: Jinzhen Lin <[email protected]>

epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 27, 2025

[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files (vllm-project#…

dd489af

…20560) Signed-off-by: jiang1.li <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files #20560

[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files #20560

Uh oh!

bigPYJ1151 commented Jul 7, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Jul 7, 2025

Uh oh!

gemini-code-assist bot Jul 7, 2025

Uh oh!

gemini-code-assist bot Jul 7, 2025

Uh oh!

DarkLight1337 Jul 7, 2025 •

edited

Loading

Uh oh!

DarkLight1337 Jul 7, 2025

Uh oh!

Uh oh!

DarkLight1337 commented Jul 7, 2025

Uh oh!

mgoin Jul 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		if current_platform.is_cpu() and os.environ.get("VLLM_USE_V1", "0") == "0":
		pytest.skip("CPU only supports V1")

Uh oh!

[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files #20560

[CI/Build][CPU] Fix CPU CI and remove all CPU V0 files #20560

Uh oh!

Conversation

bigPYJ1151 commented Jul 7, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Essential Elements of an Effective PR Description Checklist

Purpose

Test Plan

Test Result

(Optional) Documentation Update

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

DarkLight1337 Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DarkLight1337 Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DarkLight1337 commented Jul 7, 2025

Uh oh!

mgoin Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bigPYJ1151 commented Jul 7, 2025 •

edited by github-actions bot

Loading

DarkLight1337 Jul 7, 2025 •

edited

Loading