NWOR masking via cumprod; add SCV modes (graph/adaptive)

[yuz207]

Summary

• Reworks NWOR masking logic to a vectorized cumprod-based approach, preserving masking semantics.
• Adds speculative chunk verify (SCV) mode support via VLLM_SCV_MODE environment variable with options "off", "graph", or "adaptive" (default "off"). Runtime support integrated into GPUModelRunner with graph-backed execution path and vectorized/graph-based SCV paths.

Changes

NWOR masking

• In vllm/v1/worker/gpu_model_runner.py, replaced per-slice accepted/reject branching with vectorized computation:
  • Compute comparison = (row == draft_slice)
  • Compute prefix = torch.cumprod(comparison.to(torch.int32), dim=0)
  • Apply mask_work[start:end] = prefix.to(torch.bool)
• Introduced SCV-aware vectorized path: when SCV is enabled, attempt to use the vectorized SCV routine to produce a mask early.
• Removed prior explicit accepted/reject handling and related control flow.

SCV mode support

• In vllm/envs.py, added VLLM_SCV_MODE environment variable with options "off", "graph", or "adaptive" (default "off").
• In vllm/v1/worker/gpu_model_runner.py, introduced self._scv_mode initialized from envs.VLLM_SCV_MODE.lower() and added _scv_enabled() to validate and enable SCV modes.
• Added new vectorized SCV path and graph-backed execution support:
  • _scv_vectorized_mask(...) handles SCV mask computation across modes.
  • _SCVGraphEntry and SCVGraphExecutor enable CUDA graph-based SCV execution when in graph mode.

Rationale

• Simplifies NWOR masking logic and leverages tensor operations for potential performance gains while preserving the same masking semantics (true up to first mismatch, false thereafter).
• Introduces optional speculative chunk verify modes that can be enabled as needed without altering public APIs.

Tests

• Validate NWOR mask generation on synthetic slices:
  • All tokens match: mask is all True for the slice
  • Mismatch at various positions: mask is True up to first mismatch, then False
  • Mismatch at first element: mask is all False for the slice
• Ensure no public API changes and that downstream NWOR paths behave identically.
• Validate SCV mode parsing and enabling logic (default off; graph/adaptive when configured).
• Verify graph-based SCV path executes when enabled and CUDA graph is available.
• New test to ensure SCV vectorized mask matches reference:
  • test_scv_vectorized_mask_matches_reference validates the SCV-based mask equals the expected reference for a given metadata and sampled inputs.

Backwards Compatibility

• No API changes; internal refactor only. SCV mode defaults to off, maintaining compatibility.

How to Review

• Focus on the GPUModelRunner NWOR masking block in vllm/v1/worker/gpu_model_runner.py.
• Verify edge cases (no mismatch, first-element mismatch, later mismatch) behave identically to the previous implementation.
• Verify SCV mode parsing, enabling logic, and vectorized/graph paths are wired correctly.
• Review the new SCV graph entry/executor logic to ensure safe graph-based execution and fallback behavior.

📎 Task: https://www.terragonlabs.com/task/cfda0cbe-9bb8-4b82-9858-540824550cf2