[V1] Logits processor docs

[afeldman-nm]

Purpose

• Document the vLLM v1 logits processor functionality including built-in logits processors and custom logits processors

Test Plan

N/A

Test Result

N/A

(Optional) Documentation Update

See Purpose

Essential Elements of an Effective PR Description Checklist

• [x ] The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• [ x] The test plan, such as providing test command.
• [ x] The test results, such as pasting the results comparison before and after, or e2e results
• [ x] (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request introduces documentation for the custom logits processor extensibility feature. The new markdown file explains how to create and use custom logits processors, including a code example. My review focuses on fixing several issues in this code example to make it functional and clear for developers.

[gemini-code-assist]

The example DummyLogitsProcessor has a few issues that prevent it from working correctly and could confuse users:

1. The introductory sentence on line 16 is incomplete.
2. The type hint for self.req_info in __init__ is dict[int, SamplingParams], but it is used to store integer target_token values. This should be dict[int, int].
3. The apply method is incomplete. When self.req_info is populated, it doesn't return the logits tensor, which will lead to a runtime error. The logic is also unfinished.

I've provided a corrected version of the example that addresses these points, making it a complete and functional illustration of a custom logits processor.

The contrived example below implements a logits processor that forces the model to select a specific `target_token` for requests that provide it.

??? code "Example custom logits processor definition"

    ```python
    from typing import Optional
    import torch
    from vllm.config import VllmConfig
    from vllm.sampling_params import SamplingParams
    from vllm.v1.sample.logits_processor import (BatchUpdate,
                                                LogitsProcessor,
                                                MoveDirectionality)

    class DummyLogitsProcessor(LogitsProcessor):
        """Fake logit processor to support unit testing and examples"""

        def __init__(self, vllm_config: "VllmConfig", device: torch.device,
                    is_pin_memory: bool):
            self.req_info: dict[int, int] = {}

        def is_argmax_invariant(self) -> bool:
            """Never impacts greedy sampling"""
            return False

        def update_state(self, batch_update: Optional[BatchUpdate]):
            if not batch_update:
                return

            # Process added requests.
            for index, params, _, _ in batch_update.added:
                assert params is not None
                if params.extra_args and (target_token :=
                                        params.extra_args.get("target_token")):
                    self.req_info[index] = target_token

            if self.req_info:
                # Process removed requests.
                for index in batch_update.removed:
                    self.req_info.pop(index, None)

                # Process moved requests, unidirectional move (a->b) and swap
                # (a<->b)
                for adx, bdx, direct in batch_update.moved:
                    a_val = self.req_info.pop(adx, None)
                    b_val = self.req_info.pop(bdx, None)
                    if a_val is not None:
                        self.req_info[bdx] = a_val
                    if direct == MoveDirectionality.SWAP and b_val is not None:
                        self.req_info[adx] = b_val

        def apply(self, logits: torch.Tensor) -> torch.Tensor:
            if not self.req_info:
                return logits

            rows_list = list(self.req_info.keys())
            cols = torch.tensor([self.req_info[i] for i in rows_list],
                                dtype=torch.long,
                                device=logits.device)
            rows = torch.tensor(rows_list, dtype=torch.long, device=logits.device)

            # Get the original logits for the target tokens.
            values_to_keep = logits[rows, cols].clone()

            # For requests with a target token, set all other logits to -inf.
            # This is a contrived example to force the model to select the
            # target token.
            for row_idx in rows_list:
                logits[row_idx, :] = -float("inf")

            logits[rows, cols] = values_to_keep
            return logits
    ```

[afeldman-nm]

Thank you @JosephMarinier for your review, I believe I addressed everything you mentioned

[JosephMarinier]

Thank you for the cool feature! 🙏

[njhill]

Thanks @afeldman-nm, looks great apart from the one comment, we could merge this since that will likely need to change soon anyhow.

[njhill]

Thanks @afeldman-nm