Skip to content

[Deprecation] Remove prompt_token_ids arg fallback in LLM.generate and LLM.embed #18800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Aug 22, 2025
Merged

[Deprecation] Remove prompt_token_ids arg fallback in LLM.generate and LLM.embed #18800

merged 13 commits into from
Aug 22, 2025

Conversation

DarkLight1337
Copy link
Member

@DarkLight1337 DarkLight1337 commented May 28, 2025 •
edited by github-actions bot
Loading

First introduced all the way back in #4328, with deprecation flag turned on in #11129. This should be safe to remove.

Note: lm-eval has to be updated to continue working after this change

@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label May 28, 2025
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@DarkLight1337 DarkLight1337 changed the title [Deprecation] Remove prompt_token_ids arg fallback [Deprecation] Remove prompt_token_ids arg fallback in LLM May 28, 2025
@DarkLight1337 DarkLight1337 changed the title [Deprecation] Remove prompt_token_ids arg fallback in LLM [Deprecation] Remove prompt_token_ids arg fallback in LLM.generate and LLM.embed May 28, 2025
@DarkLight1337 DarkLight1337 mentioned this pull request May 28, 2025
Closed
@DarkLight1337
Fix example
75c7dcd 
Signed-off-by: DarkLight1337 <[email protected]>
@mergify mergify bot added the documentation Improvements or additions to documentation label May 28, 2025
@DarkLight1337
Copy link
Member Author

DarkLight1337 commented May 28, 2025 •
edited
Loading

Looks like lm_eval still uses the deprecated parameter prompt_token_ids. Let me open an issue on their end.

Edit: Opened EleutherAI/lm-evaluation-harness#3024

@DarkLight1337 DarkLight1337 mentioned this pull request May 28, 2025
Merged
@DarkLight1337
Update and clean up tests
34440dd 
Signed-off-by: DarkLight1337 <[email protected]>
@mergify mergify bot added the v1 label May 28, 2025
@mergify Mergify
Copy link

mergify bot commented May 29, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @DarkLight1337.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label May 29, 2025
@DarkLight1337 DarkLight1337 marked this pull request as draft May 29, 2025 16:41
@mergify mergify bot removed the needs-rebase label May 29, 2025
@mergify Mergify
Copy link

mergify bot commented Jul 3, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @DarkLight1337.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Jul 3, 2025
@mergify mergify bot removed the needs-rebase label Aug 3, 2025
@DarkLight1337
Bump lm-eval version
9064ad5 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
Update more tests
bf1fad0 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
Copy link
Member Author

After updating lm-eval with the patch, tests pass now

@DarkLight1337 DarkLight1337 merged commit 8896eb7 into vllm-project:main Aug 22, 2025
70 of 71 checks passed
@DarkLight1337 DarkLight1337 deleted the deprecate-prompt-token-ids-arg branch August 22, 2025 02:56
@hmellor
Copy link
Member

hmellor commented Aug 22, 2025

There has been a new lm-eval release which contains your PR. Can we use that pin instead of the commit hash?

@DarkLight1337
Copy link
Member Author

EleutherAI/lm-evaluation-harness#3206 has only just been merged, I don't think it's released yet?

Xu-Wenqing pushed a commit to Xu-Wenqing/vllm that referenced this pull request Aug 23, 2025
@DarkLight1337 @Xu-Wenqing
[Deprecation] Remove prompt_token_ids arg fallback in `LLM.generate…
891f148 
…` and `LLM.embed` (vllm-project#18800)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: root <[email protected]>
@hmellor
Copy link
Member

hmellor commented Aug 25, 2025

Oh I was looking at EleutherAI/lm-evaluation-harness#3024, which is in a release. I didn't notice the second PR

epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025
@DarkLight1337 @epwalsh
[Deprecation] Remove prompt_token_ids arg fallback in `LLM.generate…
f595b60 
…` and `LLM.embed` (vllm-project#18800)

Signed-off-by: DarkLight1337 <[email protected]>
tuukkjs added a commit to lcskrishna/fmwork that referenced this pull request Aug 28, 2025
@tuukkjs
Deprecate prompt_token_ids
934479c 
Remove `prompt_token_ids` and use `prompts` instead.

`prompt_token_ids` was removed from `LLM.generate` args in this
[PR](vllm-project/vllm#18800).
@tuukkjs tuukkjs mentioned this pull request Aug 28, 2025
Merged
xiao-llm pushed a commit to xiao-llm/vllm that referenced this pull request Aug 28, 2025
@DarkLight1337 @amd-xiaoyu12
[Deprecation] Remove prompt_token_ids arg fallback in `LLM.generate…
c53eb32 
…` and `LLM.embed` (vllm-project#18800)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Xiao Yu <[email protected]>
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025
@DarkLight1337 @zhewenl
[Deprecation] Remove prompt_token_ids arg fallback in `LLM.generate…
a4aa575 
…` and `LLM.embed` (vllm-project#18800)

Signed-off-by: DarkLight1337 <[email protected]>
mengxingkongzhouhan pushed a commit to mengxingkongzhouhan/vllm that referenced this pull request Aug 30, 2025
@DarkLight1337 @mengxingkongzhouhan
[Deprecation] Remove prompt_token_ids arg fallback in `LLM.generate…
82ab4dc 
…` and `LLM.embed` (vllm-project#18800)

Signed-off-by: DarkLight1337 <[email protected]>
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Sep 3, 2025
@DarkLight1337 @zhewenl
[Deprecation] Remove prompt_token_ids arg fallback in `LLM.generate…
3997efd 
…` and `LLM.embed` (vllm-project#18800)

Signed-off-by: DarkLight1337 <[email protected]>
@Potabk Potabk mentioned this pull request Sep 11, 2025
Closed
@junya-takayama junya-takayama mentioned this pull request Sep 22, 2025
Merged
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
@DarkLight1337 @FeiDaLI
[Deprecation] Remove prompt_token_ids arg fallback in `LLM.generate…
410d90b 
…` and `LLM.embed` (vllm-project#18800)

Signed-off-by: DarkLight1337 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@simon-mo simon-mo simon-mo approved these changes

@njhill njhill Awaiting requested review from njhill

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@russellb russellb Awaiting requested review from russellb russellb is a code owner

@aarnphm aarnphm Awaiting requested review from aarnphm aarnphm is a code owner

@hmellor hmellor Awaiting requested review from hmellor

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 is a code owner

Assignees

No one assigned

Labels

ci/build documentation Improvements or additions to documentation frontend ready ONLY add when PR is ready to merge/full CI is needed rocm Related to AMD ROCm speculative-decoding structured-output v1

Projects

Structured Output
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@DarkLight1337 @hmellor @simon-mo