-
-
Notifications
You must be signed in to change notification settings - Fork 10.3k
[v0][structured output] Support reasoning output #12955
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
vllm/model_executor/guided_decoding/reasoner/deepseek_reasoner.py
Outdated
Show resolved
Hide resolved
a2c9058
to
f5ec168
Compare
50be0e9
to
6b76121
Compare
8416648
to
3034e5c
Compare
@russellb Could you please help review this PR? |
3034e5c
to
eaaced1
Compare
Please let me know if anyone is ready to review and merge this. Once confirmed, I will resolve the conflicts here. It’s time-consuming to address them daily. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have few questions regarding implementations, otherwise lgtm
vllm/model_executor/guided_decoding/reasoner/deepseek_reasoner.py
Outdated
Show resolved
Hide resolved
This pull request has merge conflicts that must be resolved before it can be |
471eb35
to
80bce05
Compare
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
0211291
to
17969af
Compare
vllm/model_executor/guided_decoding/outlines_logits_processors.py
Outdated
Show resolved
Hide resolved
Signed-off-by: Ce Gao <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the great progress on this feature @gaocegege !
Signed-off-by: Ce Gao <[email protected]>
Signed-off-by: Ce Gao <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>
Signed-off-by: Ce Gao <[email protected]>
Ref #12619
Only apply Guided/Structured grammar after reasoning steps in Reasoning models. It enabled reasoning outputs for xgrammar and outlines, didn't support lm-format-enforcer.
Usage: