[Feature]: Add Support for thinking_budget for Qwen3 Models

[raghavgarg97]

🚀 The feature, motivation and pitch

Alibaba Documentation (https://www.alibabacloud.com/help/en/model-studio/deep-thinking) mentions the usage of thinking_budget to control the thinking length hence generation time !

can we please get it added to vllm where /think is hard inserted after thinking_budget number of thinking tokens to stop thinking..

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

Does their HF model support this? From my understanding they only support this in their official service.

[raghavgarg97]

yes i suppose this is only available in their Cloud API and not integrated into transformers..
But this might be a good feature to add given that many of the recent open source models are integrating dynamic and token controlled thinking_
eg: https://huggingface.co/deepcogito/cogito-v1-preview-llama-70B

[DarkLight1337]

I suggest you ask in Qwen team's upstream repo.

[raghavgarg97]

yes,have opened the same: QwenLM/Qwen3#1388

thanks!

[hwb96]

Thinking Budget. An additional advantage of Thinking Mode Fusion is that, once the model learns to respond in both non-thinking and thinking modes, it naturally develops the ability to handle intermediate cases—generating responses based on incomplete thinking. This capability lays the foundation for implementing budget control over the model’s thinking process. Specifically, when the length of the model’s thinking reaches a user-defined threshold, we manually halt the thinking process and insert the stop-thinking instruction: “Considering the limited time by the user, I have to give the solution based on the thinking directly now.\n.\n\n”. After this instruction is inserted, the model proceeds to generate a final response based on its accumulated reasoning up to that point. It is worth noting that this ability is not explicitly trained but emerges naturally as a result of applying Thinking Mode Fusion.

Who can help incorporate this new feature based on this principle?

[MrBlue-1996]

How to achieve this "Consider ...." ,if we only directly truncated ,will it affect the inference results?

[raghavgarg97]

@MrBlue-1996 the paper mentions that it should still follow the instructions(quality and results can be little different) ..but we can have the power to control the exact tradeoff of quality and inference time

[Rqcker]

In Alibaba's documentation this feature is simple to implement. It's just an inference engine function, just need to configure a limit and calculate the generated tokens until it reaches the Max. Are VLLm or Transformers planned to be added?

[DarkLight1337]

The thinking budget feature can be handled from client side, see the official response in #18127 (comment)

[remusao]

If this is done client-side, wouldn't it require that all instances share the same prefix cache in order to not perform extra computation (e.g. if the thinking is done on one instance of vLLM but then resuming after capping thinking is done on another, the shared prefix will be processed again)? Whereas if integrated into vLLM it could be done more efficiently.

[tomasruizt]

Ok, strictly speaking this can be done from the client-side. Nevertheless, a clean API where a user can pass e.g. thinking_budget=1000 is going to facilitate experimentation a lot! Having to implemen custom logic in the client-side only for this hyperparameter is tedious, and could tip the balance between inference frameworks.

Reasoning models are becoming more widespread, but they often suffer from overthinking and long latency. Being able to control the thinking budget could be a strategically smart feature to implement in vLLM.

[nicofretti]

Ongoing feature request here