[Usage]: vllm启动qwen2.5vl-7b以后 为什么显存使用越来越多

[wqw0806]

Your current environment

使用vllm0.7.3启动qwen2.5vl-7b模型，模型启动命令是：nohup env CUDA_VISIBLE_DEVICES=4, vllm serve /Qwen/Qwen2___5-VL-7B-Instruct/ --trust-remote-code --served-model-name qwen_model --gpu-memory-utilization 0.9 --tensor-parallel-size 4 --port 8000 &>qwen.log &
使用显卡是4张nvidia 4090 最开始启动的时候显存占用是每张卡约8G左右，运行时间越长，显存占用越多，一晚上的显存占用增加到约12G左右。

请问为什么显存会越来越多呀？应该怎么解决这个问题呀？

How would you like to use vllm

I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.
我应该怎么使用vllm才不会造成这种显存越来越多的现象呢？

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[linpan]

docker logs 查看

[DarkLight1337]

Can you check if this problem still exists in the latest version of vLLM? There have been a couple of bugfixes since v0.7.3

[giftyang]

按理vllm不是会先把配置使用的90%显存占下来的吗

[qiagerlala]

我也遇到这个现象了。随着服务灰度放量，显存占用呈跳跃式持续增长不回落
vllm==0.8.5

@wqw0806 请问有找打原因和解决方案吗？

[qiagerlala]

Can you check if this problem still exists in the latest version of vLLM? There have been a couple of bugfixes since v0.7.3

What issues were fixed in v0.7.3? @DarkLight1337

[DarkLight1337]

Memory leak issues. e.g. #14326, #16472, #17567