[Bug]: Failed to load Qwen3-Embedding model.

[Xu-Wenqing]

Your current environment

Qwen3-Embedding-0.6B: https://www.modelscope.cn/models/Qwen/Qwen3-Embedding-0.6B/summary
Failed to load

🐛 Describe the bug

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
ERROR 06-06 11:53:56 [core.py:515] EngineCore failed to start.
ERROR 06-06 11:53:56 [core.py:515] Traceback (most recent call last):
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/v1/engine/core.py", line 506, in run_engine_core
ERROR 06-06 11:53:56 [core.py:515]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 06-06 11:53:56 [core.py:515]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/v1/engine/core.py", line 390, in __init__
ERROR 06-06 11:53:56 [core.py:515]     super().__init__(vllm_config, executor_class, log_stats,
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/v1/engine/core.py", line 76, in __init__
ERROR 06-06 11:53:56 [core.py:515]     self.model_executor = executor_class(vllm_config)
ERROR 06-06 11:53:56 [core.py:515]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/executor/executor_base.py", line 53, in __init__
ERROR 06-06 11:53:56 [core.py:515]     self._init_executor()
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/executor/uniproc_executor.py", line 48, in _init_executor
ERROR 06-06 11:53:56 [core.py:515]     self.collective_rpc("load_model")
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
ERROR 06-06 11:53:56 [core.py:515]     answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 06-06 11:53:56 [core.py:515]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/utils.py", line 2656, in run_method
ERROR 06-06 11:53:56 [core.py:515]     return func(*args, **kwargs)
ERROR 06-06 11:53:56 [core.py:515]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/v1/worker/gpu_worker.py", line 165, in load_model
ERROR 06-06 11:53:56 [core.py:515]     self.model_runner.load_model()
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/v1/worker/gpu_model_runner.py", line 1592, in load_model
ERROR 06-06 11:53:56 [core.py:515]     self.model = model_loader.load_model(
ERROR 06-06 11:53:56 [core.py:515]                  ^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/model_executor/model_loader/base_loader.py", line 41, in load_model
ERROR 06-06 11:53:56 [core.py:515]     self.load_weights(model, model_config)
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/model_executor/model_loader/default_loader.py", line 269, in load_weights
ERROR 06-06 11:53:56 [core.py:515]     loaded_weights = model.load_weights(
ERROR 06-06 11:53:56 [core.py:515]                      ^^^^^^^^^^^^^^^^^^^
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/model_executor/models/qwen3.py", line 321, in load_weights
ERROR 06-06 11:53:56 [core.py:515]     return loader.load_weights(weights)
ERROR 06-06 11:53:56 [core.py:515]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/model_executor/models/utils.py", line 278, in load_weights
ERROR 06-06 11:53:56 [core.py:515]     autoloaded_weights = set(self._load_module("", self.module, weights))
ERROR 06-06 11:53:56 [core.py:515]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-06 11:53:56 [core.py:515]   File "/mnt/workspace/vllm/vllm/model_executor/models/utils.py", line 264, in _load_module
ERROR 06-06 11:53:56 [core.py:515]     raise ValueError(msg)
ERROR 06-06 11:53:56 [core.py:515] ValueError: There is no module or parameter named 'embed_tokens' in Qwen3ForCausalLM

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Xu-Wenqing]

Already figured out the failed reason, I will try to fix this.

[iicaicai]

@Xu-Wenqing Can you explain how to solve this problem

[pengjiang80]

Explicitly set --task embed should help

[Xu-Wenqing]

@Xu-Wenqing Can you explain how to solve this problem

compared this model file: https://www.modelscope.cn/models/Qwen/Qwen3-Embedding-8B/file/view/master/model-00001-of-00004.safetensors with Qwen3-8B model file: https://www.modelscope.cn/models/Qwen/Qwen3-8B/file/view/master/model-00001-of-00005.safetensors the tensor name in Qwen3-8B has a prefix "model", so "Qwen3ForCausalLM" in vLLM not works for Qwen3-Embedding-8B.

[Xu-Wenqing]

@pengjiang80 Cool! It works for me. Also, I found in this PR: #11469 solved the "embedding model tensor not match" issue. So there's no need to create extra PR to fix it, just add --task embed.

[dengcao]

You can use Vllm to deploy the Qwen3-Embedding&Qwen3-Reranker large model.There is already a temporary solution:
以下是通过vllm快捷安装Qwen3-Embedding-0.6B/4B/8B，Qwen3-Reranker-0.6B/4B/8B的办法。

Docker 快速部署（一键安装包）：
Docker Compose Quick Installation:

Hugging Face：
https://huggingface.co/dengcao/Qwen3-Embedding-0.6B
https://huggingface.co/dengcao/Qwen3-Embedding-4B
https://huggingface.co/dengcao/Qwen3-Embedding-8B
https://huggingface.co/dengcao/Qwen3-Reranker-0.6B
https://huggingface.co/dengcao/Qwen3-Reranker-4B
https://huggingface.co/dengcao/Qwen3-Reranker-8B

ModelScope：
https://www.modelscope.cn/models/dengcao/Qwen3-Embedding-0.6B
https://www.modelscope.cn/models/dengcao/Qwen3-Embedding-4B
https://www.modelscope.cn/models/dengcao/Qwen3-Embedding-8B
https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-0.6B
https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-4B
https://www.modelscope.cn/models/dengcao/Qwen3-Reranker-8B

GitHub：
https://github.com/dengcao/Qwen3-Reranker-8B
https://github.com/dengcao/Qwen3-Reranker-4B
https://github.com/dengcao/Qwen3-Reranker-0.6B