[BUG] Qwen-7B-Chat AttributeError: 'LlamaSplitFuseInferStateInfo' object has no attribute 'logn_values'

[exceedzhang]

Before you submit an issue, please search for existing issues to avoid duplicates.

Issue description:
AttributeError: 'LlamaSplitFuseInferStateInfo' object has no attribute 'logn_values'

Please provide a clear and concise description of your issue.

Steps to reproduce:
python -m lightllm.server.api_server --model_dir /root/autodl-tmp/Qwen-7B-Chat --tp 1 --trust_remote_code --splitfuse_mode

Please list the steps to reproduce the issue, such as:

Expected behavior:

Please describe what you expected to happen.

Error logging:

If applicable, please copy and paste the error message or stack trace here. Use code blocks for better readability.

Environment:

Please provide information about your environment, such as:

• Using container

• OS: (Ubuntu 14.04, CentOS7)

• GPU info:

  • nvidia-smi (e.g. NVIDIA-SMI 525.116.04 Driver Version: 525.116.04 CUDA Version: 12.0)
  • Graphics cards: (4090x1)

• Python: (e.g. CPython3.10)

• LightLLm: (git commit-hash)

• openai-triton: pip show triton
  Name: triton
  Version: 2.1.0
  Summary: A language and compiler for custom Deep Learning operations
  Home-page: https://github.com/openai/triton/
  Author: Philippe Tillet
  Author-email: phil@openai.com
  License:
  Location: /root/miniconda3/lib/python3.10/site-packages
  Requires: filelock
  Required-by: lightllm, torch

Additional context:

Language:

lightllm run Qwen-7B-Chat use splitfuse_mode not worked.

[hiworldwzj]

@exceedzhang splitfuse mode is in test. so this mode only support llama and llama2 now.

[hiworldwzj]

we will try to support other mode types soon.

[ObliviousDonkey]

does it support qwen VL?

[hiworldwzj]

@ObliviousDonkey qwen VL will be supported soon.