[Usage]: can I use it with classification model (e.g. GemmaForSequenceClassification) ?

[dodler]

Your current environment

The output of `python collect_env.py`

How would you like to use vllm

can I use it with classification model (e.g. GemmaForSequenceClassification) ?

[DarkLight1337]

There is a WIP PR (#6260) to enable this.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[DarkLight1337]

You can now create a version of Gemma2EmbeddingModel, change the pooling type to CLS (or whatever your model uses for classification), then register it to vLLM as an OOT model