How to set the works for the bge-reranker-v2-m3 model? #401
Unanswered
kennard520
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
When I deploy the bge-reranker-v2-m3 model on the TEI I discover it doesn't support multiple threads, How to make it support mutiple threads? bellows are my configuration
- MODEL_ID=BAAI/bge-reranker-v2-m3 - TOKENIZATION_WORKERS=10 - MAX_BATCH_REQUESTS=10 - MAX_CLIENT_BATCH_SIZE=500 - POOLING=mean - HOSTNAME=0.0.0.0Beta Was this translation helpful? Give feedback.
All reactions