You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, Can you give some advice about how to inference finetuned Starcoder model with this code? Since lora finetune changed some of layers of the model, some of the code in starcoder.cpp should be changed, how can I use this code to inference with my finetuned Starcoder model?
Unfortunately this is not supported yet, but I think llama.cpp does support it.
It would be great if somebody can adapt that to starcoder.cpp ggml-org/llama.cpp#820
Activity
NouamaneTazi commentedon Jun 8, 2023
Unfortunately this is not supported yet, but I think llama.cpp does support it.
It would be great if somebody can adapt that to starcoder.cpp
ggml-org/llama.cpp#820