GGUF community on Hugging Face

[olegshulyakov]

Hey @ggerganov,

I have noticed there are a lot of model duplicates on HF for same GGUFs. Since people are using llama.cpp as a base for quantization I have an idea to create a gguf-community like mlx-community does?

I think we can combine models from TheBloke, bartowski, mradermacher and others in one place.

What do you think?

[ggerganov]

What would be the purpose of this new community? I don't think it improves any use cases or the user experience, so not sure it makes sense.

[cmp-nct]

The main concern I've always had is compatibility. A lot of gguf files are not going to run on llama.cpp.
A ton of gguf files on HF are not going to work with llama.cpp and 2/3 of the remaining ones are outdated (tokenizer old, meta data missing, etc).

Maybe it's time for a standardized sub extension or even two extensions behind "gguf".
modelname.project.version.gguf
The subextension should give an idea for which project (like llama.cpp, whisper, hf transformers etc) and for which major version it was created.