[Feature]: Support for Gemma2

[tinkle-bell]

Background & Description

I think the backend should be updated to support for gemma2.

API & Usage

No response

How to implement

Update the backend.

[martindevans]

There are still PRs open in llama.cpp for proper gemma support (e.g. ggml-org/llama.cpp#8197) which I'm waiting on before starting a backend update :)

[mhjabreel]

The PR ggml-org/llama.cpp#8197 is merged, do you have any ETA for this feature?

[martindevans]

I've started on a binary update, but it's held up at the moment due to an issue in the MacOS build pipeline.

[SignalRT]

#810 is the PR where I'm working on it

[martindevans]

Gemma2 is supported now that 0.14.0 is released 🥳