Support for OpenELM of Apple

[Ce-daros]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Feature Description

Support for OpenELM of Apple

https://huggingface.co/apple/OpenELM-3B-Instruct/tree/main

[ggerganov]

Nice to see the LLaMA* idea implemented in these models:

[joshcarp]

Not sure if anyone is working on this yet but i'm happy to pick it up

[mertbozkir]

same for me as well, I can pick it up. I am trying to upload the model on ollama, but got NotImplementedError: Architecture 'OpenELMForCausalLM' not supported!

[joshcarp]

Okay update: It's more difficult as I first expected it and this is a new codebase so it's stumped me a bit.
OpenELM have defined the variables: a_min, a_max for scaling the attention heads and b_min and b_max for scaling the ffn. This is both captured in their config but you can also derive it from the equation:

Still attempting it but i don't think i'm gonna be done any time soon

[Wladastic]

@joshcarp
Waiting patiently to try it out hopefully 🤞
Are you having any troubles figuring it out?

[joshcarp]

Yeah, so basically, i think I can't figure out how to calculate the kqv offsets for the kqv tensor every layer.
I'd be lying if i didn't admit i've just been throwing spaghetti at the wall as shown my the git history on this: https://github.com/joshcarp/llama.cpp

I'm using this as a reference: https://huggingface.co/apple/OpenELM-270M/blob/main/modeling_openelm.py
https://github.com/apple/corenet/tree/main/mlx_examples/open_elm

If anyone else wants to implement this feel free

[joshcarp]

If anyone can help out: #6986

[kevinsuo]

Does this work? Did anyone try this?

https://huggingface.co/LiteLLMs/OpenELM-GGUF

[Wladastic]

Does this work? Did anyone try this?

https://huggingface.co/LiteLLMs/OpenELM-GGUF

Have you checked the files?
It specifically states that it is just a placeholder, there is no model

[olumolu]

Is it possible to have support forr openelm
I am curious for this because this can be run on mobile devices with less power don't know about the performance though. Compare to gemma:2b or phi3