Feature Request: Falcon-H1

[wrapss]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

https://falcon-lm.github.io/blog/falcon-h1/
https://huggingface.co/collections/tiiuae/falcon-h1-6819f2795bc406da60fab8df

Motivation

look very competitive

Possible Implementation

No response

[henk717]

Authors made a fork instead of a PR : https://github.com/tiiuae/llama.cpp-Falcon-H1

[jacekpoplawski]

They wrote "A fork of llama.cpp with added support for the Falcon-H1 architecture. We are actively working on submitting a pull request to merge these changes into the official llama.cpp repository. "

I tried that fork but I see problems tiiuae/llama.cpp-Falcon-H1#2

[crisdesivo]

Any news on this?

[jacekpoplawski]

looks like their fork doesn't work correctly and there was no code update for a week

[younesbelkada]

With the help from community and other integration effort for Mamba2-related models, we were able to open: #14238 that hopefully will get merged soon into llama.cpp master 🙏 It is possible to try that branch and use existing official GGUFs from tiiuae organization: https://huggingface.co/collections/tiiuae/falcon-h1-6819f2795bc406da60fab8df