Skip to content

[Feature]: does LLamaSharp support linux-arm64 runtime? #1137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nipeone opened this issue Mar 25, 2025 · 5 comments
Open

[Feature]: does LLamaSharp support linux-arm64 runtime? #1137

nipeone opened this issue Mar 25, 2025 · 5 comments
Labels
backend enhancement New feature or request feature request good first issue Good for newcomers

Comments

@nipeone
Copy link

nipeone commented Mar 25, 2025

Background & Description

I found that llamasharp has supported below runtime for a while.but not found linux-arm64, Is there any plan to support it?

linux-musl-x64
linux-x64
osx-arm64
osx-x64
win-x64

API & Usage

No response

How to implement

No response

@martindevans
Copy link
Member

There's no support for Linux-arm64 at the moment. Would you be interested in working on a PR to include it?

The first step would be modifying compile.yml to include a linux-arm64 build.

@martindevans martindevans added enhancement New feature or request good first issue Good for newcomers feature request backend labels Mar 25, 2025
@nipeone
Copy link
Author

nipeone commented Mar 26, 2025

okay, i'll have a try

@nipeone
Copy link
Author

nipeone commented Mar 26, 2025

I made a few attempts, but PR is a bit difficult due to my inexperience with github actions. I see that the releases of llama.cpp have a compiled ubuntu-arm64 version, can I copy it directly into the project and use it?

@martindevans
Copy link
Member

It needs to fit into our pipeline, but it should act as a good guide. The matrix (line 27) is a set of different build targets. The rest of the pipeline will be run once for each thing in the matrix. So adding an arm64 target should involve adding a new item to the matrix, and working out what build defines are needed for arm64

@nipeone
Copy link
Author

nipeone commented Mar 27, 2025

I have completed a version and am running actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend enhancement New feature or request feature request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants