Skip to content

Fix: GCC Target Mismatch for vdotq_s32 on AArch64 (Apple M1, M2, etc) #743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

russellwmy
Copy link

Problem

Building llama.cpp on AArch64 (e.g. in Docker on macOS M1/M2) fails when compiling NEON dot product code using vdotq_s32. GCC throws the following error:

error: inlining failed in call to 'always_inline' 'int32x4_t vdotq_s32(...)': target specific option mismatch

This error occurs because vdotq_s32 requires the ARMv8.x-A dot product extension (+dotprod), which is not enabled by default in most toolchains.

This PR disables GGML's use of native optimizations as a temporary fix, without modifying llama.cpp directly. Since Apple Silicon CPUs may not be a primary target for the llama.cpp project, this workaround allows successful builds.

Feel free to close this PR if it does not apply to your use case.

@MarcusDunn
Copy link
Contributor

I think this triggers too often (i.e. some builds on arm64 linux will want native optimizations). I'm not opposed to allowing this to be disabled, but an env var or a disabled default feature would be preferred.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants