Skip to content

Conversation

ggerganov
Copy link
Member

@ggerganov ggerganov commented Nov 9, 2024

fix #10208
cont #10171

This is the relevant change for the build fix:

typedef decltype(kernel_flash_attn_ext_vec<FA_TYPES, half4x4, 1, dequantize_f16, half4x4, 1, dequantize_f16, 128>) flash_attn_ext_vec_t;

The FA vec implementation currently requires D%128 == 0.

@ggerganov ggerganov merged commit 39a334a into master Nov 9, 2024
54 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 15, 2024
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: ggml_metal_init error: zero-length arrays are not permitted in C++ float4x4 lo[D16/NW4];
1 participant