-
Notifications
You must be signed in to change notification settings - Fork 12k
ggml-alloc : use virtual memory for measurement #2973
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I have tested this successfully under Linux and Windows, including 32-bit builds, but I would need some help testing with OS X. |
I'll do. What kind of test could I perform? |
All you would need to do is make sure it builds without errors and generate a few tokens using |
Awesome, thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did some testing on M2 Ultra and it works.
I wonder, should we fallback to the old strategy if mmap
is not available?
I'm planning soon to update whisper.cpp
to use ggml-alloc
and update the WASM examples.
I have added a fallback for systems without virtual memory, however I noticed that emscripten has a shim for |
As a sidenote Considering mmap, this seems related: WebAssembly/WASI#304 There's also this project which I have used successfully before, and it's simple enough so extending it is not very complicated, and it uses clang to produce wasm: https://github.com/schellingb/wajic Edit: It should be possible to get rid of emscripten in favour of clang, if that helps in any way. |
Instead of using a fixed memory address, allocates an uncommitted virtual memory region that is guaranteed to not overlap with any other allocations. Should solve issues with most 32-bit platforms and OS X. Wasm may still be an issue, as I don't think it supports
mmap
.