Skip to content

Add Q8_0 quantization for intermediate results #951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 15, 2023
Merged

Add Q8_0 quantization for intermediate results #951

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3b894ec
ggml : add Q8_0 quantization for intermediate results
ggerganov Apr 13, 2023
19e7a65
quantize-stats : fix test + add it to Makefile default
ggerganov Apr 13, 2023
2c4f9b6
Q8: use int8_t, AVX/AVX2 optimizations
sw Apr 13, 2023
312a927
ggml : fix quantize_row_q8_0() ARM_NEON rounding
ggerganov Apr 14, 2023
3a111ab
minor : updates after rebase to latest master
ggerganov Apr 14, 2023
01de5c5
quantize-stats : delete obsolete strings
ggerganov Apr 14, 2023
60f27ed
ggml : fix q4_1 dot func
ggerganov Apr 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ $(info I CC: $(CCV))
$(info I CXX: $(CXXV))
$(info )

default: main quantize perplexity embedding
default: main quantize quantize-stats perplexity embedding

#
# Build library
Expand Down
Loading