The quantize.py script is not needed anymore. Just fetch the latest code and do this as a quantization step:

[huangzhimin4read]

          The `quantize.py` script is not needed anymore. Just fetch the latest code and do this as a quantization step:

./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2

Originally posted by @prusnak in #621 (comment)

I tried this method in Colab, but it still reports an error:
/bin/bash: ./quantize: No such file or directory

[CoderRC]

You have to compile the library first. Try: #103 (comment)

[prusnak]

Follow the instructions in the README file which tells you to compile the things first using make

[montvid]

I used cmake to build and have to write the path to quantize:
./build/bin/quantize models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_1.bin q4_1

[xxxwp123]

          The `quantize.py` script is not needed anymore. Just fetch the latest code and do this as a quantization step:

./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2

Originally posted by @prusnak in #621 (comment)

I tried this method in Colab, but it still reports an error: /bin/bash: ./quantize: No such file or directory

Have you solved this problem?I have the same problem.thanks