From 5d83a294d1a977b6f0d09b692fabc92bf165cad3 Mon Sep 17 00:00:00 2001 From: Pavol Rusnak Date: Sat, 18 Mar 2023 21:58:46 +0100 Subject: [PATCH 1/2] Add memory/disk requirements to readme --- README.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 808d54e89ec96..fc8b2fda387c6 100644 --- a/README.md +++ b/README.md @@ -155,7 +155,17 @@ python3 convert-pth-to-ggml.py models/7B/ 1 When running the larger models, make sure you have enough disk space to store all the intermediate files. -TODO: add model disk/mem requirements +### Memory/Disk Requirements + +As the models are currently fully loaded into memory, you will need adequate disk space to save them +and sufficient RAM to load them. At the moment, memory and disk requirements are the same. + +| model | original size | quantized size (4-bit) | +|-------|---------------|------------------------| +| 7B | 13 GB | 3.9 GB | +| 15B | 24 GB | 7.8 GB | +| 30B | 60 GB | 19.5 GB | +| 65B | 120 GB | 38.5 GB | ### Interactive mode From 8cb60021fa1ccb3a7902c662c30bf1401aedfebe Mon Sep 17 00:00:00 2001 From: Pavol Rusnak Date: Sat, 18 Mar 2023 22:20:04 +0100 Subject: [PATCH 2/2] Add note about Python 3.11 to readme --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index fc8b2fda387c6..187f82f619626 100644 --- a/README.md +++ b/README.md @@ -153,6 +153,8 @@ python3 convert-pth-to-ggml.py models/7B/ 1 ./main -m ./models/7B/ggml-model-q4_0.bin -n 128 ``` +Currently, it's best to use Python 3.9 or Python 3.10, as `sentencepiece` has not yet published a wheel for Python 3.11. + When running the larger models, make sure you have enough disk space to store all the intermediate files. ### Memory/Disk Requirements