[Windows / Pycharm] llama_cpp_python installs and runs successfully without BLAS backend, but does not work with cuBLAS flags

[technicolor-twelve]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I was able to run pip install llama-cpp-python successfully in PyCharm Terminal and use it in my Python code, so when I saw that additional flags needed to be set for cuBLAS, I expected the following code to run successfully:

$env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'
$env:FORCE_CMAKE=1
$env:LLAMA_CUBLAS=1
pip install llama-cpp-python

Current Behavior

I'm still getting the BLAS = 0 indicator, based on the printout of the terminal:
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |

Environment and Context

I'm running PyCharm 2022.2 with Python 3.9, and I have llama-cpp-python 0.1.50 installed. I have an NVIDIA GeForce RTX 3070 and have been able to get cuBLAS up and running with the regular llama.cpp.

[gjmulder]

Can we see the output of your package build with the --verbose flag added?

[technicolor-twelve]

@gjmulder This is the output when I run pip install llama-cpp-python with the previous flags enabled. Is this what you're looking for?

llama-cpp-python pip install output.txt

[abetlen]

@technicolor-twelve try pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir as it's using a cached version of the package.

[technicolor-twelve]

That forced a new install, though now when I run from llama_cpp import Llama I get the following runtime error:

llama runtime error.txt

Which is weird since I could navigate to the referenced and see the .dll file there. I tried adding an os.add_dll_directory to specifically include the path but the error still remains.

[gjmulder]

That looks like a path issue. Can you review #225, please?

[technicolor-twelve]

That worked! Thank you so much for your help!

…

On Tue, May 23, 2023, 5:07 AM Gary Mulder ***@***.***> wrote: That looks like a path issue. Can you review #225 <#225>, please? — Reply to this email directly, view it on GitHub <#228 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJ4R2UQUEJMFW7PBVRUOKITXHSD57ANCNFSM6AAAAAAYFYEWIE> . You are receiving this because you were mentioned.Message ID: ***@***.***>