[Windows] [Windows] "Failed building wheel for llama-cpp-python

[kerbi]

Edit : For now i've installed the wheel from "https://github.com/Loufe/llama-cpp-python/blob/main/wheels/llama_cpp_python-0.1.26-cp310-cp310-win_amd64.whl". The installation of the wheel works. So everything is fine for me. Got things working also in WSL with no issue.
I would still be happy to build the wheel myself, first as a learning experience, to understand what I did wrong, and secondly, because if I understood well from "#40", it might lead to better performance If I compile it myself ? "The issue is that the binaries will likely not be built with the correct optimizations for the users particular CPU which will likely result in much worse performance than the user expects."
Though maybe I did not understood correctly and it doesn't matter.
I leave the issue in case it might be useful to someone, or in case someone wants to try to help me build the wheel for fun.

Hi !
I've been trying to install this package for a while, but I can't get it working on windows.

When I run "pip install llama-cpp-python", I get the following errors :

(short version, i'll put the full output at the end of the message)

ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

It seems that it is trying to find a C compiler and then build the wheel for the library (as far as I understand it).
At some point it seems to find one :

-- Trying 'Visual Studio 16 2019 x64 v142' generator - success

But then it seems to fail :

CMake Error at C:/Users/Antoine/AppData/Local/Temp/pip-build-env-m7g4zo_5/overlay/Lib/site-packages/cmake/data/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:67 (message):
        The C compiler
          "C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe"
        is not able to compile a simple test program.

And then give me a very long output full of directory paths to explain to me why it failed.

Close to the end I can see an error that might be relevant :

An error occurred while configuring with CMake.
 Command: 'C:\Users\Antoine\AppData\Local\Temp\pip-build-env-m7g4zo_5\overlay\Lib\site-packages\cmake\data\bin/cmake.exe' 'C:\Users\Antoine\AppData\Local\Temp\pip-install-plc32gz9\llama-cpp-python_ca1a0aba562945a18534f1636d884ac7' -G 'Visual Studio 16 2019' '-DCMAKE_INSTALL_PREFIX:PATH=C:\Users\Antoine\AppData\Local\Temp\pip-install-plc32gz9\llama-cpp-python_ca1a0aba562945a18534f1636d884ac7\_skbuild\win-amd64-3.10\cmake-install' -DPYTHON_VERSION_STRING:STRING=3.10.6 -DSKBUILD:INTERNAL=TRUE '-DCMAKE_MODULE_PATH:PATH=C:\Users\Antoine\AppData\Local\Temp\pip-build-env-m7g4zo_5\overlay\Lib\site-packages\skbuild\resources\cmake' '-DPYTHON_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPYTHON_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' '-DPYTHON_LIBRARY:PATH=D:\Anaconda\envs\textgen\libs\python310.lib' '-DPython_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPython_ROOT_DIR:PATH=D:\Anaconda\envs\textgen' '-DPython_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' -DPython_FIND_REGISTRY:STRING=NEVER '-DPython3_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPython3_ROOT_DIR:PATH=D:\Anaconda\envs\textgen' '-DPython3_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' -DPython3_FIND_REGISTRY:STRING=NEVER -T v142 -A x64 -DCMAKE_BUILD_TYPE:STRING=Release
        Source directory:
          C:\Users\Antoine\AppData\Local\Temp\pip-install-plc32gz9\llama-cpp-python_ca1a0aba562945a18534f1636d884ac7
        Working directory:
          C:\Users\Antoine\AppData\Local\Temp\pip-install-plc32gz9\llama-cpp-python_ca1a0aba562945a18534f1636d884ac7\_skbuild\win-amd64-3.10\cmake-build
      Please see CMake's output for more information. 

But I don't really know what to make of it.
I would really love to understand how to make it work on windows, but I lack knowledge on building wheels.
I've tried :
- Upgrading pip and setup tools
- I installed Visual studio AND build tools for C++ (therefore I have cmake on my computer, but I don't know if it's even used when trying to build the wheel considering the previous output...)
- Find 'CMake's output for more information.', but I have no idea where to find it and google didn't helped me on that one.
- Downloading the repo and trying to build it using cmake, but maybe I did it wrong :

(textgen) PS F:\ChatBots\text-generation-webui\repositories\GPTQ-for-LLaMa\cmaketentative\llama-cpp-python> cmake ./ -B./build
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19045.
CMake Error at CMakeLists.txt:21 (add_subdirectory):
  The source directory

    F:/ChatBots/text-generation-webui/repositories/GPTQ-for-LLaMa/cmaketentative/llama-cpp-python/vendor/llama.cpp

  does not contain a CMakeLists.txt file.


CMake Error at CMakeLists.txt:22 (install):
  install TARGETS given target "llama" which does not exist.


-- Configuring incomplete, errors occurred!

-Tried doing the same thing with cygwin after installing it

I've tried since yesterday to make it work but I can't figure it out. Is there someone that could help me get this working on my windows machine ?

Thank you very much in advance.

Additional info :
I'm trying to install it in a conda environment named 'textgen", but not sure it is relevant.

Full error output :

(textgen) PS F:\ChatBots\text-generation-webui\repositories\GPTQ-for-LLaMa> pip install llama-cpp-python
Collecting llama-cpp-python
  Using cached llama_cpp_python-0.1.27.tar.gz (529 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in d:\anaconda\envs\textgen\lib\site-packages (from llama-cpp-python) (4.5.0)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [263 lines of output]


      --------------------------------------------------------------------------------
      -- Trying 'Ninja (Visual Studio 17 2022 x64 v143)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is unknown
      CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
        No CMAKE_C_COMPILER could be found.

        Tell CMake where to find the compiler by setting either the environment
        variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
        the compiler, or to the compiler name if it is in the PATH.


      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja (Visual Studio 17 2022 x64 v143)' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 17 2022 x64 v143' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Generator

          Visual Studio 17 2022

        could not find any instance of Visual Studio.



      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 17 2022 x64 v143' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'Ninja (Visual Studio 16 2019 x64 v142)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is unknown
      CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
        No CMAKE_C_COMPILER could be found.

        Tell CMake where to find the compiler by setting either the environment
        variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
        the compiler, or to the compiler name if it is in the PATH.


      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja (Visual Studio 16 2019 x64 v142)' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 16 2019 x64 v142' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19045.
      -- The C compiler identification is MSVC 19.29.30148.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- The CXX compiler identification is MSVC 19.29.30148.0
      CMake Warning (dev) at C:/Users/Antoine/AppData/Local/Temp/pip-build-env-yc585726/overlay/Lib/site-packages/cmake/data/share/cmake-3.26/Modules/CMakeDetermineCXXCompiler.cmake:168 (if):
        Policy CMP0054 is not set: Only interpret if() arguments as variables or
        keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
        details.  Use the cmake_policy command to set the policy and suppress this
        warning.

        Quoted variables like "MSVC" will no longer be dereferenced when the policy
        is set to NEW.  Since the policy is not set the OLD behavior will be used.
      Call Stack (most recent call first):
        CMakeLists.txt:4 (ENABLE_LANGUAGE)
      This warning is for project developers.  Use -Wno-dev to suppress it.

      CMake Warning (dev) at C:/Users/Antoine/AppData/Local/Temp/pip-build-env-yc585726/overlay/Lib/site-packages/cmake/data/share/cmake-3.26/Modules/CMakeDetermineCXXCompiler.cmake:189 (elseif):
        Policy CMP0054 is not set: Only interpret if() arguments as variables or
        keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
        details.  Use the cmake_policy command to set the policy and suppress this
        warning.

        Quoted variables like "MSVC" will no longer be dereferenced when the policy
        is set to NEW.  Since the policy is not set the OLD behavior will be used.
      Call Stack (most recent call first):
        CMakeLists.txt:4 (ENABLE_LANGUAGE)
      This warning is for project developers.  Use -Wno-dev to suppress it.

      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Configuring done (11.1s)
      -- Generating done (0.0s)
      -- Build files have been written to: C:/Users/Antoine/AppData/Local/Temp/pip-install-kab8dxp_/llama-cpp-python_6aab703992964fd9953365ad8cceacea/_cmake_test_compile/build
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 16 2019 x64 v142' generator - success
      --------------------------------------------------------------------------------

      Configuring Project
        Working directory:
          C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build
        Command:
          'C:\Users\Antoine\AppData\Local\Temp\pip-build-env-yc585726\overlay\Lib\site-packages\cmake\data\bin/cmake.exe' 'C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea' -G 'Visual Studio 16 2019' '-DCMAKE_INSTALL_PREFIX:PATH=C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-install' -DPYTHON_VERSION_STRING:STRING=3.10.6 -DSKBUILD:INTERNAL=TRUE '-DCMAKE_MODULE_PATH:PATH=C:\Users\Antoine\AppData\Local\Temp\pip-build-env-yc585726\overlay\Lib\site-packages\skbuild\resources\cmake' '-DPYTHON_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPYTHON_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' '-DPYTHON_LIBRARY:PATH=D:\Anaconda\envs\textgen\libs\python310.lib' '-DPython_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPython_ROOT_DIR:PATH=D:\Anaconda\envs\textgen' '-DPython_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' -DPython_FIND_REGISTRY:STRING=NEVER '-DPython3_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPython3_ROOT_DIR:PATH=D:\Anaconda\envs\textgen' '-DPython3_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' -DPython3_FIND_REGISTRY:STRING=NEVER -T v142 -A x64 -DCMAKE_BUILD_TYPE:STRING=Release

      -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19045.
      -- The C compiler identification is MSVC 19.29.30148.0
      -- The CXX compiler identification is MSVC 19.29.30148.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - failed
      -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
      -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - broken
      CMake Error at C:/Users/Antoine/AppData/Local/Temp/pip-build-env-yc585726/overlay/Lib/site-packages/cmake/data/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:67 (message):
        The C compiler

          "C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe"

        is not able to compile a simple test program.

        It fails with the following output:

          Change Dir: C:/Users/Antoine/AppData/Local/Temp/pip-install-kab8dxp_/llama-cpp-python_6aab703992964fd9953365ad8cceacea/_skbuild/win-amd64-3.10/cmake-build/CMakeFiles/CMakeScratch/TryCompile-gyt569

          Run Build Command(s):C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/MSBuild/Current/Bin/MSBuild.exe cmTC_903a4.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=16.0 /v:n && Microsoft (R) Build Engine version 16.11.2+f32259642 pour .NET Framework
          Copyright (C) Microsoft Corporation. Tous droits réservés.

          La génération a démarré 09/04/2023 16:06:07.
          Projet "C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj" sur le noud 1 (cibles par défaut).
          PrepareForBuild:
            Création du répertoire "cmTC_903a4.dir\Debug\".
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppBuild.targets(517,5): warning MSB8029: Le répertoire intermédiaire ou le répertoire de sortie ne peut pas se trouver sous le répertoire temporaire car cela risque de créer des problèmes avec la génération incrémentielle. [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
            Création du répertoire "C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\Debug\".
            Création du répertoire "cmTC_903a4.dir\Debug\cmTC_903a4.tlog\".
          InitializeBuildStatus:
            Création de "cmTC_903a4.dir\Debug\cmTC_903a4.tlog\unsuccessfulbuild", car "AlwaysCreate" a été spécifié.
          ClCompile:
            C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\CL.exe /c /Zi /W1 /WX- /diagnostics:column /Od /Ob0 /D _MBCS /D WIN32 /D _WINDOWS /D "CMAKE_INTDIR=\"Debug\"" /Gm- /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"cmTC_903a4.dir\Debug\\" /Fd"cmTC_903a4.dir\Debug\vc142.pdb" /external:W1 /Gd /TC /errorReport:queue "C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\testCCompiler.c"
            Compilateur d'optimisation Microsoft (R) C/C++ version 19.29.30148 pour x64
            testCCompiler.c
            Copyright (C) Microsoft Corporation. Tous droits réservés.
            cl /c /Zi /W1 /WX- /diagnostics:column /Od /Ob0 /D _MBCS /D WIN32 /D _WINDOWS /D "CMAKE_INTDIR=\"Debug\"" /Gm- /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"cmTC_903a4.dir\Debug\\" /Fd"cmTC_903a4.dir\Debug\vc142.pdb" /external:W1 /Gd /TC /errorReport:queue "C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\testCCompiler.c"
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003: Impossible d'exécuter la tâche exécutable spécifiée "CL.exe". System.IO.DirectoryNotFoundException: Impossible de trouver une partie du chemin d'accès 'C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.dir\Debug\cmTC_903a4.tlog'. [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à System.IO.FileSystemEnumerableIterator`1.CommonInit() [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à System.IO.FileSystemEnumerableIterator`1..ctor(String path, String originalUserPath, String searchPattern, SearchOption searchOption, SearchResultHandler`1 resultHandler, Boolean checkHost) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à System.IO.Directory.GetFiles(String path, String searchPattern) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.Utilities.TrackedDependencies.ExpandWildcards(ITaskItem[] expand) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.Utilities.CanonicalTrackedOutputFiles.InternalConstruct(ITask ownerTask, ITaskItem[] tlogFiles, Boolean constructOutputsFromTLogs) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.Utilities.CanonicalTrackedOutputFiles..ctor(ITaskItem[] tlogFiles) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.CPPTasks.CL.PostExecuteTool(Int32 exitCode) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.CPPTasks.TrackedVCToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.Utilities.ToolTask.Execute() [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          Génération du projet "C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj" terminée (cibles par défaut) -- ÉCHEC.

          ÉCHEC de la build.

          "C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj" (cible par défaut) (1) ->
          (PrepareForBuild cible) ->
            C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppBuild.targets(517,5): warning MSB8029: Le répertoire intermédiaire ou le répertoire de sortie ne peut pas se trouver sous le répertoire temporaire car cela risque de créer des problèmes avec la génération incrémentielle. [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]


          "C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj" (cible par défaut) (1) ->
          (ClCompile cible) ->
            C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003: Impossible d'exécuter la tâche exécutable spécifiée "CL.exe". System.IO.DirectoryNotFoundException: Impossible de trouver une partie du chemin d'accès 'C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.dir\Debug\cmTC_903a4.tlog'. [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à System.IO.FileSystemEnumerableIterator`1.CommonInit() [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à System.IO.FileSystemEnumerableIterator`1..ctor(String path, String originalUserPath, String searchPattern, SearchOption searchOption, SearchResultHandler`1 resultHandler, Boolean checkHost) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à System.IO.Directory.GetFiles(String path, String searchPattern) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.Utilities.TrackedDependencies.ExpandWildcards(ITaskItem[] expand) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.Utilities.CanonicalTrackedOutputFiles.InternalConstruct(ITask ownerTask, ITaskItem[] tlogFiles, Boolean constructOutputsFromTLogs) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.Utilities.CanonicalTrackedOutputFiles..ctor(ITaskItem[] tlogFiles) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.CPPTasks.CL.PostExecuteTool(Int32 exitCode) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.CPPTasks.TrackedVCToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]
          C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    à Microsoft.Build.Utilities.ToolTask.Execute() [C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build\CMakeFiles\CMakeScratch\TryCompile-gyt569\cmTC_903a4.vcxproj]

              1 Avertissement(s)
              1 Erreur(s)

          Temps écoulé 00:00:00.82





        CMake will not be able to correctly generate this project.
      Call Stack (most recent call first):
        CMakeLists.txt:3 (project)


      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "C:\Users\Antoine\AppData\Local\Temp\pip-build-env-yc585726\overlay\Lib\site-packages\skbuild\setuptools_wrap.py", line 634, in setup
          env = cmkr.configure(
        File "C:\Users\Antoine\AppData\Local\Temp\pip-build-env-yc585726\overlay\Lib\site-packages\skbuild\cmaker.py", line 332, in configure
          raise SKBuildError(

      An error occurred while configuring with CMake.
        Command:
          'C:\Users\Antoine\AppData\Local\Temp\pip-build-env-yc585726\overlay\Lib\site-packages\cmake\data\bin/cmake.exe' 'C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea' -G 'Visual Studio 16 2019' '-DCMAKE_INSTALL_PREFIX:PATH=C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-install' -DPYTHON_VERSION_STRING:STRING=3.10.6 -DSKBUILD:INTERNAL=TRUE '-DCMAKE_MODULE_PATH:PATH=C:\Users\Antoine\AppData\Local\Temp\pip-build-env-yc585726\overlay\Lib\site-packages\skbuild\resources\cmake' '-DPYTHON_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPYTHON_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' '-DPYTHON_LIBRARY:PATH=D:\Anaconda\envs\textgen\libs\python310.lib' '-DPython_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPython_ROOT_DIR:PATH=D:\Anaconda\envs\textgen' '-DPython_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' -DPython_FIND_REGISTRY:STRING=NEVER '-DPython3_EXECUTABLE:PATH=D:\Anaconda\envs\textgen\python.exe' '-DPython3_ROOT_DIR:PATH=D:\Anaconda\envs\textgen' '-DPython3_INCLUDE_DIR:PATH=D:\Anaconda\envs\textgen\Include' -DPython3_FIND_REGISTRY:STRING=NEVER -T v142 -A x64 -DCMAKE_BUILD_TYPE:STRING=Release
        Source directory:
          C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea
        Working directory:
          C:\Users\Antoine\AppData\Local\Temp\pip-install-kab8dxp_\llama-cpp-python_6aab703992964fd9953365ad8cceacea\_skbuild\win-amd64-3.10\cmake-build
      Please see CMake's output for more information.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

[abetlen]

Hey @kerbi yes that's correct you need to essentially build it on the system that you're going to be running it on in order for the compiler to detect processor specific optimizations (SSE, AVX, AVX2, etc). These optimizations can have a significant impact on performance.

For debugging this cmake issue, can you try to seperately build the base llama.cpp library following the windows instructions here https://github.com/ggerganov/llama.cpp#usage

If you discover the correct msvc version / magic incantation to build this for windows from source I'll be happy to add it to the docs.

@MillionthOdin16 you're on a windows system do you have any suggestions on what I could add to the docs for setting up the right compiler on windows?

[hansfarrell]

I am having the exact same problem, "ERROR: Could not build wheels for llama-cpp-python which use PEP 517 and cannot be installed directly", do you know how to fix it specifically?

[kikikita]

The same problem.

[Dhaizei]

I have this issue in the ubuntu 18.04.

Building wheels for collected packages: llama-cpp-python, peft
Building wheel for llama-cpp-python (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [131 lines of output]

  --------------------------------------------------------------------------------
  -- Trying 'Ninja' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 7.5.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is GNU 7.5.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done (0.6s)
  -- Generating done (0.0s)
  -- Build files have been written to: /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/_cmake_test_compile/build
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Ninja' generator - success
  --------------------------------------------------------------------------------
  
  Configuring Project
    Working directory:
      /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/_skbuild/linux-x86_64-3.10/cmake-build
    Command:
      /tmp/pip-build-env-0es8vymx/overlay/lib/python3.10/site-packages/cmake/data/bin/cmake /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9 -G Ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/_skbuild/linux-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.9 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-0es8vymx/overlay/lib/python3.10/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/root/anaconda3/envs/textgen/bin/python -DPYTHON_INCLUDE_DIR:PATH=/root/anaconda3/envs/textgen/include/python3.10 -DPYTHON_LIBRARY:PATH=/root/anaconda3/envs/textgen/lib/libpython3.10.so -DPython_EXECUTABLE:PATH=/root/anaconda3/envs/textgen/bin/python -DPython_ROOT_DIR:PATH=/root/anaconda3/envs/textgen -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/root/anaconda3/envs/textgen/include/python3.10 -DPython3_EXECUTABLE:PATH=/root/anaconda3/envs/textgen/bin/python -DPython3_ROOT_DIR:PATH=/root/anaconda3/envs/textgen -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/root/anaconda3/envs/textgen/include/python3.10 -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-0es8vymx/overlay/lib/python3.10/site-packages/ninja/data/bin/ninja -DCMAKE_BUILD_TYPE:STRING=Release
  
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 7.5.0
  -- The CXX compiler identification is GNU 7.5.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  CMake Warning (dev) in CMakeLists.txt:
    A logical block opening on the line
  
      /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/CMakeLists.txt:9 (if)
  
    closes on the line
  
      /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/CMakeLists.txt:31 (endif)
  
    with mis-matching arguments.
  This warning is for project developers.  Use -Wno-dev to suppress it.
  
  -- Configuring done (0.5s)
  -- Generating done (0.0s)
  -- Build files have been written to: /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/_skbuild/linux-x86_64-3.10/cmake-build
  [1/2] Generating /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/vendor/llama.cpp/libllama.so
  FAILED: /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/vendor/llama.cpp/libllama.so
  cd /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/vendor/llama.cpp && make libllama.so
  I llama.cpp build info:
  I UNAME_S:  Linux
  I UNAME_P:  x86_64
  I UNAME_M:  x86_64
  I CFLAGS:   -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -march=native -mtune=native
  I CXXFLAGS: -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native
  I LDFLAGS:
  I CC:       cc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
  I CXX:      g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
  
  g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -c llama.cpp -o llama.o
  llama.cpp: In function ‘size_t llama_set_state_data(llama_context*, const uint8_t*)’:
  llama.cpp:2610:36: warning: cast from type ‘const uint8_t* {aka const unsigned char*}’ to type ‘void*’ casts away qualifiers [-Wcast-qual]
               kin3d->data = (void *) in;
                                      ^~
  llama.cpp:2614:36: warning: cast from type ‘const uint8_t* {aka const unsigned char*}’ to type ‘void*’ casts away qualifiers [-Wcast-qual]
               vin3d->data = (void *) in;
                                      ^~
  cc  -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -march=native -mtune=native   -c ggml.c -o ggml.o
  ggml.c: In function ‘ggml_vec_dot_q4_2_q8_0’:
  ggml.c:3253:40: warning: implicit declaration of function ‘_mm256_set_m128’; did you mean ‘_mm256_set_epi8’? [-Wimplicit-function-declaration]
           const __m256 d = _mm256_mul_ps(_mm256_set_m128(d1, d0), _mm256_broadcast_ss(&y[i].d));
                                          ^~~~~~~~~~~~~~~
                                          _mm256_set_epi8
  ggml.c:3253:40: error: incompatible type for argument 1 of ‘_mm256_mul_ps’
  In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:41:0,
                   from ggml.c:189:
  /usr/lib/gcc/x86_64-linux-gnu/7/include/avxintrin.h:318:1: note: expected ‘__m256 {aka __vector(8) float}’ but argument is of type ‘int’
   _mm256_mul_ps (__m256 __A, __m256 __B)
   ^~~~~~~~~~~~~
  ggml.c:3257:22: warning: implicit declaration of function ‘_mm256_set_m128i’; did you mean ‘_mm256_set_epi8’? [-Wimplicit-function-declaration]
           __m256i bx = _mm256_set_m128i(bx1, bx0);
                        ^~~~~~~~~~~~~~~~
                        _mm256_set_epi8
  ggml.c:3257:22: error: incompatible types when initializing type ‘__m256i {aka __vector(4) long long int}’ using type ‘int’
  Makefile:185: recipe for target 'ggml.o' failed
  make: *** [ggml.o] Error 1
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/tmp/pip-build-env-0es8vymx/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 674, in setup
      cmkr.make(make_args, install_target=cmake_install_target, env=env)
    File "/tmp/pip-build-env-0es8vymx/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 696, in make
      self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
    File "/tmp/pip-build-env-0es8vymx/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 741, in make_impl
      raise SKBuildError(msg)
  
  An error occurred while building with CMake.
    Command:
      /tmp/pip-build-env-0es8vymx/overlay/lib/python3.10/site-packages/cmake/data/bin/cmake --build . --target install --config Release --
    Install target:
      install
    Source directory:
      /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9
    Working directory:
      /tmp/pip-install-ipumiwzr/llama-cpp-python_f59a00faff7f4a7aa4cc7c5dbfdee0a9/_skbuild/linux-x86_64-3.10/cmake-build
  Please check the install target is valid and see CMake's output for more information.
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
Building wheel for peft (pyproject.toml) ... done
Created wheel for peft: filename=peft-0.4.0.dev0-py3-none-any.whl size=56159 sha256=2cd8e1b6a797ee1d131b056b6b420c821c8cef90233e7ba63db1a06d4194fc48
Stored in directory: /tmp/pip-ephem-wheel-cache-1kawfsbp/wheels/4c/16/67/1002a2d4daa822eff130e6d85b90051b75d2ce0d26b9448e4a
Successfully built peft
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

[gjmulder]

This is likely because you're building inside a virtual environment and HyperV has disabled Intel AVX extensions. See #244

[riyanswat]

The same problem while installing PrivateGPT on windows 10. :/

  error: subprocess-exited-with-error
  
  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [308 lines of output]
     
     
      --------------------------------------------------------------------------------
      -- Trying 'Ninja (Visual Studio 17 2022 x64 v143)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is unknown
      CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
        No CMAKE_C_COMPILER could be found.
     
        Tell CMake where to find the compiler by setting either the environment
        variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
        the compiler, or to the compiler name if it is in the PATH.
     
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja (Visual Studio 17 2022 x64 v143)' generator - failure
      --------------------------------------------------------------------------------
     
     
     
      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 17 2022 x64 v143' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Generator
     
          Visual Studio 17 2022
     
        could not find any instance of Visual Studio.
     
     
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 17 2022 x64 v143' generator - failure
      --------------------------------------------------------------------------------
     
     
     
      --------------------------------------------------------------------------------
      -- Trying 'Ninja (Visual Studio 16 2019 x64 v142)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is unknown
      CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
        No CMAKE_C_COMPILER could be found.
     
        Tell CMake where to find the compiler by setting either the environment
        variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
        the compiler, or to the compiler name if it is in the PATH.
     
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja (Visual Studio 16 2019 x64 v142)' generator - failure
      --------------------------------------------------------------------------------
     
     
     
      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 16 2019 x64 v142' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Generator
     
          Visual Studio 16 2019
     
        could not find any instance of Visual Studio.
     
      
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 16 2019 x64 v142' generator - failure
      --------------------------------------------------------------------------------
     
     
     
      --------------------------------------------------------------------------------
      -- Trying 'Ninja (Visual Studio 15 2017 x64 v141)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is unknown
      CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
        No CMAKE_C_COMPILER could be found.
     
        Tell CMake where to find the compiler by setting either the environment
        variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
        the compiler, or to the compiler name if it is in the PATH.
     
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja (Visual Studio 15 2017 x64 v141)' generator - failure
      --------------------------------------------------------------------------------
     
     
     
      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 15 2017 x64 v141' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Generator
     
          Visual Studio 15 2017
     
        could not find any instance of Visual Studio.
     
     
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 15 2017 x64 v141' generator - failure
      --------------------------------------------------------------------------------
     
     
     
      --------------------------------------------------------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 17 2022 x64 v143)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Running
     
         'nmake' '-?'
     
        failed with:
     
         The system cannot find the file specified
     
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 17 2022 x64 v143)' generator - failure
      --------------------------------------------------------------------------------
     
     
     
      --------------------------------------------------------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 16 2019 x64 v142)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Running
     
         'nmake' '-?'
     
        failed with:
     
         The system cannot find the file specified
     
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 16 2019 x64 v142)' generator - failure
      --------------------------------------------------------------------------------
     
      
     
      --------------------------------------------------------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 15 2017 x64 v141)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Running
     
         'nmake' '-?'
     
        failed with:
     
         The system cannot find the file specified
     
     
      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 15 2017 x64 v141)' generator - failure
      --------------------------------------------------------------------------------
     
                      ********************************************************************************
                      scikit-build could not get a working generator for your system. Aborting build.
     
                      Building windows wheels for Python 3.10 requires Microsoft Visual Studio 2022.
      Get it with "Visual Studio 2017":
     
        https://visualstudio.microsoft.com/vs/
     
      Or with "Visual Studio 2019":
     
          https://visualstudio.microsoft.com/vs/
     
      Or with "Visual Studio 2022":
      
          https://visualstudio.microsoft.com/vs/
      
                      ********************************************************************************
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
  Building wheel for hnswlib (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'hnswlib' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hnswlib
Failed to build llama-cpp-python hnswlib
ERROR: Could not build wheels for llama-cpp-python, hnswlib, which is required to install pyproject.toml-based projects        

[pgebert]

@riyanswat The error message says that it doesn't find any instance of Visual Studio (not to be confused with Visual Studio Code!). Can you try to install Visual Studio with C++ Build Tools and try it again?