server: Windows 7 compatibility

[Zor-X-L]

I want to run the server on a Win7 box, and the API calls to CreateFile2/CreateFileMappingFromApp/MapViewOfFileFromApp in httplib.h prevent this (need Win8 at least), so I change them to CreateFileW/CreateFileMappingW/MapViewOfFile.

I tested the code on Win7 with qwen2-0_5b-instruct-q4_k_m (about 400MiB) and on Win11 with Yi-1.5-34B-Chat-IQ4_XS (about 18GiB), they all work fine.

In theory it can run on XP or Vista, but I don't have test environments.

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[ngxson]

I'm not very confident with this change, because httplib is copied from external repo: https://github.com/yhirose/cpp-httplib

All the changes made here in llama.cpp will be overwritten when we sync with upstream source code

[Zor-X-L]

@ngxson the upstream repo has fixed the issue (yhirose/cpp-httplib#1869), set -D_WIN32_WINNT=_WIN32_WINNT_WIN7 -DWINVER=_WIN32_WINNT_WIN7 will make it call the proper API on Win7.

After I update the httplib to the latest version, I find GGML_WIN_VER was set to 0x602 which is Win8 because PrefetchVirtualMemory. But the call to PrefetchVirtualMemory is dynamic so actually it doesn't need -D_WIN32_WINNT=_WIN32_WINNT_WIN7, but to define WIN32_MEMORY_RANGE_ENTRY and PWIN32_MEMORY_RANGE_ENTRY when _WIN32_WINNT < _WIN32_WINNT_WIN8.

I set GGML_WIN_VER to _WIN32_WINNT_WIN7 (0x601 doesn't work, dunno why) and add _WIN32_WINNT=${GGML_WIN_VER} WINVER=${GGML_WIN_VER} to places that need it. Makefile also edited.

I compile the project on win11 with vs2022 and msys2-mingw64 and both executables works on win7 (msys2-ucrt doesn't work).

What do you think?

[ngxson]

Sorry for the late response. As I'm not using windows (and I don't know any of llama.cpp's collaborators use windows 7), I can't verify if this PR have any side effects or not. I can only merge if the change involve upgrading to upstream httplib. Hope you understand that. In case there's nobody who can test this PR, we can still keep this as a demo.

[ngxson]

I'm not sure if this has any effects on other versions of windows

[Zor-X-L]

I'm not sure if this has any effects on other versions of windows

According to https://learn.microsoft.com/en-us/cpp/porting/modifying-winver-and-win32-winnt: "The preprocessor macros WINVER and _WIN32_WINNT specify the minimum operating system version your code supports".

This will prevent windows sdk header file declare functions only available on higher windows versions. So as long as the project can compile, it should have no effects on other windows versions.

[sharpden]

2567073 works on my Windows 7, thank you!

[vignesh1507]

works in mine too, thx for making the changes in the code.

[mhnoni]

I want to run the server on a Win7 box, and the API calls to CreateFile2/CreateFileMappingFromApp/MapViewOfFileFromApp in httplib.h prevent this (need Win8 at least), so I change them to CreateFileW/CreateFileMappingW/MapViewOfFile.

I tested the code on Win7 with qwen2-0_5b-instruct-q4_k_m (about 400MiB) and on Win11 with Yi-1.5-34B-Chat-IQ4_XS (about 18GiB), they all work fine.

In theory it can run on XP or Vista, but I don't have test environments.

Hi Zor-X-L

Any chance you remember which cuda version you used to compile it? or you only test it on CPU? I was able to build the old version with the help of your pr for CPU but not cuda.

And do you think you can mak a pr for the latest version too for win 7?

Thanks!

[sharpden]

I want to run the server on a Win7 box, and the API calls to CreateFile2/CreateFileMappingFromApp/MapViewOfFileFromApp in httplib.h prevent this (need Win8 at least), so I change them to CreateFileW/CreateFileMappingW/MapViewOfFile.
I tested the code on Win7 with qwen2-0_5b-instruct-q4_k_m (about 400MiB) and on Win11 with Yi-1.5-34B-Chat-IQ4_XS (about 18GiB), they all work fine.
In theory it can run on XP or Vista, but I don't have test environments.

Hi Zor-X-L

Any chance you remember which cuda version you used to compile it? or you only test it on CPU? I was able to build the old version with the help of your pr for CPU but not cuda.

And do you think you can mak a pr for the latest version too for win 7?

Thanks!

Last CUDA that officially supports Windows 7 is 10.2, but build with 10.2 is probably broken long ago #4123
(For XP x32 - 6.5, x64 - 6.0)

[mhnoni]

Last CUDA that officially supports Windows 7 is 10.2, but build with 10.2 is probably broken long ago #4123 (For XP x32 - 6.5, x64 - 6.0)

oh, that makes sense, thanks, I spent my whole day trying to find the issue and I couldn't I though I have some issue on my side, IO was getting "nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified" and I thought I have the wrong cuda, I even tried to compile it using 11.2 on windows 10 but couldn't either (the old version of llama.cpp that supported win 7). I guess its not easy anymore to let llama.cpp to support win 7. I wonder if there is any other runtimes that I could use with win 7 that support cuda.

[zhaofeng-shu33]

I want to run the server on a Win7 box, and the API calls to CreateFile2/CreateFileMappingFromApp/MapViewOfFileFromApp in httplib.h prevent this (need Win8 at least), so I change them to CreateFileW/CreateFileMappingW/MapViewOfFile.
I tested the code on Win7 with qwen2-0_5b-instruct-q4_k_m (about 400MiB) and on Win11 with Yi-1.5-34B-Chat-IQ4_XS (about 18GiB), they all work fine.
In theory it can run on XP or Vista, but I don't have test environments.

Hi Zor-X-L

Any chance you remember which cuda version you used to compile it? or you only test it on CPU? I was able to build the old version with the help of your pr for CPU but not cuda.

And do you think you can mak a pr for the latest version too for win 7?

Thanks!

The latest version has merge conflict with this pr, but I think these conflicts are not difficult to solve.
As others methoned, it is difficult to support win7+cuda build version, but win7+pure cpu can be reserved.I Based on this pr, I suggest add some "CMake switch" to differentiate cuda build and non-cuda windows build. By doing so, all CICD can pass.

[sharpden]

I have successfully built quite recent b4568 with patches only for 4 CMakeFiles.txt and 1 Makefile. @Zor-X-L , @ngxson , could you please merge this PR already? 8 months should be enough for 9 lines change.

[mhnoni]

I have successfully built quite recent b4568 with patches only for 4 CMakeFiles.txt and 1 Makefile. @Zor-X-L , @ngxson , could you please merge this PR already? 8 months should be enough for 9 lines change.

Could you please link the pr? does your pr let us compile it and run on windows 7 including the webclient?
And which cuda version we will need for it? my problem last time trying to compile for Win 7 was with cuda.

[zhaofeng-shu33]

I have successfully built quite recent b4568 with patches only for 4 CMakeFiles.txt and 1 Makefile. @Zor-X-L , @ngxson , could you please merge this PR already? 8 months should be enough for 9 lines change.

Could you please link the pr? does your pr let us compile it and run on windows 7 including the webclient? And which cuda version we will need for it? my problem last time trying to compile for Win 7 was with cuda.

Please remove the following lines in CMakeLists.txt

if(MSVC)
    add_compile_options("$<$<C_COMPILER_ID:MSVC>:/utf-8>")
    add_compile_options("$<$<CXX_COMPILER_ID:MSVC>:/utf-8>")
endif()

nvcc from cuda 10.2 cannot recognize /utf8 option.

I have successfully compile this PR version using VS2017 + Cuda 10.2 on win7.
Some other macros also neeed to be patched.