Skip to content

src build on windows not working #12947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gregorsoll opened this issue May 31, 2023 · 24 comments
Open

src build on windows not working #12947

gregorsoll opened this issue May 31, 2023 · 24 comments
Assignees
Labels

Comments

@gregorsoll
Copy link

What version of protobuf and what language are you using?
Version: main/v3.6.0/v3.5.0 etc. (NOTE: please try updating to the latest version of protoc/runtime possible beforehand to attempt to resolve your problem)
Language: C++/Java/Python/C#/Ruby/PHP/Objective-C/Javascript

What operating system (Linux, Windows, ...) and version?
Windows

What runtime / compiler are you using (e.g., python version or gcc version)
VC 14.16.27023

What did you do?
Steps to reproduce the behavior:

  1. cloned src and submodules
  2. installed bazel, c++ buildtools
  3. bazel build :protoc :protobuf

What did you expect to see
build completed

What did you see instead?
C:/users/xyz/_bazel_xyz/y7fcwcs2/external/com_google_protobuf/src/google/protobuf/compiler/csharp/BUILD.bazel:22:11: Compiling src/google/protobuf/compiler/csharp/csharp_repeated_enum_field.cc [for tool] failed: (Exit 2): cl.exe failed: error executing command (from target @com_google_protobuf//src/google/protobuf/compiler/csharp:csharp) C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 65 arguments skipped)
cl : Command line warning D9002 : ignoring unknown option '-std=c++14'
external/com_google_protobuf/src/google/protobuf/compiler/csharp/csharp_repeated_enum_field.cc(31): fatal error C1083: Cannot open include file: 'google/protobuf/compiler/csharp/csharp_repeated_enum_field.h': No such file or directory
ERROR: C:/users/xyz/_bazel_xyz/y7fcwcs2/external/com_google_protobuf/src/google/protobuf/compiler/BUILD.bazel:74:11: Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool] failed: (Exit 2): cl.exe failed: error executing command (from target @com_google_protobuf//src/google/protobuf/compiler:command_line_interface) C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 68 arguments skipped)
cl : Command line warning D9002 : ignoring unknown option '-std=c++14'
external/com_google_protobuf/src/google/protobuf/compiler/command_line_interface.cc(35): fatal error C1083: Cannot open include file: 'google/protobuf/compiler/command_line_interface.h': No such file or directory
ERROR: C:/users/xyz/_bazel_xyz/y7fcwcs2/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/BUILD.bazel:53:11: Compiling src/google/protobuf/compiler/objectivec/primitive_field.cc [for tool] failed: (Exit 2): cl.exe failed: error executing command (from target @com_google_protobuf//src/google/protobuf/compiler/objectivec:objectivec) C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 67 arguments skipped)
cl : Command line warning D9002 : ignoring unknown option '-std=c++14'
external/com_google_protobuf/src/google/protobuf/compiler/objectivec/primitive_field.cc(31): fatal error C1083: Cannot open include file: 'google/protobuf/compiler/objectivec/primitive_field.h': No such file or directory
INFO: Elapsed time: 4.749s, Critical Path: 0.59s
INFO: 29 processes: 29 internal.
FAILED: Build did NOT complete successfully

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

Anything else we should know about your project / environment

@gregorsoll gregorsoll added the untriaged auto added to all issues by default when created. label May 31, 2023
@deannagarcia
Copy link
Member

It looks like the protobuf version you're using is pretty old and is no longer supported. Can you upgrade to the newest version (23.2) and see if you still see the issue?

@laramiel
Copy link
Contributor

laramiel commented Jun 14, 2023

I ran into the same issue trying to upgrade to 23.x and here's my diagnosis:

MSVC cl.exe has a limitation (https://developercommunity.visualstudio.com/t/clexe-compiler-driver-cannot-handle-long-file-path/975889) on filename lengths of 260 characters (MAX_PATH). There appears to be no workaround using UNC paths.

Within Protobuf there are a number of source files organized like this:

https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/objectivec/BUILD.bazel

with rules like (abbreviated):

cc_library(
    name = "objectivec",
    srcs = [ ... ],
    hdrs = [
        ...
        "text_format_decode_data.h",
    ],
    include_prefix = "google/protobuf/compiler/objectivec",
)

When bazel encounters rules like this it constructs include paths to provide the include_prefix mapping similar to this:

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/_virtual_includes/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

Counting the hash, using fastbuild for the config, but not counting <bazel_outout_base> & <root workspace>, this filename is already at 215 characters.

That leaves a maximum of 40 characters for the other parts.

It's pretty easy to run out of that. The default output base for my username is >30 characters.

One option to fix this is to reorganize the src/google/protobuf/compiler files to remove the nesting.

One workaround which I am using now is to set a bazel startup option to gain that 30 characters back: --output_base=C:\O, but that's probably not a great long-term solution.

See related bazel issues:

bazelbuild/bazel#15961
bazelbuild/bazel#4149
bazelbuild/bazel#18683

@deannagarcia deannagarcia removed the untriaged auto added to all issues by default when created. label Jun 14, 2023
copybara-service bot pushed a commit to google/tensorstore that referenced this issue Jun 14, 2023
protocolbuffers/protobuf#12947

PiperOrigin-RevId: 540401592
Change-Id: I8998cf6927ad3a59ce296fd6a815eb63d104d1d1
copybara-service bot pushed a commit to google/tensorstore that referenced this issue Jun 14, 2023
This saves 10 characters in build paths.
protocolbuffers/protobuf#12947

PiperOrigin-RevId: 540416043
Change-Id: I5c4abeb751e7ee476d9f183f381e4f8dbe0a0a93
@laramiel
Copy link
Contributor

laramiel commented Jun 15, 2023

I tried to rename my dependency from com_google_protobuf to protobuf. There are two things which encumber that.

  1. The built-in repository @bazel_tools references @com_google_protobuf://protoc. There does not appear to be a way to override the repo_mapping for this repository to make it reference something else. There is a repository https://github.com/bazelbuild/rules_proto which maybe bazel is using to externalize proto rules, but it doesn't seem very active. I'm not sure of the state of that.

  2. grpc binds 'protobuf' already. This may not be the place to lament the style of the grpc bazel rules, but their extensive use of bind complicates my attempt to get a hermetic environment where my repository controls all the dependencies and does not allow dependencies to load third-party repositories.

So it turns out that it's a bit challenging to save characters simply by renaming the repository.

copybara-service bot pushed a commit to google/tensorstore that referenced this issue Jun 15, 2023
Upgrade protobuf upb to corresponding v23.x branch
Upgrade grpc to v1.55.0

NOTE: -DTENSORSTORE_USE_SYSTEM_PROTOBUF=ON doesn't appear to work properly to build gRPC after this change. I suspect that we need to generate calls to `include_directories(${Protobuf_INCLUDE_DIRS})` for protobuf dependencies, however bazel_to_cmake doesn't really have a mechanism for that.

NOTE: While building on windows I ran into:
fatal error C1083: Cannot open include file: 'google/protobuf/compiler/objectivec/text_format_decode_data.h': No such file or directory

If this happens, then underlying issue is that MSVC cannot handle long path lengths. See related bugs:
* bazelbuild/bazel#18683
* protocolbuffers/protobuf#12947

PiperOrigin-RevId: 540626233
Change-Id: I0887d0126bab695956c9d11793ac7c6764537d79
laramiel added a commit to google/tensorstore that referenced this issue Jun 16, 2023
copybara-service bot pushed a commit to google/tensorstore that referenced this issue Jun 16, 2023
protocolbuffers/protobuf#12947

PiperOrigin-RevId: 540800426
Change-Id: I67793ade781c0f77b93ae2b343b8a1c6914e278a
@hlopko
Copy link
Contributor

hlopko commented Jun 19, 2023

@meteorcloudy could you recommend the best solution for us?

@meteorcloudy
Copy link
Contributor

Using a shorter output base / output user root is the only direct workaround I can think of as recommended at https://bazel.build/configure/windows#long-path-issues

@oquenchil Is it possible to create the virtual_includes dir under some path that the length is O(1) instead of O(package path length)?

@hlopko
Copy link
Contributor

hlopko commented Jun 19, 2023

CC @jskeet, would it be an option to not use include_prefix in the C# BUILD files at all?

@jskeet
Copy link
Contributor

jskeet commented Jun 19, 2023

@hlopko: I'm afraid I'm really not the person to ask about this - my understanding of both C++ and Bazel is pretty low. I have no objection to it being changed, but it's not something I'd have confidence either doing or reviewing.

@meteorcloudy
Copy link
Contributor

I guess we can at least try and see if it breaks any tests? ;)

@oquenchil
Copy link

@oquenchil Is it possible to create the virtual_includes dir under some path that the length is O(1) instead of O(package path length)?

I don't think so.

  • Assuming source code cannot be changed the directory structure must be as listed on the include directive
  • The symlink must be in a folder belonging to the package of the target creating the symlink

That doesn't leave out anything that can be shortened using a hash like we do for shared libraries on Windows.

I can't think of anything except what's already been mentioned: not using include_prefix or shorter output base.

@meteorcloudy
Copy link
Contributor

meteorcloudy commented Jun 20, 2023

The symlink must be in a folder belonging to the package of the target creating the symlink

This is currently also a package path right? Can we replace it with a hash?

Essentially

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/_virtual_includes/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

becomes

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/_virtual_includes/<hash of package path>/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

@laramiel
Copy link
Contributor

I think that this would require a bazel change; basically bazel would need to be able to hand out such a directory with a shorter prefix. I don't know what that would entail.

I filed a feature request for that: bazelbuild/bazel#18683

It was recategorized to c++ rules, which may not be a great category.

@hlopko hlopko added c# and removed c# labels Jul 13, 2023
@hlopko
Copy link
Contributor

hlopko commented Jul 13, 2023

Looking around, it seems that the reason to have all the 73 instances if include_prefix in the repo is easier compatibility with the Google internal clone of this repository. One solution here would be to fix header include paths in all C++ sources so we don't need include_prefix at all.

CC @mkruskal-google

@zhangyanjun2020
Copy link

I ran into the same issue trying to upgrade to 23.x and here's my diagnosis:

MSVC cl.exe has a limitation (https://developercommunity.visualstudio.com/t/clexe-compiler-driver-cannot-handle-long-file-path/975889) on filename lengths of 260 characters (MAX_PATH). There appears to be no workaround using UNC paths.

Within Protobuf there are a number of source files organized like this:

https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/objectivec/BUILD.bazel

with rules like (abbreviated):

cc_library(
    name = "objectivec",
    srcs = [ ... ],
    hdrs = [
        ...
        "text_format_decode_data.h",
    ],
    include_prefix = "google/protobuf/compiler/objectivec",
)

When bazel encounters rules like this it constructs include paths to provide the include_prefix mapping similar to this:

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/_virtual_includes/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

Counting the hash, using fastbuild for the config, but not counting <bazel_outout_base> & <root workspace>, this filename is already at 215 characters.

That leaves a maximum of 40 characters for the other parts.

It's pretty easy to run out of that. The default output base for my username is >30 characters.

One option to fix this is to reorganize the src/google/protobuf/compiler files to remove the nesting.

One workaround which I am using now is to set a bazel startup option to gain that 30 characters back: --output_base=C:\O, but that's probably not a great long-term solution.

See related bazel issues:

bazelbuild/bazel#15961 bazelbuild/bazel#4149 bazelbuild/bazel#18683

I ran into the same issue trying to upgrade to 23.x and here's my diagnosis:

MSVC cl.exe has a limitation (https://developercommunity.visualstudio.com/t/clexe-compiler-driver-cannot-handle-long-file-path/975889) on filename lengths of 260 characters (MAX_PATH). There appears to be no workaround using UNC paths.

Within Protobuf there are a number of source files organized like this:

https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/objectivec/BUILD.bazel

with rules like (abbreviated):

cc_library(
    name = "objectivec",
    srcs = [ ... ],
    hdrs = [
        ...
        "text_format_decode_data.h",
    ],
    include_prefix = "google/protobuf/compiler/objectivec",
)

When bazel encounters rules like this it constructs include paths to provide the include_prefix mapping similar to this:

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/_virtual_includes/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

Counting the hash, using fastbuild for the config, but not counting <bazel_outout_base> & <root workspace>, this filename is already at 215 characters.

That leaves a maximum of 40 characters for the other parts.

It's pretty easy to run out of that. The default output base for my username is >30 characters.

One option to fix this is to reorganize the src/google/protobuf/compiler files to remove the nesting.

One workaround which I am using now is to set a bazel startup option to gain that 30 characters back: --output_base=C:\O, but that's probably not a great long-term solution.

See related bazel issues:

bazelbuild/bazel#15961 bazelbuild/bazel#4149 bazelbuild/bazel#18683

i try it do work, thanks

copybara-service bot pushed a commit to tink-crypto/tink-cc that referenced this issue Nov 29, 2023
This requires:
 * Tentatively abandoning the use of Bzlmod. Protobuf doesn't support bzlmod yet [1] and the latest available protobuf in the Bazel Central Registry is 21.7 which provides incompatible JSON APIs [2]. Moving forward, we can re-introduce bzlmod when protobuf and Abseil officially support it.
 * Using the `--output_base=C:\O ` Bazel statup option on Windows [3] when building for Kokoro tests.

[1] protocolbuffers/protobuf#14313
[2] https://github.com/bazelbuild/bazel-central-registry/tree/7004deef786dc4152eedd3dfedf91b5b6b4590d4/modules/protobuf
[3] protocolbuffers/protobuf#12947

PiperOrigin-RevId: 586313367
Change-Id: Idc3f6d5a6e34b7da8fe51ea42ca354027a94a369
tholenst pushed a commit to tink-crypto/tink that referenced this issue Dec 4, 2023
This requires:
 * Tentatively abandoning the use of Bzlmod. Protobuf doesn't support bzlmod yet [1] and the latest available protobuf in the Bazel Central Registry is 21.7 which provides incompatible JSON APIs [2]. Moving forward, we can re-introduce bzlmod when protobuf and Abseil officially support it.
 * Using the `--output_base=C:\O ` Bazel statup option on Windows [3] when building for Kokoro tests.

[1] protocolbuffers/protobuf#14313
[2] https://github.com/bazelbuild/bazel-central-registry/tree/7004deef786dc4152eedd3dfedf91b5b6b4590d4/modules/protobuf
[3] protocolbuffers/protobuf#12947

PiperOrigin-RevId: 586313367
copybara-service bot pushed a commit to google/tensorstore that referenced this issue Dec 7, 2023
As of Sept, upb has moved into protobuf; update to support that.

This reworks how protos are generated to feel a bit closer to the aspect-like code generation of Bazel. Now each proto generator is registered, and each proto_library() rule will invoke all registered aspects; this will allow consistent and reliable invocation of protoc across all sub-repositories.

The change permits removing duplicate .proto files from local_proto_mirror, and simplifies the boostrapping process.

Also note, on windows the updated protobuf is more likely to trigger file not found issues due to path length; if this happens, add the bazel startup option:

  startup --output_user_root=C:/_out

See also:

bazelbuild/bazel#18683
protocolbuffers/protobuf#12947
grpc/grpc#33986

PiperOrigin-RevId: 588606143
Change-Id: Iafa5f16b0a9bbc1058a876c0cc739d82743f96a4
@freeform-andre
Copy link
Contributor

I'm running into this issue on v28.3 even with a short output base. The only other cause I can think of is that I'm building without runfiles enabled as my windows environment does not have symlinks enabled. If there is a way to eliminate the virtual imports or fix whatever issue in upstream rules_cc that'd be fantastic.

phst added a commit to phst/rules_elisp that referenced this issue Jan 4, 2025
The setup-bazel action sets the output base on Windows to D:\_bazel, which
should be short enough to avoid
protocolbuffers/protobuf#12947.
phst added a commit to phst/rules_elisp that referenced this issue Jan 4, 2025
The setup-bazel action sets the output base on Windows to D:\_bazel, which
should be short enough to avoid
protocolbuffers/protobuf#12947.
Copy link

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

@github-actions github-actions bot added the inactive Denotes the issue/PR has not seen activity in the last 90 days. label Feb 24, 2025
@esrauchg esrauchg removed the inactive Denotes the issue/PR has not seen activity in the last 90 days. label Feb 24, 2025
@esrauchg
Copy link

esrauchg commented Feb 24, 2025

The severe file length limits under by bazel+msvc are a known painpoint, not just for external users but also for the protobuf team's development itself (we have been warping our own file names and directory structure to keep our max length of paths to be approx 60 chars, with remainder of the 260 character limit is burned by other overhead).

Since we don't have any good way to mitigate this issue, and because bazel+msvc usage is low compared to other ways to build on Windows, we are planning to drop support for bazel+msvc in Protobuf 34 (the Q1 2026 release). Bazel 30 already disables support for this combination, with a flag to reenable support for it which gives time to gather feedback on this as well as give folks a longer horizon to migrate their builds to either make+msvc or bazel+clang-cl which don't suffer from this artificial limitation without being stuck on Protobuf 29 or lower:

https://protobuf.dev/news/v30/#poison-msvc--bazel

my windows environment does not have symlinks enabled

Unfortunately a lot of bazel behaviors are inherently symlink related (besides what was mentioned above, also e.g. strip_prefix on proto library), I don't think it'll be a winning battle to use bazel on windows without symlinks.

If you need to build on an env without symlinks, I think you probably want look into using Make builds on that environment; I'm actually unsure if it also will still have a different set of problems in that env, but at least the most obvious behaviors that are inherently symlink-related are in bazel behaviors.

@neilconway
Copy link

@esrauchg The engdoc.corp.google.com link you posted is not publicly accessible.

@esrauchg
Copy link

Apologies, accidentally used the incorrect url: corrected to https://protobuf.dev/news/v30/#poison-msvc--bazel

@crt-31
Copy link

crt-31 commented Mar 1, 2025

and because bazel+msvc usage is low compared to other ways to build on Windows,

Um, I don't know if this is necessarily a proper statement. Regardless, bazel+msvc should be a supported configuration as it has been. Please don't 'drop support'... lets find a way forward, I know we can.

mbland added a commit to mbland/rules_scala that referenced this issue Mar 1, 2025
Keeps the `README` guidance in sync with what we're actually using in
`WORKSPACE` for consistency's sake.

@crt-31 and I found that the Windows build failure for bazel-contrib#1710 mentioned
in the earlier commit is related to the Windows/MSVC file path length
limit. `src/google/protobuf/compiler/java/java_features.pb.h`, the path
specified in the error message, doesn't exist until `protobuf` v25.0.

- protocolbuffers/protobuf#12947

Furthermore, the Protobuf team currently plans to just drop MSVC
support:

- https://protobuf.dev/news/v30/#poison-msvc--bazel
- protocolbuffers/protobuf#20085

I plan to experiment again with "Protobuf Toolchainization", which I'd
tried in October when beginning the Bzlmod experiment. Here are some
interesting background resources before I dig in on that:

- bazelbuild/rules_proto#213
- bazelbuild/rules_proto#179
- https://github.com/bazelbuild/rules_proto/releases/tag/6.0.0
- https://github.com/aspect-build/toolchains_protoc/
- protocolbuffers/protobuf#20182
- protocolbuffers/protobuf#19679
- protocolbuffers/protobuf#19558
mbland added a commit to mbland/rules_scala that referenced this issue Mar 1, 2025
Registers a precompiled protocol compiler toolchain when
`--incompatible_enable_proto_toolchain_resolution` is `True`.
Part of bazel-contrib#1482 and bazel-contrib#1652.

Stops `protoc` recompilation, and fixes the build breakage in bazel-contrib#1710 due
to `protobuf` include paths exceeding the Visual Studio path length
limit.

The updates to `scala_proto/scala_proto_toolchain.bzl` were inspired by:

- protocolbuffers/protobuf: bazel: Remove hardcoded dependency on
  //:protoc from language runtimes #19679
  protocolbuffers/protobuf#19679

The `proto_lang_toolchain` call was inspired by the `README` from:

- https://github.com/aspect-build/toolchains_protoc/

Adds `scripts/update_protoc_integrity.py` to automatically update
`scala/private/protoc/protoc_integrity.bzl`.

This should make builds of `rules_scala` much faster all around. Given
the fact that this feature depends on recent `protobuf` versions, and
the Windows `protobuf` build breaks without it, we have a catch-22. It
likely can't be separated from the rest of bazel-contrib#1710, though I would prefer
that.

It also seems likely that we'd eventually need to do this to continue
supporting Windows, per:

- protocolbuffers/protobuf#12947
- https://protobuf.dev/news/v30/#poison-msvc--bazel
- protocolbuffers/protobuf#20085

More background on proto toolchainization:

- Proto Toolchainisation Design Doc
  https://docs.google.com/document/d/1CE6wJHNfKbUPBr7-mmk_0Yo3a4TaqcTPE0OWNuQkhPs/edit

- bazelbuild/bazel: Protobuf repo recompilation sensitivity
  bazelbuild/bazel#7095

- bazelbuild/rules_proto: Implement proto toolchainisation
  bazelbuild/rules_proto#179

- rules_proto 6.0.0 release notes mentioning Protobuf Toolchainisation
  https://github.com/bazelbuild/rules_proto/releases/tag/6.0.0
@esrauchg
Copy link

esrauchg commented Mar 3, 2025

Please provide any feedback or questions about our ongoing bazel+msvc support on #20085

Note that my short comment didn't reflect all of the effort and thinking that went into it before taking a decision to move in that direction: we do not take the decision lightly, but unfortunately have not been able to find any sufficient technical solution to the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests