Attempt to use pkg-config even on Windows

[SpaceManiac]

The buildscript currently never attempts pkg-config on Windows, even though MSYS2 provides pkg-config and a libcurl package. This need not be the case.

This is especially relevant because compiling curl from unpatched source under MSYS2 is tricky because MSYS2 does not provide the strtok_r function, leading to errors of this sort:

...\target\release\deps\libcurl_sys-d0dfe3f28a72a4ef.rlib(libcurl_la-cookie.o):cookie.c:
(.text$Curl_cookie_add+0x152): undefined reference to `strtok_r'

[alexcrichton]

Thanks for the PR! This is actually intentional though because using reams of code from MSYS2 ends up not playing nicely sometimes.

Unfortunately the standard installer for Rust that targets MinGW also installs an instance of gcc.exe. When the Rust compiler links binaries it then uses this gcc.exe instance, and then it also prioritizes using bundled versions of libraries with Rust rather than the system.

This ends up meaning that if you have the bundled gcc.exe instsalled, you can compile C code with one runtime and link it with another, causing lots of problems! I believe your original error (an undefined reference to strtok_r) can be fixed by uninstalling the Rust-bundled MinGW gcc.exe (it's an option in the installer).

I'm somewhat hesitant to use libcurl/pkg-config from the system as well as I've generally had not much luck in the past with doing so. I'm a little hazy on the exact details right now but would uninstalling the gcc.exe that Rust installs suffice for you for now?

[SpaceManiac]

Ah, I think it's perhaps the other way around - compiling curl during the build step finds the headers from MSYS2, where strtok_r exists, and then links against the bundled MinGW libraries which lack it. Deleting all the $toolchain/lib/rustlib/$target/lib/*.a files allowed curl to be built again. While the option not to install this may exist in the .msi installer, it is sadly missing from rustup.

Is it even possible for the gcc crate to compile the curl dependency without a full (i.e. more than the bundled) MinGW toolchain installed? (Discounting -msvc, which I would hope not to break but which shouldn't be affected because pkg-config wouldn't exist.)

I'm also curious specifically what kind of issues you've run into when using MSYS2's packages.

[alexcrichton]

Support should be coming soon to rustup to configure this! But yes this library requires a full MinGW installation. The gcc.exe we ship is only used as a linker, not a C compiler.

I believe one issue with MSYS2 packages is that they're compiled with the pthread threading model rather than the Win32 one which Rust needs? I'm not sure exactly how that played out though.

[crlf0710]

Hi, i just come across this issue as well. I'm using MSYS2 dist and used to using rustup, cargo and other tools under cmd.exe, so libcurl 's sh ./configure doesn't execute correctly. Never mind, that's another story.

I think the problem mentioned in this thread is as follows:

• We CAN use windows-gnu version of rust (at least almost) flawlessly with w64-mingw32 versions of prebuilt packages with corresponding target cpu. We can argue that there're chances that the gcc-libs or other dependencies versions are different, compiler bugs etc. However then it's user's duty to make them consistent. If the user can't get a consistent version. Then they'll have to build one. (I still think prepare a environment compatible to that of rust-mingw, write a PKGBUILD file and makepkg and install with pacman is much much more desirable)

• We CAN NOT use windows-gnu version of rust with anything built within the real MSYS2 runtime (those not prefixed with x86_64-w64-mingw32 etc) and use the corresponding gcc toolchain, because essentially that's a very different game in the name of POSIX. As you mentioned: "they're compiled with the pthread threading model rather than the Win32", etc.

To ensure this, I think:

• The gcc crate: when using windows-gnu toolchain of rust, shall always execute the corresponding x86_64-w64-mingw32-gcc.exe or i686-w64-mingw32-gcc.exe. It is not allowed to use the MSYS2\usr\bin\gcc.exe, which is actually x86_64-pc-msys-gcc.exe.

• The pkg-config crate: when using windows-gnu toolchain of rust, shall always execute the corresponding x86_64-w64-mingw32-pkg-config.exe or i686-w64-mingw32-pkg-config.exe. It is not allowed to use the MSYS2\usr\bin\pkg-config.exe, which is actually x86_64-pc-msys-gcc.exe.
  (those pkg-config.exe can manage the library directories themselves)

After that, we're always safe to use gcc crate and pkg-config crate to resolve those dependencies.
If unsuccessful, download and compile and build from source as a fallback.

[alexcrichton]

@crlf0710 the incompatibilities aren't just with misusing the wrong compiler or such, but rather the Rust distribution itself. Accepting a change like this would make curl-sys stop compiling by default on many Windows installations of Rust due to the installed rust-mingw component.

[crlf0710]

@alexcrichton Yes, I understand your concerns about this, and i also think that now is not the right time for accepting this.

One thing i don't really understand is that, even in the rust-mingw case, There's no pkg-config.exe or libcurl.a shipped with it at all. Why would that be a problem?

I believe that what pkg-config(-rs) does (or should do) is exactly to probe and provide the correctly dependencies if they exist. If they do not exist or are not compatible, pkg-config should not return anything.

Well, i'm not complaining, but it seems to me that something's broken... Now i'm also not sure what is the correct thing to do... But...rust is actually still linking things from my mingw64 dist because of other crates, like libz or things like that and... it's just weird to see that just this crate is requiring a library compilation locally even if i've installed libcurl.a and libcurl-4.dll (they're compatible with the only gcc.exe i'm able to use) and i'm not sure whether some crate else is actually using that copy directly or indirectly...

I don't know what to do now... What do you think?

[alexcrichton]

I believe the problem happens when:

1. You have the rust-mingw component installed
2. You're running in an MSYS environment with pkg-config and libcurl.a
3. You attempt to link in libcurl.a, but you also link in older support of libc and such that's in rust-mingw

That causes undefined symbols and such because libcurl.a is compiled against a different version of the libraries that are being linked.

[SpaceManiac]

What I do to avert the problem is uninstall the rust-mingw component. Unfortunately rustup doesn't make that easy.

[crlf0710]

@alexcrichton @SpaceManiac
Thanks for the explanation! Then the situation is much better than what i thought. Thanks again.

[alexcrichton]

@SpaceManiac yeah that's what I'd prefer to happen by default. We're in sort of an unfortunate situation here... Currently though we're basically in a state where landing a PR like this would break curl-rust by default on many MinGW installations, so to land this we'd have to change the current state somehow. Unfortunately I'm not entirely sure how :(

[alexcrichton]

Closing due to inactivity