Manylinux/GCC versions

[0phoff]

I used cibuildwheel 1.1.0 in order to generate wheels for my library (which uses cython code) and have been using this for a few months already. 🎉

Now I received an issue from a user, where his program segfaults when importing my library (brambox) and a another one (pytorch). I'm still not sure what the cause is, but by googling around, I think it might be because I used manylinux1 and thus GCC 4.8, which is ABI incompatible with newer GCC versions.
The weird thing is that I have been using the official package of pytorch, which is reportedly compiled with GCC 4.9, without any issues... 🤔
The user in question used a pytorch version from his package manager (pacman - arch user), and there it crashes.
This leaves my quite puzzled and I don't really know what to do here. 😧

I checked out this repo and noticed the new version supports manylinux2010 and manylinux2014 which use newer GCC versions, so I might be tempted to migrate to using those.
However, as I do not fully understand all the implications of this, I would like to have your advice! 😇

• Are there any drawbacks to using the newer manylinux images ?
• Will my library still be compatible with packages like numpy and pandas, which use manylinux1 ?
• Is it a good idea to use these newer images, just for the sake of using them, or should we still stay on manylinux1 if possible ?

[YannickJadoul]

Hi @0phoff. I've not run into these kinds of issues or run into them, I'm afraid. Do you have any more information about this ABI incompatibility, and do you know what your user is using, then?

I can try to reply to your questions, but to be honest, maybe this is something you should ask in the manylinux repository (https://github.com/pypa/manylinux/). cibuildwheel mostly just takes the manylinux setup for granted and provides a tool around it, but as far as I know, no one of the regular contributors is a huge expert on this topic. (And the people at the manylinux repository are really friendly, I've noticed before ;-) )

Anyway, to the best of my knowledge: my answers (but take them with a pinch of salt!):

• Are there any drawbacks to using the newer manylinux images ?

The produced wheels are compatible with less versions of Linux (cfr. the glibc ABI versions, etc, manylinux1 vs. manylinux2010). That being said, being based on CentOS 5 and CentOS 6, those are released in 2007 and 2011, respectively. So manylinux2010 should still be many, many Linuxes ;-)

• Will my library still be compatible with packages like numpy and pandas, which use manylinux1 ?

They should, as far as I know. Unless you'd somehow be accessing internals that require full binary compatibility, I think different libraries should be independent.

• Is it a good idea to use these newer images, just for the sake of using them, or should we still stay on manylinux1 if possible ?

I think there were even security concerns about using the old, non-supported manylinux1 image (see pypa/manylinux#179 (comment)), so I think moving on to manylinux2010 is even encouraged.

So I think there's no problem if you want to switch to manylinux2010, as everyone will be doing it. Still, that doesn't mean manylinux1 images should have the issue you described, so it might still be good to take this up with the manylinux people?

[Czaki]

Also it is possible that when you switch to manylinux2010 it still will produce manylinux1 wheel next to manylinux2010. see https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image

[YannickJadoul]

Also it is possible that when you switch to manylinux2010 it still will produce manylinux1 wheel next to manylinux2010. see https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image

But that's (probably) not really relevant, right now, I think.

[0phoff]

Thanks for your quick replies, I will update to the new cibuildwheel version and use manylinux2010. I will also use this opportunity to build arm binaries with manylinux2014!