Closed
Description
I'd like to ship a few wheels including pre-built C modules along with the musl-based standalone Python from here.
When I try to run python -m ensurepip
(e.g., #84) or any other imports that involves dynamic loading of C libraries (e.g., ctypes
, pyzmq
, etc.) then it exits with OSError: Dynamic loading not supported
.
Could we have a musl-based static build that allows dynamic loading of additional non-standard modules?
It seems that 0d3ae2a has disabled the ability to use dynamic loading.
Metadata
Metadata
Assignees
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
indygreg commentedon Sep 26, 2021
When you produce a fully static ELF executable, you can't load dynamic libraries. That
Dynamic loading not supported
error is coming from the bowels of libc I believe.Now, what we probably could do is produce a Python distribution that is dynamically linked against musl libc and the executable only has a dependency on musl. This should enable the distributions to load extensions that were also compiled to target musl libc.
We could potentially also include a placeholder shared library to force the resulting executable to be dynamic. Or maybe there's another way to force the linker into making the final executable dynamic.
Something to keep in mind here is once you support loading dynamic libraries, you have to worry about multiple libc versions being loaded. I believe musl - like glibc - is sensitive about multiple versions existing in the same process space. So we probably don't want to statically link musl and enable extensions to also statically link musl, as this is a recipe for disaster. Having a dynamically linked musl is probably the safest route here.
Out of curiosity, could you please explain your use case? Is this for better support for Alpine Linux and/or compatibility with the new platform tag for musl (https://discuss.python.org/t/pep-656-platform-tag-for-linux-distributions-using-musl/7165)?
achimnol commentedon Oct 12, 2021
Thanks for the reply.
My use case is to mount and execute a Python program with self-contained runtime inside arbitrary Docker containers based on arbitrary images. So far, it's been successful with glibc-flavor of static Python builds and glibc-based Docker images (e.g., Ubuntu/CentOS based images). But for Alpine-based images, a dynamic version of Python build works as expected while no static version did work because of the library import issue.
I think there are two ways if it's not feasible to build static-musl version of Python with import capability:
docs: document musl linking quirks
docs: document musl linking quirks
bigcat88 commentedon Feb 18, 2022
@indygreg can I help you anything to make musl dynamic build possible for x64 - aarch64?
Familiar with this(not with musl\Alpine itself, but with linkers and compilers), can try clone your repo and do that.
If you just point me in right direction. :)
As I understand I need:
And not related question:
my friend and i currently developing an open source project where we will use your python standalone, and we need to include some additional python packages in it, that not providing a binary wheels. We plan to do that in GitHub actions and remove all except install and licenses folder and add your License file to it. Do we need to include some additional info(except a link to your project on a main project readme page). Example: alpha_repack
andrey18106 commentedon Apr 9, 2022
Hi!
When will dynamic Python standalone be available on Alpine 3.13+?
bpcreech commentedon Mar 5, 2024
Just piping* up with another use case:
Using Hatch to run Python matrix tests, for wheels built on Alpine. Hatch uses
python-build-standalone
for this, plus pip to download wheels.Here is what it looks like when you try that today.
pip
fails, just like Joongi said.I see this patch,
patch-pip-static-binary.patch
, fixes thesite-packages/pip
which is bundled with thepython-build-standalone
distro, but that doesn't help because Python now runs the bundled pip wheel inlib/python3.12/ensurepip/_bundled/pip-24.0-py3-none-any.whl
which does not have that patch.Note that it's not just code that directly uses
ctypes
which is broken, but basically any use ofpip
, including hauling in deps for matrix testing.One could also upstream
patch-pip-static-binary.patch
to pip. I think that would fixpip
(and thus Hatch's matrix testing would work) but it wouldn't help for code under test which actually usesctypes
(which happens to include my use case anyway).(*piping up, not pip-ing up, because I can't pip. Sorry, bad pip joke.)
mwarkentin commentedon Feb 6, 2025
I ran into this when attempting to use
uv
inside of an Alpine docker container (which is the first "derived image" mentioned on https://docs.astral.sh/uv/guides/integration/docker/#available-images.My use case: working on a docker container that contains a grab bag of scripts that our team has built over time, and hoping to use
uv
to manage python versions and dependencies which may be different for each script, using the inline/// script
definitions.Will switch over to a Debian image for now instead.
zanieb commentedon Mar 12, 2025
#541 adds dynamic musl builds, this does not statically link libpython as described in the title of this issue, but we're tracking that separately, e.g., #540.