Tracking issue: PyPI dependencies pixi can't manage yet. #771
Comments
|
Mediapipe
|
|
I wanted to use this library that is packaged in pypi: https://github.com/spirali/elsie Here is the error I was seeing when trying to use pixi add: Posting in the discord channel got it working with this fix: The reason for this is:
|
|
@pablovela5620 I am also finding this necessary quite often, probably more often than @ruben-arts would like :) In this pattern the environment definition is fragmented between |
@liquidcarbon This is indeed not what we want as UX but we simply need to develop more to support all PyPI packages. Its a weird bunch of requirements we have to support to be equivalent to |
|
@liquidcarbon the |
|
@liquidcarbon I cannot reproduce this, neither on OSX or on windows. It's strange it does not select the |
|
@tylerjw actually it also fails for the same version in pip on apple silicon, like pip we error out when we cannot build the first source distribution. |
|
@pablovela5620 so it seems mediapipe 10.9 is a package with an invalid RECORD file, I manually checked it and it's incorrect. This is mentioned in the PyPa
Mediapipe has a Which in this case triggers the error, I'm unsure why pip does no do this. But I feel its good to adhere to the standard here. In any case, mediapipe 10.8 does seem to work, you could use that instead. Also see: google-ai-edge/mediapipe#5025 |
These PR address 2 major issues and 1 small issue * zip file stamps issue alexcrichton/tar-rs#349 which we encountered when installing tomli-2.1.0 * changed from_filename signature so it can also produce Stree, which it's needed from pixi side * lowercase Root-Is-Purelib so we can install elsie library ( prefix-dev/pixi#771 (comment))
@tdejager Just tried on Win10, original comment was on Win11 PS C:\Users\a\Desktop\code\duckdb-pip> pixi add python=3.11 I was on Pixi 0.9.1, upped to 0.13.0 - same thing But |
|
The conda package you add with https://prefix.dev/channels/conda-forge/packages/python-duckdb |
|
@ruben-arts noted -- I just wasn't sure which part @tdejager was trying to reproduce |
|
Tensorflow metal on Apple silicon MacOS 14.2.1: pixi init tf-metal && cd tf-metal
pixi add "python>=3.11" "tensorflow>=2.13" pip
pixi add --pypi tensorflow-metal
> × failed to resolve `pypi-dependencies`, due to underlying error
╰─▶ The following packages are incompatible
└─ tensorflow-metal * cannot be installed because there are no viable options:
└─ tensorflow-metal 0.1.0 | 0.1.1 | 0.1.2 | 0.2.0 | 0.3.0 | 0.4.0 | 0.5.0 | 0.5.1 | 0.6.0 | 0.7.0 | 0.7.1
| 0.8.0 | 1.0.0 | 1.0.1 | 1.1.0 is excluded because none of the artifacts are compatible with the Python
interpreter or glibc version and there are no supported sdists
pixi run pip install tensorflow-metal
> Successfully installed tensorflow-metal-1.1.0 |
|
Most likely you are missing a system requirement: https://pixi.sh/latest/configuration/#the-system-requirements-table Most likely macos=12.0 |
|
Nice, that fixed it. EDIT: Nevermind, I found the example demonstrating how to do this. Thank you! |
|
Installing (base) ➜ idtracker pixi add --pypi opencv-python-headless
× failed to resolve `pypi-dependencies`, due to underlying error
╰─▶ No metadata could be extracted for the following available artifacts:
- opencv-python-headless-4.9.0.80.tar.gz
Error: × error while processing source distribution 'opencv-python-headless-4.9.0.80.tar.gz':
│ could not build wheel: Traceback (most recent call last):
│ File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/build_frontend.py", line 124, in <module>
│ get_requires_for_build_wheel(backend, work_dir)
│ File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/build_frontend.py", line 58, in get_requires_for_build_wheel
│ result = f()
│ ^^^
│ File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in
│ get_requires_for_build_wheel
│ return self._get_build_requires(config_settings, requirements=['wheel'])
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
│ File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
│ self.run_setup()
│ File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in run_setup
│ super().run_setup(setup_script=setup_script)
│ File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
│ exec(code, locals())
│ File "<string>", line 10, in <module>
│ ModuleNotFoundError: No module named 'skbuild'
│
help: Probably an error during processing of source distributions. Please check the error message above. |
|
@roaldarbol The error message is absolutely terrible but if you add: [system-requirements]
macos = "11.0"It should work. |
|
It does indeed! Thanks! |
|
Trying to install
|
It does indeed not work. Testing the repro gives me this error: In both the Thanks for the info @jacobbieker |
Would be kind-of useful if we can keep the build environments for uv as well, so it's easier to debug these things. |
pixi add --pypi -vv "trimesh[all]"
Log
Yes.
The above log shows that the problem seems to be with |
This happens on the osx-64 and osx-arm64 platforms (it works on win-64 and linux-64). I expect this to be some mismatching in the name/metadata of/in the .whl files for the macos specific ones.
Not yet, it would help if the resolver can be more verbose ( Encountered in Deltares/Ribasim#1137 |
|
@evetion that error often means that the Unfortunately, setting that in my |
|
Sorry, my bad. We should make the error more actionable and figure out a better default behavior. |
|
@liblaf I wonder if for you it's a similar issue and adding glibc 2.28 or higher would fix it. e.g. |
|
@ruben-arts got around to test this again and now I'm not seeing this error anymore 🤷. |
|
On @ruben-arts do you prefer this as a new issue or here? Thanks! fyi @xela-95 |
@ruben-arts no hurry on our side, but just to understand, do you prefer this bug report as a comment here or on another issue, thanks! |
|
@traversaro That problem has to do with the system requirements. I really want to get rid of this issue through #346 Adding the following to the Edit: sometimes it helps to look at https://pypi.org/simple/drake/ available wheels. |
|
Thanks @ruben-arts, this fixed our problem! Cross-linking #346 that seems to be the actual root issue here. |
|
Hello, uploading my Error message❯ pixi install
WARN The feature 'notebooks' is defined but not used in any environment
WARN The feature 'matrepr' is defined but not used in any environment
× failed to solve the pypi requirements of 'mlir-dev' 'linux-64'
├─▶ failed to resolve pypi dependencies
╰─▶ Because only finch-mlir<=0.0.2 is available and finch-mlir==0.0.2 has no wheels with a matching Python implementation tag, we can
conclude that finch-mlir>=0.0.2 cannot be used.
And because you require finch-mlir>=0.0.2, we can conclude that your requirements are unsatisfiable.pixi.toml[project]
authors = ["Hameer Abbasi <[email protected]>"]
channels = ["conda-forge"]
name = "sparse"
platforms = ["osx-arm64", "osx-64", "linux-64", "win-64"]
[pypi-dependencies]
sparse = { path = ".", editable = true }
[system-requirements]
libc = "2.35"
[dependencies]
python = ">=3.10"
numpy = ">=1.17"
numba = ">=0.49"
[feature.extras.dependencies]
dask = ">=2024"
scipy = ">=0.19"
scikit-learn = "*"
[feature.docs.pypi-dependencies]
mkdocs-material = "*"
mkdocstrings = { version = "*", extras = ["python"] }
mkdocs-gen-files = "*"
mkdocs-literate-nav = "*"
mkdocs-section-index = "*"
mkdocs-jupyter = "*"
[feature.tests.tasks]
test = "pytest --pyargs sparse -n auto"
test-mlir = { cmd = "pytest --pyargs sparse.mlir_backend -v" }
test-finch = { cmd = "pytest --pyargs sparse/tests -n auto -v", depends-on = ["precompile"] }
[feature.tests.dependencies]
pytest = ">=3.5"
pytest-cov = "*"
pytest-xdist = "*"
pre-commit = "*"
pytest-codspeed = "*"
[feature.notebooks.dependencies]
nbmake = "*"
matplotlib = "*"
[feature.matrepr.dependencies]
matrepr = "*"
[feature.finch.tasks]
precompile = "python -c 'import finch'"
[feature.finch.pypi-dependencies]
scipy = ">=0.19"
finch-tensor = ">=0.1.31"
[feature.finch.activation.env]
SPARSE_BACKEND = "Finch"
[feature.finch.target.osx-arm64.activation.env]
SPARSE_BACKEND = "Finch"
PYTHONFAULTHANDLER = "${HOME}/faulthandler.log"
[feature.mlir.dependencies]
scipy = ">=0.19"
[feature.mlir.pypi-dependencies]
finch-mlir = ">=0.0.2"
[feature.mlir.activation.env]
SPARSE_BACKEND = "MLIR"
[environments]
tests = ["tests", "extras"]
docs = ["docs", "extras"]
mlir-dev = {features = ["tests", "mlir"], no-default-feature = true}
finch-dev = {features = ["tests", "finch"], no-default-feature = true}To reproduce simply run |
|
The pypi dependency |
|
Hey @enrico5,
For my machine that looks like this: ❯ pixi info | grep glibc
: __glibc=2.35=0Adding the given version to the [tool.pixi.system-requirements]
libc = "2.35" |
|
That works great. Thank you! |
Did you find a workaround, if so please explain.
|
|
having some weirdness with rerun-sdk, feedstock isnt yet merged for latest 0.22.1 update. If I try to add via cli but if I directly link to the url it works just fine |
Kind OT, but anyhow: I pinged in conda-forge/rerun-sdk-feedstock#61, feel free to ping yourself in the future if something like this happens, thanks! |
|
Yes, I should have started with that!
|
|
|
The hint is relevant: Probably you can disable build-isolation for |
Thank you, it helped. I overlooked the thing about build isolation, got confused after seeing uv mentioned, sorry. It would be great though if there would be a message from pixi with the link you provided, but not sure it worth investing. |
|
When trying to install a Example
|
|
Yes, you are right this is kind of a big issue we should pass the environment markers correctly to uv. Which we currently do not seem to do. |
|
I have to have things working on both Windows and Linux so this one is definitely biting me. It's not a show-stopper though - I just make those deps |
|
Aha okay! Good to know 😄 |
|
Also ran into this issue, and the proposed workaround breaks compatibility with for example direct Any estimate on if/when this will be fixed? |
and others
There are a few PyPI packages
pixican't install yet, wherepipcan.Please paste your examples in this issue so we have a list of known packages we can track, test and benchmark with along the way.
Information we would like:
What did you run and what was the outcome?
e.g.
pixi add --pypi packagexIf it doesn't recreate in an empty environment please share your
pixi.tomlthat recreates the issue.What error did
pixireturn?e.g.
Can pip install the package?
Does
pip install packagexwork?What platform are you on?
e.g.
linux-64Did you find a workaround, if so please explain.
e.g. build it into a conda package, using a custom fork, etc.
Your input would greatly help us improve the pixi's experience!
Thanks in advance! ❤️
The text was updated successfully, but these errors were encountered: