Pip issue during environment creation

[mestinso]

I've been using PythonCall successfully for the past few months. Not sure what exactly triggered this issue, but all of a sudden I'm getting a strange issue during environment creation related to pip:

And if I manually take a look in windows explorer, the scripts folder is not present, so the error makes sense. So then the question is why isn't the scripts folder being created? Or potentially it's being created but quickly deleted? Haven't investigated that second possibility. Either way, any ideas what might be going on or how to resolve? Or is there any way to fully reset pythoncall and try again?

[cjdoris]

I recently saw the exact same error in CI. Are you on windows? What version of MicroMamba and micromamba_jll are you on?

[mestinso]

Glad to hear it's not just me, and yes I am on windows.

I usually keep pretty up to date. The versions I have are:
MicrcoMamba v0.1.12
micromamba_jll v1.4.3+0

Do let me know if there are any tests I can run to support debugging. Thanks.

[cjdoris]

Could you try various versions of micromamba_jll please to see which ones don't work?

[mestinso]

ok, I was actually able to test this surprisingly quickly.

micromamba 1.4.0 and 1.4.1 work.
1.4.2 seems nonexistent (doesn't want to install when I try).
and 1.4.3 does not work

[mestinso]

Resolved via JuliaPy/MicroMamba.jl@2ade9cf

Thanks!

[mestinso]

Note that I found Pkg.jl has some odd behavior such that depending on what other packages you have installed, it won't use MicroMamba 0.1.13...even though it can (marked as upgradeable). You may have to explicitly install MicroMamba 0.1.13 to resolve this issue.

For example, if I only have PythonCall.jl in my Project.toml, MicroMamba 0.1.13 (and micromamba_jll 1.4.1) will be used (as expected). However, as an example, if I also have SciMLSensitivity.jl in my Project.toml, MicroMamba 0.1.12 (and micromamba_jll 1.4.3) will be used instead. For these cases, I am having to explicitly install the desired version of MicroMamba to not hit the errors as mentioned in my first post. Note that for the example given (SciMLSensitivity), it has zero dependencies on PythonCall, CondaPkg, MicroMamba, or micromamba_jll. As far as I can tell it's just a weird corner case in Pkg.jl's logic for determining which packages to upgrade or not. And in this case, it's preferring using an older version of MicroMamba and a newer version of micromamba_jll rather than the opposite. In my mind this is arguably a bug in Pkg.jl, but I just wanted to leave this note here in case others were still getting this issue.

[cjdoris]

Yes indeed, the "fix" in MicroMamba was largely just an experiment. I'm still figuring out the best way to solve this issue properly - I might just get the offending version of micromamba_jll yanked from the registry.

I don't think this is a bug in Pkg, it's just having to make an arbitrary choice between two optimal solutions.

[mestinso]

Right, so not to make this a Pkg.jl discussion, but while I agree the two Pkg solutions are both correct from the compatibility rules standpoint, I would argue:
a) It shouldn't feel random or unpredictable which of those two solutions it chooses.
b) Since MicroMamba depends on micromamba_jll (and not the other way around), it seems natural that MicroMamba should have the precedence for being up to date first.

[cjdoris]

That's a solid argument - seems like a reasonable way to break ties. You could make an issue over on Pkg.jl.