rules_python is failing with Bazel@HEAD in downstream CI

[sgowroji]

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3107#0188d169-3b4f-4f76-85d8-ad27beaf313b

Platforms : MacOS, Windows, Linux

Logs:

"py2_runtime": attr.label(
--
  | Error in label: Illegal argument: element in 'providers' is of unexpected type. Either all elements should be providers, or all elements should be lists of providers, but got an element of type Provider.

Steps :

git clone -v https://github.com/bazelbuild/rules_python.git
git reset 643a14bf80179b2eb96de022344b850047788b51 --hard
export USE_BAZEL_VERSION=38ce7e18aec2ba078edfbbc5245c656d2c6312e0
cd examples/bzlmod
bazel build ...

Culprit : 18aeb52a59e65efcbeac7041faf60d81f9d434cb

CC Greenteam @fweikert

[rickeylev]

This is probably a bug in Bazel. It looks like a recurrence of the bug where builtins injection doesn't occur during workspace evaluation. It looks like this might be bzlmod specific?

fyi to @comius @Wyverald

[rickeylev]

Yeah, hacking up the providers = values a bit, this is because the PyRuntimeInfo and PyInfo objects are instances of ProviderStub.

Per bazelbuild/bazel#17713 (comment), workspace was fixed to inject builtins, but not bzlmod.

[rickeylev]

For posterity (I'll be unavailable for a bit in a couple days): We might be able to avoid this error by carefullly restructuring who-loads-what. Basically make sure none of the bzlmod stuff directly or indirectly loads a file with a rule implementation referencing the PyInfo and PyRuntimeInfo providers. This would be good hygiene, anyways; extensions and repo rules have no need for regular rule definitions. From some initial poking, I think it's the py_import rule somehow coming in through pip that is the cause.

[meteorcloudy]

Fixed by bazelbuild/bazel@c2553cc

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3127