pip should read configuration from `pyproject.toml` for local projects

[notatallshaw]

What's the problem this feature will solve?

Increasingly users expect tools to be able to read from their pyproject.toml for their local project. At this point pip stands as an outlier in not being able to define pip configuration in pyproject.toml.

Describe the solution you'd like

When installing a local package that has a pyproject.toml check for a section [tool.pip] and prioritize any configuration there as more important than configuration in environmental variables or configuration in the users environment (e.g. system directory, user directory, venv directory), but not as important as configuration passed on the command line.

Alternative Solutions

Do nothing

Additional context

This ticket is created to track this feature and point issues that require it to here. I think this would require a lot of design choices and I am not volunteering to lead it. But perhaps someone else might be sufficiently motivated!

Some immediate design choices that come to mind are:

• Most tools that offer pyproject.toml configuration in [tool.{tool-name}] also offer {tool-name}.toml configuration, would pip want to also read from a dedicated pip.toml?
• If pip supports pip.toml should it be possible to use a pip.toml in all the current places pip allows pip.[conf|ini] and in which case would it be the recommended format going forward?
• Should there be an option to specify the file location of the toml configuration file?
• For places where pip allows the user to provide a list of values via space separation would it make sense in the toml file to only allow this as an actual list?

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[RonnyPfannschmidt]

Without opt in this can be abused

[notatallshaw]

Without opt in this can be abused

Can you expand, this approach is already being used by other front end package tools such as uv, rye, poetry, hatch, etc.

[potiuk]

Without opt in this can be abused

Yeah. I think it's a good idea to add "pip frontend" configuration options for pyproject.toml users. And I cannot see easily scenario where it could be abused. It would likely require a new tool.pip section in pyproject.toml - which is unlikely to be used by anyone else.

When it comes to security/abuse - the only thing I see where It could be abused is when some external "tooling" (IDE's mainly) would automatically start using it and use it in some kind of security-related scenario. I think that could be mitigated by only adding configuration that is not "security-scenario-sensitive" to be handled by pyproject.toml. But there also might be other scenarios I have not thought about. Maybe you can elaborate and give some examples @RonnyPfannschmidt ?

[potiuk]

Also just to add a concrete "positive" scenario - I can imagine it would be super useful to have private package registries configured in pyproject.toml - so that the "corporate users" do not have to have any extra configuration (env vars etc.)

[pfmoore]

For abuse, how about a sdist that included an index URL in its pyproject.toml alongside a build requirement that looked OK, but was substituted with malicious code on the replacement index? Yes, if you're building a sdist you're already exposed, but this hides the risk and so makes it easier to fool an audit.

[potiuk]

For abuse, how about a sdist that included an index URL in its pyproject.toml alongside a build requirement that looked OK, but was substituted with malicious code on the replacement index? Yes, if you're building a sdist you're already exposed, but this hides the risk and so makes it easier to fool an audit.

Yes, I see that there are some potential "sensitive" properties, but I'd argue if those are declarative parameters are made part of published standard (and maybe even finding some way to mark such options are sensitive) it's far easier to make audit tools to support it, than for example analyse "hatch_build.py" code that can be triggered automatically by just adding "hatchling" as your build tool.

So I'd argue that increases the "auditability" rather than decreases it, because declarative specification is far easier to audit than dynamic python code (which is currently already supported).

[notatallshaw]

For abuse, how about a sdist that included an index URL in its pyproject.toml alongside a build requirement that looked OK, but was substituted with malicious code on the replacement index? Yes, if you're building a sdist you're already exposed, but this hides the risk and so makes it easier to fool an audit.

This proposal is for local projects, not packaged ones, so this scenario wouldn't apply.

And yes, sdists can already run arbitrary code, including calling pip with any any arguments.

[pfmoore]

This proposal is for local projects, not packaged ones

Sorry, I'd missed that. How would it work then? Would pip have to change its config settings mid-run, when it detected that it was installing from a local project? How would something like pip install ./foo ./bar work? Would it use the settings from foo/pyproject.toml or from bar/pyproject.toml? If both foo and bar depend on a named package baz, which project's settings would be used when searching for baz?

The point here is that pip currently reads its config at startup, which is before it's determined what (if any) local project(s) are being installed. So this would be a significant change to pip's config handling, to change it dynamically, during a single run.

I'm -1 on this as it stands. But I understand that it's a placeholder for someone who's interested in this feature to make a fully-specified proposal, so maybe it would be better to say that I doubt that it will be possible to come up with a satisfactory proposal for this.