Trusted Publishing attestations missing

[letmaik]

Describe the bug
I updated my packages to use trusted publishing and found that some .whl files return 200, some 404 when retrieving provenance.

PR: https://github.com/letmaik/pyvirtualcam/pull/132/files

200: https://pypi.org/integrity/pyvirtualcam/0.13.0/pyvirtualcam-0.13.0-cp310-cp310-macosx_11_0_arm64.whl/provenance
404: https://pypi.org/integrity/pyvirtualcam/0.13.0/pyvirtualcam-0.13.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/provenance

The job that generates and uploads the provenance didn't show any errors:
https://github.com/letmaik/pyvirtualcam/actions/runs/14983471840/job/42098613667

Expected behavior
All .whl files should have attestations uploaded to PyPI.

To Reproduce
Not sure. See linked PR.

My Platform
GitHub Actions

Additional context
I notice this behavior in all my three packages that I upgraded: pyvirtualcam, rawpy, lensfunpy.

[di]

404: https://pypi.org/integrity/pyvirtualcam/0.13.0/pyvirtualcam-0.13.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/provenance

In https://github.com/letmaik/pyvirtualcam/actions/runs/14983471840/job/42098613667 I see your workflow generating a publish attestation for a file named pyvirtualcam-0.13.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

DSSE PAE: b'DSSEv1 28 application/vnd.in-toto+json 313 {"_type":"https://in-toto.io/Statement/v1","subject":[{"name":"pyvirtualcam-0.13.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl","digest":{"sha256":"95fd2fb36b59708bce3d5ff6e67a44c31570490fe5aab0ac1cd60ef374a38551"}}],"predicateType":"https://docs.pypi.org/attestations/publish/v1","predicate":null}'

but it's not in the list of uploaded files at the end of that workflow run:

Uploading distributions to https://upload.pypi.org/legacy/
Uploading pyvirtualcam-0.13.0-cp310-cp310-macosx_10_9_x86_64.whl
Uploading pyvirtualcam-0.13.0-cp310-cp310-macosx_11_0_arm64.whl
Uploading 
pyvirtualcam-0.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Uploading 
pyvirtualcam-0.13.0-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.wh
l
Uploading pyvirtualcam-0.13.0-cp310-cp310-win_amd64.whl
Uploading pyvirtualcam-0.13.0-cp311-cp311-macosx_10_9_x86_64.whl
Uploading pyvirtualcam-0.13.0-cp311-cp311-macosx_11_0_arm64.whl
Uploading 
pyvirtualcam-0.13.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Uploading 
pyvirtualcam-0.13.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.wh
l
Uploading pyvirtualcam-0.13.0-cp311-cp311-win_amd64.whl
Uploading pyvirtualcam-0.13.0-cp312-cp312-macosx_10_9_x86_64.whl
Uploading pyvirtualcam-0.13.0-cp312-cp312-macosx_11_0_arm64.whl
Uploading 
pyvirtualcam-0.13.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Uploading 
pyvirtualcam-0.13.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.wh
l
Uploading pyvirtualcam-0.13.0-cp312-cp312-win_amd64.whl
Uploading pyvirtualcam-0.13.0-cp313-cp313-macosx_10_9_x86_64.whl
Uploading pyvirtualcam-0.13.0-cp313-cp313-macosx_11_0_arm64.whl
Uploading 
pyvirtualcam-0.13.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Uploading 
pyvirtualcam-0.13.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.wh
l
Uploading pyvirtualcam-0.13.0-cp[313](https://github.com/letmaik/pyvirtualcam/actions/runs/14983471840/job/42098613667#step:3:319)-cp313-win_amd64.whl
Uploading pyvirtualcam-0.13.0-cp39-cp39-macosx_10_9_x86_64.whl
Uploading 
pyvirtualcam-0.13.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Uploading 
pyvirtualcam-0.13.0-cp39-cp39-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl
Uploading pyvirtualcam-0.13.0-cp39-cp39-win_amd64.whl

View at:
https://pypi.org/project/pyvirtualcam/0.13.0/

...nor does it appear at https://pypi.org/project/pyvirtualcam/0.13.0/#files.

Instead, I see pyvirtualcam-0.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl, which has the platform tags swapped. This file does have a publish attestation: https://pypi.org/project/pyvirtualcam/0.13.0/#pyvirtualcam-0.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl, which has the same SHA (95fd2fb36b59708bce3d5ff6e67a44c31570490fe5aab0ac1cd60ef374a38551) but the subject name is the original filename, not the swapped filename.

So, PyPI is behaving as expected, but something between the attestation generation step and the upload step is renaming the wheel files with compressed tag sets. @webknjaz @woodruffw does that ring any bells?

[di]

Actually, perhaps PyPI should behave slightly better here and reject attestations for which the subject name doesn't match the corresponding filename?

[woodruffw]

So, PyPI is behaving as expected, but something between the attestation generation step and the upload step is renaming the wheel files with compressed tag sets. @webknjaz @woodruffw does that ring any bells?

I think this is product of some last-second normalization within pypi-attestations: pypi-attestations "ultranormalizes" the distribution's filename by ordering the compressed tag set if present, since that's the one part of the wheel filename specification that isn't already canonical.

That happens here:

https://github.com/trailofbits/pypi-attestations/blob/a5e7f0373dbdbaf45ead9b2a04c892259cf2253f/src/pypi_attestations/_impl.py#L387

In practice this is strictly superfluous, since PEP 740 says that attestation verification is done by comparing the subject (i.e. dist filename) by parsing, rather than assuming string equality. So, we could probably remove this entirely 🙂

TL:DR: This is probably happening in pypi-attestations and can be removed without breakage, since it's a conservative step beyond what PEP 740 requires. At the same time, I'm curious how it got surfaced here since the "ultranormalized" form only occurs within the attestation itself, while PyPI uses the form that it receives. @letmaik how did you end up with that .whl URL?

[di]

Per https://peps.python.org/pep-0425/, these tags should be sorted, so I think the 'right' filename is pyvirtualcam-0.13.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

[di]

PyPI should probably start rejecting wheels where the tag sets are not ordered as well: #18129

But we should also determine what's creating the non-compliant wheel filenames in the first place.

[di]

Based on https://github.com/letmaik/pyvirtualcam/actions/runs/14983471840/job/42092719292#step:4:357, looks like auditwheel is the one rewriting the tags:

+ auditwheel repair dist-tmp/pyvirtualcam-0.13.0-cp39-cp39-linux_x86_64.whl -w dist
INFO:auditwheel.main_repair:Repairing pyvirtualcam-0.13.0-cp39-cp39-linux_x86_64.whl
INFO:auditwheel.wheeltools:Previous filename tags: linux_x86_64
INFO:auditwheel.wheeltools:New filename tags: manylinux_2_17_x86_64, manylinux2014_x86_64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp39-cp39-linux_x86_64
INFO:auditwheel.wheeltools:New WHEEL info tags: cp39-cp39-manylinux_2_17_x86_64, cp39-cp39-manylinux2014_x86_64
INFO:auditwheel.main_repair:
Fixed-up wheel written to /io/dist/pyvirtualcam-0.13.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

I filed pypa/auditwheel#583 as a result.

Closing this in favor of pypa/auditwheel#583, #18129 and trailofbits/pypi-attestations#123.