chore: update to Python 3.9

[miketheman]

Resolves: #10807

[miketheman]

Test failure is for coverage missing a branch-jump for a raise, will need to investigate further.

[miketheman]

I'm a bit stymied here.

Current output:

+ python -m coverage report -m --fail-under 100
Name                        Stmts   Miss Branch BrPart  Cover   Missing
-----------------------------------------------------------------------
warehouse/search/tasks.py     112      0     20      1    99%   199->225
-----------------------------------------------------------------------
TOTAL                        9104      0   2551      1    99%

Here's a link to the segment in question, with some added context:

https://github.com/pypa/warehouse/blob/d2cafdb6b59a5a0c53931cb280e1322659bd4cf6/warehouse/search/tasks.py#L185-L226

If I read the coverage report correctly, it's telling us that the request.db.close() in the finally block isn't being exercised due to the raise on L196 , so the code jumps all the way down to L225, the except clause.

I've tried upgrading coverage to the latest 6.3.2 (just in case), but am still getting the same result. 🤔

The only test I was able to track down that would be relevant seemed to be test_fails_when_raising - but wasn't able to figure out why coverage reports the segment covered in 3.8, but not in 3.9.

Any ideas?

[miketheman]

A little more from htmlcov:

[di]

Looks like this is nedbat/coveragepy#1041

[di]

(I'm fine with adding a # pragma: no cover here if that will suppress the failure)

[miketheman]

@di thanks for the linkage, I've added another commit disabling coverage reporting for that step.

[miketheman]

@ewdurbin Thanks! How does one find out when this is released, and live? I was looking at https://p.datadoghq.com/sb/7dc8b3250-85dcf667bd - hoping to see if there was better CPU/memory utilization, bit I don't think that's currently graphed.

[ewdurbin]

You can see it going live in the "PyPI Backend HTTP Requests (req/sec)" when the color of the graph changes (those show releases).

We don't publicly chart cpu/mem for the services, and I'm not sure that we should necessarily (as it gives insight if someone wants to DoS us), but here's a snapshot of the last four hours:

I wouldn't take much away from that on resource utilization since the workloads on PyPI backends do vary hour by hour based on what's going on (heavy XMLRPC use, large quantity of new releases, etc).

But overall looks pretty much equivalent, so that's good.

Is there a particularly reason you expected to see a difference in utilization?

[miketheman]

Thanks for sharing the image! Totally understand why it's not public.

Is there a particularly reason you expected to see a difference in utilization?

I read there was a memory leak fixed with 3.9 where multiple instantiations of Python - but I may have been off.

[abitrolly]

In somebody want to DoS PyPI, absence of public metrics won't stop them. On the other hand public metrics not only help to build DoS resilient service, but also help to bring decent performance.