Skip to content

bpo-43950: optimize column table assembling with pre-sizing the column table #26997

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 2, 2021

Conversation

isidentical
Copy link
Member

@isidentical isidentical commented Jul 2, 2021

The new resizing system works like this;

$ cat t.py
a + a + a + b + c + a + a + a + b + c + a + a + a + b + c + a + a + a + b + c
[repeated 99 more times]
$ ./python t.py
RESIZE: prev len = 32, new len = 66
FINAL SIZE: 56
-----------------------------------------------------
RESIZE: prev len = 32, new len = 66
RESIZE: prev len = 66, new len = 134
RESIZE: prev len = 134, new len = 270
RESIZE: prev len = 270, new len = 542
RESIZE: prev len = 542, new len = 1086
RESIZE: prev len = 1086, new len = 2174
RESIZE: prev len = 2174, new len = 4350
RESIZE: prev len = 4350, new len = 8702
FINAL SIZE: 8004

So now we do considerably lower number of _PyBytes_Resize calls.

https://bugs.python.org/issue43950

Automerge-Triggered-By: GH:isidentical

Copy link
Member

@pablogsal pablogsal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can confirm that this speeds up the allocation quite a lot as expected 👍

❯ time ./python -m test test_compile -m test_stack_overflow -v
== CPython 3.11.0a0 (heads/optimize-co-positions:760daf9858, Jul 2 2021, 19:08:05) [GCC 11.1.0]
== Linux-5.12.13-arch1-2-x86_64-with-glibc2.33 little-endian
== cwd: /home/pablogsal/github/python/main/build/test_python_15988æ
== CPU count: 36
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 2.15 Run tests sequentially
0:00:00 load avg: 2.15 [1/1] test_compile
test_stack_overflow (test.test_compile.TestSpecifics) ... ok

----------------------------------------------------------------------

Ran 1 test in 3.945s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 4.0 sec
Tests result: SUCCESS
./python -m test test_compile -m test_stack_overflow -v  3.77s user 0.47s system 99% cpu 4.241 total

@miss-islington
Copy link
Contributor

@isidentical: Status check is done, and it's a success ❌ .

@isidentical
Copy link
Member Author

concurrent_futures error, irrelevant, restarting the CI

@miss-islington
Copy link
Contributor

@isidentical: Status check is done, and it's a success ✅ .

@miss-islington
Copy link
Contributor

Sorry, I can't merge this PR. Reason: 3 of 5 required status checks have not succeeded: 2 expected..

@miss-islington
Copy link
Contributor

@isidentical: Status check is done, and it's a success ✅ .

@miss-islington miss-islington merged commit ec8759b into python:main Jul 2, 2021
@isidentical isidentical changed the title bpo-43950: optimize column table assembling with pre-sizing object bpo-43950: optimize column table assembling with pre-sizing the column table Jul 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants