bpo-45609: More specialization stats for STORE_SUBSCR

[sweeneyde]

https://bugs.python.org/issue45609

[sweeneyde]

With these stats added:

crypto_pyaes:

STORE_SUBSCR:
 unquickened:        4342 0.1%
    deferred:       10852 0.3%
         hit:     3539698 99.6%
  success:        1140
  failure:         225
    kind  0:        1 0.4%    (other)
    kind  4:       23 10.2%   (out of range)
    kind 18:      154 68.4%   (bytearray[int])
    kind 20:       47 20.9%   (python __setitem__)

bm_django_template:

STORE_SUBSCR:
 unquickened:       14576 0.5%
    deferred:      729034 23.2%
         hit:     2397457 76.3%
  success:        1590
  failure:       11531
    kind  0:        1 0.0%    (other)
    kind  4:       59 0.5%    (out of range)
    kind 10:       12 0.1%    (list[slice])
    kind 18:      814 7.1%    (bytearray[int])
    kind 20:     5364 46.5%   (python __setitem__)
    kind 22:     5281 45.8%   (dict subclass)

bm_fannkuch:

STORE_SUBSCR:
 unquickened:        4224 0.0%
    deferred:    57141937 66.2%
         hit:    29193702 33.8%
  success:         740
  failure:      892908
    kind  0:        1 0.0%    (other)
    kind  4:       23 0.0%    (out of range)
    kind 10:   892683 100.0%  (list[slice])
    kind 18:      154 0.0%    (bytearray[int])
    kind 20:       47 0.0%    (python __setitem__)

bm_meteor_contest:

STORE_SUBSCR:
 unquickened:        4049 0.0%
    deferred:       10852 0.1%
         hit:     8507905 99.8%
  success:         729
  failure:         225
    kind  0:        1 0.4%    (other)
    kind  4:       23 10.2%   (out of range)
    kind 18:      154 68.4%   (bytearray[int])
    kind 20:       47 20.9%   (python __setitem__)

bm_nbody:

STORE_SUBSCR:
 unquickened:        4478 0.0%
    deferred:       10852 0.0%
         hit:    49515720 100.0%
  success:         850
  failure:         225
    kind  0:        1 0.4%    (other)
    kind  4:       23 10.2%   (out of range)
    kind 18:      154 68.4%   (bytearray[int])
    kind 20:       47 20.9%   (python __setitem__)

bm_nqueens:

STORE_SUBSCR:
 unquickened:        4181 0.0%
    deferred:     3627586 33.3%
         hit:     7249549 66.6%
  success:         751
  failure:       56743
    kind  0:        1 0.0%    (other)
    kind  4:    20813 36.7%   (out of range)
    kind 10:    35728 63.0%   (list[slice])
    kind 18:      154 0.3%    (bytearray[int])
    kind 20:       47 0.1%    (python __setitem__)

bm_regex_compile:

STORE_SUBSCR:
 unquickened:        3962 0.0%
    deferred:    15657733 92.5%
         hit:     1272334 7.5%
  success:         755
  failure:      244732
    kind  0:        1 0.0%    (other)
    kind  4:     1127 0.5%    (out of range)
    kind 10:      309 0.1%    (list[slice])
    kind 18:   241858 98.8%   (bytearray[int])
    kind 20:     1437 0.6%    (python __setitem__)

bm_regex_dna:

STORE_SUBSCR:
 unquickened:        4058 0.0%
    deferred:     9771490 99.8%
         hit:       17315 0.2%
  success:         718
  failure:      152756
    kind  0:        1 0.0%    (other)
    kind  4:       23 0.0%    (out of range)
    kind 18:   152685 100.0%  (bytearray[int])
    kind 20:       47 0.0%    (python __setitem__)

bm_regex_v8:

STORE_SUBSCR:
 unquickened:        3926 0.1%
    deferred:     3965127 99.0%
         hit:       36282 0.9%
  success:         767
  failure:       62043
    kind  0:        1 0.0%    (other)
    kind  4:       69 0.1%    (out of range)
    kind 10:       12 0.0%    (list[slice])
    kind 18:    61879 99.7%   (bytearray[int])
    kind 20:       82 0.1%    (python __setitem__)

bm_scimark:

STORE_SUBSCR:
 unquickened:       20739 0.0%
    deferred:   104490573 99.9%
         hit:       83500 0.1%
  success:        3369
  failure:     1633058
    kind  0:        2 0.0%    (other)
    kind  4:      111 0.0%    (out of range)
    kind  8:  1577150 96.6%   (array[int])
    kind  9:       55 0.0%    (array[slice])
    kind 18:      726 0.0%    (bytearray[int])
    kind 20:    55014 3.4%    (python __setitem__)

[Fidget-Spinner]

Dennis, sorry for the unrelated question, but how do you normally pull those pyperformance benchmark stats? When I try to get them I get one for each process instead (I'm trying to find tricks around that).

[sweeneyde]

Dennis, sorry for the unrelated question, but how do you normally pull those pyperformance benchmark stats? When I try to get them I get one for each process instead (I'm trying to find tricks around that).

Up until yesterday, I had done that too, hastily interrupting the benchmark after a couple of runs and getting the stats from one of the earlier outputs that wasn't interrupted.

But for these results, I added #define Py_STATS to pycore_code.h, changed the path (and buffer size and assertion) and if (to_file) to if (1) in _Py_PrintSpecializationStats of specialize.c, then compiled and ran a benchmark, which generated results in the folder. Then Tools/scripts/summarize_stats.py (with the path likewise changed) generated output that included stuff like my previous comment. I then deleted all of the output files in the folder and ran the next benchmark and re-ran the summarize_stats script. Maybe there's a better way though.

[markshannon]

A couple of minor comments, otherwise LGTM.

[markshannon]

Thanks, always good to have better stats.