[experiment] Bail out early in union production if too many literals/templates #59705

jakebailey · 2024-08-21T00:21:13Z

Motivated by #59655 and similar ☹️

…templates

jakebailey · 2024-08-21T00:21:21Z

typescript-bot · 2024-08-21T00:21:24Z

Starting jobs; this comment will be updated as builds start and complete.

Command	Status	Results
`test top400`	✅ Started	✅ Results
`user test this`	✅ Started	✅ Results
`run dt`	✅ Started	✅ Results
`perf test this faster`	✅ Started	👀 Results

typescript-bot · 2024-08-21T00:42:00Z

Hey @jakebailey, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

typescript-bot · 2024-08-21T00:47:55Z

@jakebailey Here are the results of running the user tests with tsc comparing main and refs/pull/59705/merge:

Everything looks good!

typescript-bot · 2024-08-21T01:03:07Z

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-Unions - node (v18.15.0, x64)}
_Errors	₃₀	₃₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_62,153	_62,153	_~	_~	_~	_{p=1.000 n=6}
_Types	_50,242	_50,242	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{193,573k (± 0.89%)}	_{193,136k (± 0.71%)}	_~	_192,330k	_195,896k	_{p=1.000 n=6}
_{Parse Time}	_{1.57s (± 0.67%)}	_{1.57s (± 0.84%)}	_~	_1.55s	_1.58s	_{p=0.864 n=6}
_{Bind Time}	_{0.86s (± 0.88%)}	_{0.86s (± 1.27%)}	_~	_0.85s	_0.87s	_{p=0.865 n=6}
_{Check Time}	_{11.41s (± 0.36%)}	_{11.36s (± 0.26%)}	_{-0.05s (- 0.45%)}	_11.33s	_11.41s	_{p=0.045 n=6}
_{Emit Time}	_{3.24s (± 0.86%)}	_{3.24s (± 0.58%)}	_~	_3.22s	_3.27s	_{p=0.871 n=6}
_{Total Time}	_{17.09s (± 0.25%)}	_{17.04s (± 0.17%)}	_{-0.05s (- 0.28%)}	_16.99s	_17.07s	_{p=0.045 n=6}
_{angular-1 - node (v18.15.0, x64)}
_Errors	₇	₇	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_945,753	_945,753	_~	_~	_~	_{p=1.000 n=6}
_Types	_410,067	_410,067	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{1,222,666k (± 0.00%)}	_{1,222,682k (± 0.00%)}	_~	_1,222,650k	_1,222,712k	_{p=0.378 n=6}
_{Parse Time}	_{7.93s (± 0.65%)}	_{7.93s (± 0.47%)}	_~	_7.88s	_7.98s	_{p=1.000 n=6}
_{Bind Time}	_{2.22s (± 0.23%)}	_{2.22s (± 0.37%)}	_~	_2.21s	_2.23s	_{p=0.929 n=6}
_{Check Time}	_{36.46s (± 0.30%)}	_{36.38s (± 0.27%)}	_~	_36.23s	_36.49s	_{p=0.297 n=6}
_{Emit Time}	_{17.78s (± 0.58%)}	_{17.76s (± 0.45%)}	_~	_17.61s	_17.84s	_{p=1.000 n=6}
_{Total Time}	_{64.39s (± 0.26%)}	_{64.30s (± 0.16%)}	_~	_64.19s	_64.44s	_{p=0.378 n=6}
_{mui-docs - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_2,508,161	_2,508,161	_~	_~	_~	_{p=1.000 n=6}
_Types	_992,225	_992,225	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{2,375,708k (± 0.00%)}	_{2,375,665k (± 0.00%)}	_~	_2,375,561k	_2,375,770k	_{p=0.298 n=6}
_{Parse Time}	_{13.59s (± 0.60%)}	_{13.59s (± 0.27%)}	_~	_13.52s	_13.62s	_{p=0.872 n=6}
_{Bind Time}	_{3.24s (± 0.78%)}	_{3.23s (± 0.25%)}	_~	_3.22s	_3.24s	_{p=0.566 n=6}
_{Check Time}	_{108.25s (± 1.25%)}	_{107.27s (± 0.38%)}	_~	_106.58s	_107.77s	_{p=0.093 n=6}
_{Emit Time}	_{0.40s (± 2.74%)}	_{0.39s (± 3.82%)}	_~	_0.38s	_0.41s	_{p=0.554 n=6}
_{Total Time}	_{125.48s (± 1.08%)}	_{124.48s (± 0.35%)}	_~	_123.81s	_125.02s	_{p=0.078 n=6}
_{self-build-src - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_1,231,998	_1,232,008	_{+10 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_Types	_264,593	_264,598	_{+5 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{2,472,455k (± 7.43%)}	_{2,650,619k (± 5.49%)}	_~	_2,353,121k	_2,710,733k	_{p=0.298 n=6}
_{Parse Time}	_{6.01s (± 0.74%)}	_{6.03s (± 0.55%)}	_~	_5.98s	_6.06s	_{p=0.520 n=6}
_{Bind Time}	_{2.24s (± 0.61%)}	_{2.24s (± 1.01%)}	_~	_2.20s	_2.26s	_{p=0.685 n=6}
_{Check Time}	_{40.82s (± 1.09%)}	_{40.47s (± 1.20%)}	_~	_40.13s	_41.43s	_{p=0.128 n=6}
_{Emit Time}	_{4.00s (± 0.89%)}	_{4.06s (± 2.68%)}	_~	_3.95s	_4.26s	_{p=0.230 n=6}
_{Total Time}	_{53.09s (± 0.88%)}	_{52.82s (± 0.96%)}	_~	_52.32s	_53.78s	_{p=0.297 n=6}
_{self-build-src-public-api - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_1,231,998	_1,232,008	_{+10 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_Types	_264,593	_264,598	_{+5 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{2,427,654k (± 0.01%)}	_{2,428,510k (± 0.01%)}	_{+857k (+ 0.04%)}	_2,428,153k	_2,429,054k	_{p=0.005 n=6}
_{Parse Time}	_{6.23s (± 1.42%)}	_{6.22s (± 0.40%)}	_~	_6.19s	_6.25s	_{p=1.000 n=6}
_{Bind Time}	_{2.04s (± 0.57%)}	_{2.04s (± 0.74%)}	_~	_2.02s	_2.05s	_{p=1.000 n=6}
_{Check Time}	_{41.84s (± 0.59%)}	_{41.64s (± 0.74%)}	_~	_41.33s	_41.96s	_{p=0.199 n=6}
_{Emit Time}	_{4.05s (± 0.74%)}	_{4.05s (± 2.02%)}	_~	_3.97s	_4.20s	_{p=0.377 n=6}
_{Total Time}	_{54.17s (± 0.38%)}	_{53.96s (± 0.54%)}	_~	_53.60s	_54.32s	_{p=0.173 n=6}
_{self-compiler - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_256,992	_257,002	_{+10 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_Types	_105,667	_105,672	_{+5 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{429,697k (± 0.03%)}	_{429,717k (± 0.03%)}	_~	_429,539k	_429,864k	_{p=0.936 n=6}
_{Parse Time}	_{4.17s (± 0.52%)}	_{4.16s (± 0.35%)}	_~	_4.14s	_4.18s	_{p=0.226 n=6}
_{Bind Time}	_{1.59s (± 1.10%)}	_{1.59s (± 1.03%)}	_~	_1.58s	_1.61s	_{p=1.000 n=6}
_{Check Time}	_{22.43s (± 0.38%)}	_{22.41s (± 0.47%)}	_~	_22.26s	_22.51s	_{p=0.688 n=6}
_{Emit Time}	_{2.06s (± 0.84%)}	_{2.03s (± 0.67%)}	_{-0.03s (- 1.21%)}	_2.02s	_2.05s	_{p=0.028 n=6}
_{Total Time}	_{30.25s (± 0.28%)}	_{30.19s (± 0.31%)}	_~	_30.09s	_30.32s	_{p=0.198 n=6}
_{ts-pre-modules - node (v18.15.0, x64)}
_Errors	₆₈	₆₈	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_225,018	_225,018	_~	_~	_~	_{p=1.000 n=6}
_Types	_94,249	_94,249	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{370,226k (± 0.02%)}	_{370,266k (± 0.03%)}	_~	_370,130k	_370,487k	_{p=0.689 n=6}
_{Parse Time}	_{2.76s (± 0.58%)}	_{2.76s (± 0.65%)}	_~	_2.73s	_2.78s	_{p=0.867 n=6}
_{Bind Time}	_{1.56s (± 0.48%)}	_{1.57s (± 1.06%)}	_~	_1.55s	_1.59s	_{p=0.240 n=6}
_{Check Time}	_{15.75s (± 0.24%)}	_{15.71s (± 0.37%)}	_~	_15.63s	_15.78s	_{p=0.197 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{20.06s (± 0.25%)}	_{20.04s (± 0.21%)}	_~	_20.00s	_20.11s	_{p=0.521 n=6}
_{vscode - node (v18.15.0, x64)}
_Errors	₆₆	₆₆	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_3,017,608	_3,017,608	_~	_~	_~	_{p=1.000 n=6}
_Types	_1,038,333	_1,038,333	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{3,137,101k (± 0.00%)}	_{3,137,078k (± 0.00%)}	_~	_3,136,959k	_3,137,189k	_{p=0.471 n=6}
_{Parse Time}	_{14.07s (± 0.57%)}	_{14.01s (± 0.36%)}	_~	_13.93s	_14.08s	_{p=0.109 n=6}
_{Bind Time}	_{4.30s (± 0.35%)}	_{4.30s (± 0.24%)}	_~	_4.29s	_4.32s	_{p=0.615 n=6}
_{Check Time}	_{80.17s (± 0.19%)}	_{80.39s (± 0.36%)}	_~	_80.03s	_80.68s	_{p=0.229 n=6}
_{Emit Time}	_{20.51s (± 0.55%)}	_{20.48s (± 0.32%)}	_~	_20.38s	_20.56s	_{p=0.689 n=6}
_{Total Time}	_{119.04s (± 0.19%)}	_{119.17s (± 0.26%)}	_~	_118.73s	_119.46s	_{p=0.336 n=6}
_{webpack - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_275,310	_275,310	_~	_~	_~	_{p=1.000 n=6}
_Types	_112,431	_112,431	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{424,153k (± 0.01%)}	_{424,201k (± 0.01%)}	_~	_424,123k	_424,249k	_{p=0.128 n=6}
_{Parse Time}	_{3.31s (± 0.53%)}	_{3.32s (± 0.40%)}	_~	_3.30s	_3.34s	_{p=0.492 n=6}
_{Bind Time}	_{1.45s (± 0.56%)}	_{1.45s (± 0.75%)}	_~	_1.44s	_1.47s	_{p=0.666 n=6}
_{Check Time}	_{14.83s (± 0.25%)}	_{14.88s (± 0.37%)}	_~	_14.79s	_14.95s	_{p=0.127 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{19.59s (± 0.24%)}	_{19.65s (± 0.27%)}	_~	_19.56s	_19.71s	_{p=0.128 n=6}
_{xstate-main - node (v18.15.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_534,554	_534,554	_~	_~	_~	_{p=1.000 n=6}
_Types	_176,379	_176,379	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{479,418k (± 0.01%)}	_{479,428k (± 0.01%)}	_~	_479,376k	_479,499k	_{p=1.000 n=6}
_{Parse Time}	_{3.43s (± 0.62%)}	_{3.42s (± 0.86%)}	_~	_3.39s	_3.47s	_{p=0.513 n=6}
_{Bind Time}	_{1.25s (± 1.06%)}	_{1.25s (± 0.67%)}	_~	_1.24s	_1.26s	_{p=0.849 n=6}
_{Check Time}	_{18.13s (± 0.19%)}	_{18.11s (± 0.40%)}	_~	_18.00s	_18.18s	_{p=1.000 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{22.81s (± 0.14%)}	_{22.79s (± 0.38%)}	_~	_22.66s	_22.88s	_{p=0.936 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Compiler-Unions - node (v18.15.0, x64)
angular-1 - node (v18.15.0, x64)
mui-docs - node (v18.15.0, x64)
self-build-src - node (v18.15.0, x64)
self-build-src-public-api - node (v18.15.0, x64)
self-compiler - node (v18.15.0, x64)
ts-pre-modules - node (v18.15.0, x64)
vscode - node (v18.15.0, x64)
webpack - node (v18.15.0, x64)
xstate-main - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

typescript-bot · 2024-08-21T02:11:16Z

@jakebailey Here are the results of running the top 400 repos with tsc comparing main and refs/pull/59705/merge:

Everything looks good!

[experiment] Bail out early in union production if too many literals/…

0d93b47

…templates

typescript-bot assigned jakebailey Aug 21, 2024

typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Aug 21, 2024

Baseline because why not

5ae71e6

jakebailey mentioned this pull request Aug 26, 2024

Use a trie to quickly skip templates in union creation #59759

Draft

2 tasks

jakebailey closed this Apr 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[experiment] Bail out early in union production if too many literals/templates #59705

[experiment] Bail out early in union production if too many literals/templates #59705

jakebailey commented Aug 21, 2024

jakebailey commented Aug 21, 2024

typescript-bot commented Aug 21, 2024 •

edited

Loading

typescript-bot commented Aug 21, 2024

typescript-bot commented Aug 21, 2024

typescript-bot commented Aug 21, 2024

tsc

typescript-bot commented Aug 21, 2024

[experiment] Bail out early in union production if too many literals/templates #59705

[experiment] Bail out early in union production if too many literals/templates #59705

Conversation

jakebailey commented Aug 21, 2024

jakebailey commented Aug 21, 2024

typescript-bot commented Aug 21, 2024 • edited Loading

typescript-bot commented Aug 21, 2024

typescript-bot commented Aug 21, 2024

typescript-bot commented Aug 21, 2024

tsc

typescript-bot commented Aug 21, 2024

typescript-bot commented Aug 21, 2024 •

edited

Loading