Skip to content

Conversation

gabritto
Copy link
Member

@gabritto gabritto commented Jun 13, 2024

Updated version of #55234.
Fixes #3276.

Motivated by #57842.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 13, 2024
@gabritto
Copy link
Member Author

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 13, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started 👀 Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

Hey @gabritto, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the user tests with tsc comparing main and refs/pull/58859/merge:

Something interesting changed - please have a look.

Details

effect

tsconfig.json

tsconfig.base.json

lodash

/mnt/ts_downloads/_/m/lodash/tsconfig.json

  • [NEW] error TS2740: Type 'SetCache' is missing the following properties from type 'any[]': length, pop, concat, join, and 34 more.
    • /mnt/ts_downloads/_/m/lodash/node_modules/lodash/_baseUniq.js(39,5)
  • [MISSING] error TS2322: Type 'SetCache' is not assignable to type 'any[]'.
    • /mnt/ts_downloads/_/m/lodash/node_modules/lodash/_baseUniq.js(39,5)

xterm.js

src/tsconfig-library-base.json

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,729k (± 0.78%) 193,392k (± 0.98%) ~ 192,152k 195,835k p=0.471 n=6
Parse Time 1.95s (± 0.82%) 1.95s (± 0.39%) ~ 1.94s 1.96s p=0.788 n=6
Bind Time 1.07s (± 0.76%) 1.06s (± 1.41%) ~ 1.05s 1.09s p=0.451 n=6
Check Time 13.89s (± 0.40%) 13.92s (± 0.34%) ~ 13.87s 13.98s p=0.422 n=6
Emit Time 4.07s (± 3.10%) 4.02s (± 0.58%) ~ 3.98s 4.05s p=1.000 n=6
Total Time 20.98s (± 0.42%) 20.95s (± 0.17%) ~ 20.89s 21.00s p=0.810 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,104 944,104 ~ ~ ~ p=1.000 n=6
Types 407,044 407,044 ~ ~ ~ p=1.000 n=6
Memory used 1,218,275k (± 0.01%) 1,218,301k (± 0.00%) ~ 1,218,229k 1,218,362k p=0.423 n=6
Parse Time 7.94s (± 0.50%) 7.97s (± 0.64%) ~ 7.90s 8.03s p=0.334 n=6
Bind Time 2.23s (± 0.34%) 2.23s (± 0.44%) ~ 2.22s 2.24s p=0.554 n=6
Check Time 35.93s (± 0.32%) 35.89s (± 0.27%) ~ 35.76s 36.05s p=0.748 n=6
Emit Time 16.23s (± 0.31%) 16.21s (± 0.47%) ~ 16.09s 16.31s p=0.872 n=6
Total Time 62.33s (± 0.28%) 62.31s (± 0.17%) ~ 62.10s 62.40s p=0.747 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 2,132,301 2,132,301 ~ ~ ~ p=1.000 n=6
Types 926,081 926,081 ~ ~ ~ p=1.000 n=6
Memory used 2,114,644k (± 0.00%) 2,114,763k (± 0.00%) +119k (+ 0.01%) 2,114,728k 2,114,867k p=0.005 n=6
Parse Time 7.86s (± 0.15%) 7.86s (± 0.24%) ~ 7.84s 7.89s p=0.684 n=6
Bind Time 2.77s (± 0.27%) 2.76s (± 0.50%) ~ 2.75s 2.78s p=0.740 n=6
Check Time 84.38s (± 0.32%) 83.94s (± 0.33%) -0.44s (- 0.52%) 83.52s 84.22s p=0.031 n=6
Emit Time 0.16s (± 4.75%) 0.16s (± 4.99%) ~ 0.15s 0.17s p=0.306 n=6
Total Time 95.17s (± 0.28%) 94.72s (± 0.30%) -0.45s (- 0.47%) 94.30s 95.02s p=0.031 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,770 1,230,758 -12 (- 0.00%) ~ ~ p=0.001 n=6
Types 261,222 261,219 -3 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 2,355,243k (± 1.04%) 2,345,705k (± 0.06%) ~ 2,343,313k 2,346,879k p=0.936 n=6
Parse Time 5.99s (± 1.16%) 5.95s (± 1.22%) ~ 5.86s 6.06s p=0.471 n=6
Bind Time 2.24s (± 0.34%) 2.25s (± 0.83%) ~ 2.23s 2.28s p=0.073 n=6
Check Time 39.81s (± 0.20%) 39.74s (± 0.37%) ~ 39.53s 39.96s p=0.378 n=6
Emit Time 3.18s (± 3.90%) 3.10s (± 2.88%) ~ 3.00s 3.26s p=0.230 n=6
Total Time 51.23s (± 0.41%) 51.06s (± 0.51%) ~ 50.87s 51.57s p=0.173 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,770 1,230,758 -12 (- 0.00%) ~ ~ p=0.001 n=6
Types 261,222 261,219 -3 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 2,422,483k (± 0.03%) 2,431,708k (± 1.02%) ~ 2,420,791k 2,482,249k p=0.298 n=6
Parse Time 6.26s (± 0.87%) 6.23s (± 0.74%) ~ 6.17s 6.29s p=0.298 n=6
Bind Time 2.01s (± 0.77%) 1.99s (± 0.94%) -0.02s (- 1.16%) 1.96s 2.01s p=0.048 n=6
Check Time 40.15s (± 0.30%) 40.18s (± 0.43%) ~ 40.05s 40.52s p=0.810 n=6
Emit Time 3.14s (± 2.00%) 3.12s (± 1.47%) ~ 3.04s 3.17s p=0.810 n=6
Total Time 51.58s (± 0.32%) 51.53s (± 0.35%) ~ 51.39s 51.89s p=0.575 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,665 258,664 -1 (- 0.00%) ~ ~ p=0.001 n=6
Types 104,938 104,935 -3 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 428,300k (± 0.01%) 428,268k (± 0.01%) ~ 428,246k 428,322k p=0.078 n=6
Parse Time 3.30s (± 0.68%) 3.31s (± 0.88%) ~ 3.26s 3.35s p=0.572 n=6
Bind Time 1.30s (± 0.69%) 1.29s (± 0.42%) ~ 1.29s 1.30s p=0.341 n=6
Check Time 18.00s (± 0.29%) 17.96s (± 0.16%) ~ 17.91s 18.00s p=0.280 n=6
Emit Time 1.37s (± 1.49%) 1.33s (± 0.41%) -0.03s (- 2.44%) 1.33s 1.34s p=0.008 n=6
Total Time 23.97s (± 0.25%) 23.90s (± 0.16%) ~ 23.86s 23.95s p=0.053 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,414k (± 0.03%) 369,569k (± 0.04%) +154k (+ 0.04%) 369,468k 369,824k p=0.045 n=6
Parse Time 2.76s (± 0.88%) 2.78s (± 0.95%) ~ 2.74s 2.81s p=0.373 n=6
Bind Time 1.60s (± 1.51%) 1.59s (± 0.73%) ~ 1.58s 1.61s p=0.459 n=6
Check Time 15.47s (± 0.45%) 15.51s (± 0.33%) ~ 15.43s 15.57s p=0.227 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.84s (± 0.37%) 19.88s (± 0.33%) ~ 19.77s 19.95s p=0.422 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,853,853 2,853,853 ~ ~ ~ p=1.000 n=6
Types 968,176 968,176 ~ ~ ~ p=1.000 n=6
Memory used 3,020,552k (± 0.00%) 3,020,543k (± 0.00%) ~ 3,020,506k 3,020,572k p=0.873 n=6
Parse Time 16.82s (± 0.33%) 16.76s (± 0.25%) ~ 16.70s 16.81s p=0.109 n=6
Bind Time 5.09s (± 0.32%) 5.09s (± 0.42%) ~ 5.05s 5.11s p=0.805 n=6
Check Time 88.92s (± 0.54%) 89.11s (± 0.69%) ~ 88.16s 89.89s p=0.575 n=6
Emit Time 28.87s (± 1.37%) 28.57s (± 2.39%) ~ 27.61s 29.24s p=0.521 n=6
Total Time 139.70s (± 0.32%) 139.54s (± 0.27%) ~ 138.99s 140.06s p=0.378 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 266,914 266,914 ~ ~ ~ p=1.000 n=6
Types 108,685 108,685 ~ ~ ~ p=1.000 n=6
Memory used 411,295k (± 0.01%) 411,377k (± 0.02%) +82k (+ 0.02%) 411,313k 411,485k p=0.031 n=6
Parse Time 4.71s (± 0.71%) 4.69s (± 0.43%) ~ 4.66s 4.72s p=0.292 n=6
Bind Time 2.07s (± 0.36%) 2.06s (± 0.66%) ~ 2.04s 2.08s p=0.273 n=6
Check Time 20.85s (± 0.55%) 20.86s (± 0.30%) ~ 20.78s 20.92s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.63s (± 0.33%) 27.61s (± 0.19%) ~ 27.54s 27.68s p=0.470 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,429 523,429 ~ ~ ~ p=1.000 n=6
Types 177,955 177,955 ~ ~ ~ p=1.000 n=6
Memory used 461,458k (± 0.08%) 461,891k (± 0.01%) ~ 461,827k 461,946k p=0.173 n=6
Parse Time 2.62s (± 0.38%) 2.63s (± 0.61%) ~ 2.61s 2.65s p=0.271 n=6
Bind Time 0.99s (± 0.64%) 0.99s (± 0.41%) ~ 0.98s 0.99s p=0.673 n=6
Check Time 15.25s (± 0.66%) 15.27s (± 0.46%) ~ 15.21s 15.41s p=0.420 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.86s (± 0.52%) 18.89s (± 0.34%) ~ 18.83s 19.01s p=0.261 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top 400 repos with tsc comparing main and refs/pull/58859/merge:

Everything looks good!

@gabritto
Copy link
Member Author

It doesn't seem like the case of reporting the same assignability error more than once is that frequent, since there were only 3 changes in user tests, none in top 400, and no perf impact.

Two things I want to do, though:

  • Special case type comparison "overflow" cases so that we don't do the comparison again.
  • Maybe only have this consistent assignability elaboration when we know we are in an editor scenario, if that's possible.

@gabritto
Copy link
Member Author

@typescript-bot test top800

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 14, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top800 ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top 800 repos with tsc comparing main and refs/pull/58859/merge:

Everything looks good!

@jakebailey jakebailey closed this Jul 10, 2024
@jakebailey jakebailey reopened this Jul 10, 2024
@gabritto
Copy link
Member Author

@typescript-bot perf test this
@typescript-bot test top800

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 10, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results
test top800 ✅ Started ✅ Results

@gabritto gabritto changed the title Make assignability error elaboration consistent Make type comparison error elaboration consistent Jul 10, 2024
@gabritto
Copy link
Member Author

@typescript-bot user test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 10, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
user test this ✅ Started 👀 Results

@typescript-bot typescript-bot removed the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Jul 10, 2024
@typescript-bot typescript-bot added the For Backlog Bug PRs that fix a backlog bug label Jul 10, 2024
@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,916k (± 1.00%) 192,162k (± 0.03%) ~ 192,077k 192,226k p=0.298 n=6
Parse Time 1.96s (± 0.43%) 1.95s (± 0.70%) ~ 1.93s 1.96s p=0.262 n=6
Bind Time 1.05s (± 0.79%) 1.05s (± 1.11%) ~ 1.04s 1.07s p=0.555 n=6
Check Time 13.85s (± 0.61%) 13.86s (± 0.31%) ~ 13.78s 13.90s p=0.570 n=6
Emit Time 4.03s (± 0.57%) 4.04s (± 1.16%) ~ 3.95s 4.08s p=0.373 n=6
Total Time 20.89s (± 0.46%) 20.90s (± 0.17%) ~ 20.84s 20.95s p=0.688 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,369k (± 0.00%) 1,218,341k (± 0.00%) ~ 1,218,306k 1,218,388k p=0.230 n=6
Parse Time 7.94s (± 0.50%) 7.97s (± 0.65%) ~ 7.93s 8.07s p=0.226 n=6
Bind Time 2.22s (± 0.66%) 2.23s (± 0.57%) ~ 2.22s 2.25s p=0.198 n=6
Check Time 35.77s (± 0.23%) 35.88s (± 0.29%) ~ 35.75s 36.04s p=0.078 n=6
Emit Time 16.21s (± 0.82%) 16.15s (± 0.26%) ~ 16.10s 16.21s p=0.687 n=6
Total Time 62.14s (± 0.13%) 62.23s (± 0.24%) ~ 62.01s 62.43s p=0.296 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,130,296 2,130,296 ~ ~ ~ p=1.000 n=6
Types 927,656 927,656 ~ ~ ~ p=1.000 n=6
Memory used 2,107,135k (± 0.01%) 2,107,141k (± 0.01%) ~ 2,107,047k 2,107,310k p=1.000 n=6
Parse Time 7.82s (± 0.36%) 7.81s (± 0.40%) ~ 7.75s 7.83s p=0.324 n=6
Bind Time 2.74s (± 0.56%) 2.73s (± 0.95%) ~ 2.69s 2.76s p=0.565 n=6
Check Time 83.40s (± 0.35%) 83.55s (± 0.35%) ~ 83.20s 84.07s p=0.688 n=6
Emit Time 0.16s (± 4.75%) 0.16s (± 3.29%) ~ 0.15s 0.16s p=0.784 n=6
Total Time 94.12s (± 0.32%) 94.25s (± 0.31%) ~ 93.94s 94.79s p=0.748 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,784 1,225,809 +25 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,518 261,522 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,338,907k (± 0.02%) 2,339,004k (± 0.05%) ~ 2,338,265k 2,341,142k p=0.575 n=6
Parse Time 7.56s (± 0.66%) 7.54s (± 0.57%) ~ 7.49s 7.62s p=0.688 n=6
Bind Time 2.81s (± 0.83%) 2.80s (± 0.73%) ~ 2.77s 2.83s p=0.935 n=6
Check Time 49.50s (± 0.30%) 49.52s (± 0.28%) ~ 49.41s 49.75s p=0.471 n=6
Emit Time 4.76s (± 1.61%) 4.73s (± 1.26%) ~ 4.66s 4.82s p=0.423 n=6
Total Time 64.63s (± 0.31%) 64.60s (± 0.27%) ~ 64.43s 64.91s p=0.936 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,784 1,225,809 +25 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,518 261,522 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,412,730k (± 0.02%) 2,413,490k (± 0.02%) +761k (+ 0.03%) 2,412,408k 2,413,863k p=0.045 n=6
Parse Time 7.89s (± 0.48%) 7.83s (± 0.80%) ~ 7.75s 7.91s p=0.128 n=6
Bind Time 2.51s (± 0.59%) 2.53s (± 0.92%) ~ 2.50s 2.57s p=0.222 n=6
Check Time 50.12s (± 0.23%) 50.13s (± 0.25%) ~ 49.92s 50.27s p=0.575 n=6
Emit Time 4.88s (± 1.22%) 4.79s (± 1.00%) -0.10s (- 2.01%) 4.72s 4.84s p=0.019 n=6
Total Time 65.41s (± 0.23%) 65.29s (± 0.23%) ~ 65.02s 65.40s p=0.378 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,268 258,271 +3 (+ 0.00%) ~ ~ p=0.001 n=6
Types 104,775 104,779 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 427,125k (± 0.03%) 427,113k (± 0.01%) ~ 427,043k 427,221k p=0.575 n=6
Parse Time 3.29s (± 0.78%) 3.29s (± 0.61%) ~ 3.26s 3.32s p=0.746 n=6
Bind Time 1.30s (± 1.05%) 1.32s (± 1.04%) ~ 1.29s 1.33s p=0.142 n=6
Check Time 17.77s (± 0.40%) 17.82s (± 0.29%) ~ 17.72s 17.86s p=0.296 n=6
Emit Time 1.64s (± 0.46%) 1.60s (± 0.75%) -0.03s (- 2.14%) 1.59s 1.62s p=0.005 n=6
Total Time 24.00s (± 0.39%) 24.03s (± 0.24%) ~ 23.94s 24.08s p=0.748 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,534k (± 0.04%) 369,523k (± 0.04%) ~ 369,388k 369,814k p=0.936 n=6
Parse Time 2.78s (± 1.58%) 2.77s (± 1.04%) ~ 2.74s 2.81s p=0.684 n=6
Bind Time 1.59s (± 1.08%) 1.59s (± 0.76%) ~ 1.58s 1.61s p=0.459 n=6
Check Time 15.52s (± 0.39%) 15.51s (± 0.41%) ~ 15.41s 15.59s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.89s (± 0.24%) 19.87s (± 0.32%) ~ 19.76s 19.93s p=0.748 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,889,341 2,889,341 ~ ~ ~ p=1.000 n=6
Types 977,546 977,546 ~ ~ ~ p=1.000 n=6
Memory used 3,052,777k (± 0.00%) 3,052,788k (± 0.00%) ~ 3,052,700k 3,052,869k p=0.575 n=6
Parse Time 16.81s (± 0.32%) 16.84s (± 0.49%) ~ 16.71s 16.95s p=0.688 n=6
Bind Time 5.11s (± 0.27%) 5.11s (± 0.53%) ~ 5.08s 5.14s p=1.000 n=6
Check Time 92.35s (± 2.99%) 91.35s (± 2.88%) ~ 89.34s 94.74s p=0.229 n=6
Emit Time 27.01s (± 9.59%) 27.65s (± 8.98%) ~ 24.34s 29.48s p=0.810 n=6
Total Time 141.29s (± 0.16%) 140.95s (± 0.23%) ~ 140.39s 141.32s p=0.066 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,235 267,235 ~ ~ ~ p=1.000 n=6
Types 108,815 108,815 ~ ~ ~ p=1.000 n=6
Memory used 411,783k (± 0.01%) 411,811k (± 0.01%) ~ 411,775k 411,856k p=0.298 n=6
Parse Time 3.19s (± 0.74%) 3.20s (± 0.47%) ~ 3.17s 3.21s p=0.461 n=6
Bind Time 1.40s (± 0.58%) 1.41s (± 0.59%) ~ 1.39s 1.41s p=0.718 n=6
Check Time 14.19s (± 0.29%) 14.22s (± 0.37%) ~ 14.15s 14.29s p=0.296 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.78s (± 0.31%) 18.82s (± 0.28%) ~ 18.75s 18.88s p=0.199 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,958k (± 0.08%) 463,184k (± 0.02%) ~ 463,030k 463,293k p=0.471 n=6
Parse Time 2.66s (± 0.57%) 2.65s (± 0.41%) ~ 2.64s 2.67s p=0.618 n=6
Bind Time 0.98s (± 0.84%) 0.98s (± 0.53%) ~ 0.97s 0.98s p=0.929 n=6
Check Time 15.19s (± 0.35%) 15.20s (± 0.27%) ~ 15.16s 15.27s p=0.810 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.82s (± 0.29%) 18.83s (± 0.24%) ~ 18.78s 18.91s p=0.747 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,295ms (± 0.18%) 2,292ms (± 0.60%) ~ 2,271ms 2,306ms p=0.873 n=6
Req 2 - geterr 5,082ms (± 0.77%) 5,080ms (± 0.29%) ~ 5,054ms 5,093ms p=0.810 n=6
Req 3 - references 265ms (± 0.44%) 265ms (± 0.28%) ~ 264ms 266ms p=0.933 n=6
Req 4 - navto 227ms (± 1.20%) 228ms (± 0.18%) ~ 227ms 228ms p=0.474 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 91ms (± 5.94%) 93ms (± 2.49%) ~ 91ms 97ms p=0.934 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,575ms (± 1.02%) 3,583ms (± 1.53%) ~ 3,525ms 3,657ms p=0.688 n=6
Req 2 - geterr 5,539ms (± 0.59%) 5,548ms (± 0.28%) ~ 5,526ms 5,569ms p=0.810 n=6
Req 3 - references 418ms (± 0.33%) 417ms (± 0.33%) ~ 415ms 419ms p=0.253 n=6
Req 4 - navto 338ms (± 0.71%) 339ms (± 1.03%) ~ 336ms 346ms p=0.319 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 109ms (± 2.59%) 109ms (± 2.78%) ~ 103ms 111ms p=0.858 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,187ms (± 0.69%) 5,205ms (± 0.50%) ~ 5,183ms 5,255ms p=0.689 n=6
Req 2 - geterr 1,125ms (± 1.27%) 1,120ms (± 0.94%) ~ 1,111ms 1,138ms p=0.688 n=6
Req 3 - references 80ms (± 5.27%) 77ms (± 3.11%) ~ 76ms 82ms p=0.325 n=6
Req 4 - navto 456ms (± 0.38%) 458ms (± 0.26%) ~ 457ms 460ms p=0.081 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 842ms (± 1.72%) 834ms (± 1.76%) ~ 819ms 856ms p=0.336 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.04ms (± 0.18%) 156.88ms (± 0.17%) -0.16ms (- 0.10%) 155.74ms 161.19ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 232.00ms (± 0.13%) 232.11ms (± 0.16%) +0.11ms (+ 0.05%) 230.57ms 237.89ms p=0.014 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.37ms (± 0.16%) 226.23ms (± 0.14%) -0.14ms (- 0.06%) 224.58ms 228.25ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 225.97ms (± 0.17%) 226.15ms (± 0.20%) +0.18ms (+ 0.08%) 224.42ms 234.03ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the user tests with tsc comparing main and refs/pull/58859/merge:

Something interesting changed - please have a look.

Details

effect

tsconfig.json

tsconfig.build.json

tsconfig.base.json

lodash

/mnt/ts_downloads/_/m/lodash/tsconfig.json

  • [NEW] error TS2740: Type 'SetCache' is missing the following properties from type 'any[]': length, pop, concat, join, and 34 more.
    • /mnt/ts_downloads/_/m/lodash/node_modules/lodash/_baseUniq.js(39,5)
  • [MISSING] error TS2322: Type 'SetCache' is not assignable to type 'any[]'.
    • /mnt/ts_downloads/_/m/lodash/node_modules/lodash/_baseUniq.js(39,5)

xterm.js

src/tsconfig-library-base.json

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top 800 repos with tsc comparing main and refs/pull/58859/merge:

Everything looks good!

@gabritto gabritto marked this pull request as ready for review July 11, 2024 00:43
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perf still looks fine, baselines look good, and I already approved the old version of the PR :)

@gabritto gabritto merged commit 0314372 into main Jul 15, 2024
@gabritto gabritto deleted the gabritto/elaboration branch July 15, 2024 18:11
@gabritto gabritto changed the title Make type comparison error elaboration consistent https://github.com/microsoft/TypeScript/pull/58220 Jul 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Backlog Bug PRs that fix a backlog bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Diagnostic elaboration is not consistently reported in language service
4 participants