Skip to content

Fix contextual discrimination for omitted members #48448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Fix contextual discrimination for omitted members #48448

wants to merge 1 commit into from

Conversation

erikbrinkman
Copy link
Contributor

The first attempt at fixing this issue caused a large regression on one
benchmark. This new PR addresses the regression by caching optional
members of union types when they're used for discrimination. This seems
to reduce the performance hit at the cost of some extra memory, but it's
not clear if the fix is worth this regression.

For context:
first related PR: #41759
original fix: #43633
fixes: #48298
reversion: #48426

I was able to reproduce the regression, but behavior on WSL ended up proving spotty, so in some tests the regression went mostly away (to 3%) on the material-ui docs benchmark. Other times it was closer to 10%. Ideally I'd like to see how this diff performs on all the benchmarks as is. I think further optimization is goin to require deeper caching, and there's an argument that maybe performing "correctly" in this case isn't worth the performance hit.

@typescript-bot typescript-bot added the For Milestone Bug PRs that fix a bug with a specific milestone label Mar 27, 2022
@weswigham
Copy link
Member

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 27, 2022

Heya @weswigham, I've started to run the perf test suite on this PR at caa081c. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..48448

Metric main 48448 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 356,681k (± 0.02%) 356,423k (± 0.02%) -257k (- 0.07%) 356,252k 356,536k
Parse Time 2.07s (± 0.48%) 2.05s (± 0.57%) -0.02s (- 1.01%) 2.03s 2.08s
Bind Time 0.87s (± 0.57%) 0.86s (± 0.57%) -0.00s (- 0.23%) 0.85s 0.87s
Check Time 5.81s (± 0.65%) 5.79s (± 0.58%) -0.02s (- 0.43%) 5.70s 5.84s
Emit Time 6.02s (± 0.64%) 5.99s (± 0.52%) -0.04s (- 0.60%) 5.93s 6.05s
Total Time 14.77s (± 0.44%) 14.69s (± 0.34%) -0.08s (- 0.53%) 14.63s 14.84s
Compiler-Unions - node (v10.16.3, x64)
Memory used 205,764k (± 0.03%) 205,746k (± 0.03%) -18k (- 0.01%) 205,647k 205,980k
Parse Time 0.85s (± 0.76%) 0.85s (± 0.61%) -0.00s (- 0.12%) 0.84s 0.86s
Bind Time 0.52s (± 1.00%) 0.52s (± 0.95%) -0.00s (- 0.58%) 0.51s 0.53s
Check Time 8.01s (± 0.56%) 8.04s (± 0.96%) +0.02s (+ 0.29%) 7.94s 8.34s
Emit Time 2.54s (± 1.82%) 2.52s (± 0.87%) -0.03s (- 0.98%) 2.48s 2.59s
Total Time 11.93s (± 0.70%) 11.92s (± 0.81%) -0.01s (- 0.04%) 11.78s 12.29s
Monaco - node (v10.16.3, x64)
Memory used 343,686k (± 0.01%) 343,600k (± 0.02%) -86k (- 0.02%) 343,445k 343,784k
Parse Time 1.58s (± 1.11%) 1.58s (± 0.71%) -0.00s (- 0.13%) 1.54s 1.59s
Bind Time 0.76s (± 0.88%) 0.76s (± 0.81%) +0.00s (+ 0.53%) 0.75s 0.77s
Check Time 5.76s (± 0.33%) 5.76s (± 0.88%) +0.00s (+ 0.07%) 5.61s 5.88s
Emit Time 3.24s (± 0.45%) 3.24s (± 0.66%) -0.00s (- 0.06%) 3.19s 3.27s
Total Time 11.33s (± 0.36%) 11.34s (± 0.56%) +0.00s (+ 0.03%) 11.18s 11.45s
TFS - node (v10.16.3, x64)
Memory used 305,268k (± 0.02%) 305,311k (± 0.02%) +43k (+ 0.01%) 305,146k 305,469k
Parse Time 1.28s (± 0.47%) 1.28s (± 0.37%) -0.00s (- 0.39%) 1.27s 1.29s
Bind Time 0.72s (± 0.72%) 0.72s (± 0.69%) -0.00s (- 0.42%) 0.71s 0.73s
Check Time 5.26s (± 0.47%) 5.23s (± 0.60%) -0.03s (- 0.61%) 5.17s 5.31s
Emit Time 3.44s (± 1.91%) 3.45s (± 1.06%) +0.00s (+ 0.15%) 3.34s 3.52s
Total Time 10.70s (± 0.65%) 10.67s (± 0.49%) -0.03s (- 0.27%) 10.55s 10.82s
material-ui - node (v10.16.3, x64)
Memory used 469,676k (± 0.01%) 476,270k (± 0.01%) +6,593k (+ 1.40%) 476,195k 476,418k
Parse Time 1.82s (± 0.64%) 1.81s (± 0.56%) -0.01s (- 0.38%) 1.79s 1.83s
Bind Time 0.68s (± 0.77%) 0.68s (± 0.86%) -0.00s (- 0.29%) 0.66s 0.69s
Check Time 14.28s (± 0.28%) 21.49s (± 1.05%) 🔻+7.22s (+50.54%) 20.93s 21.92s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.78s (± 0.26%) 23.99s (± 0.95%) 🔻+7.21s (+42.94%) 23.42s 24.43s
xstate - node (v10.16.3, x64)
Memory used 580,507k (± 1.90%) 570,779k (± 0.01%) -9,728k (- 1.68%) 570,598k 570,898k
Parse Time 2.60s (± 0.39%) 2.58s (± 0.37%) -0.02s (- 0.65%) 2.57s 2.61s
Bind Time 1.01s (± 0.81%) 1.01s (± 0.40%) -0.00s (- 0.20%) 1.00s 1.02s
Check Time 1.53s (± 0.46%) 1.53s (± 0.34%) -0.00s (- 0.13%) 1.52s 1.54s
Emit Time 0.07s (± 4.66%) 0.07s (± 3.14%) -0.00s (- 2.74%) 0.07s 0.08s
Total Time 5.21s (± 0.36%) 5.19s (± 0.29%) -0.02s (- 0.38%) 5.16s 5.23s
Angular - node (v12.1.0, x64)
Memory used 334,491k (± 0.02%) 334,199k (± 0.07%) -292k (- 0.09%) 333,285k 334,431k
Parse Time 2.07s (± 0.28%) 2.08s (± 0.58%) +0.00s (+ 0.10%) 2.05s 2.09s
Bind Time 0.83s (± 0.67%) 0.84s (± 1.14%) +0.00s (+ 0.48%) 0.82s 0.87s
Check Time 5.66s (± 0.26%) 5.62s (± 0.75%) -0.04s (- 0.76%) 5.56s 5.75s
Emit Time 6.33s (± 1.04%) 6.28s (± 0.90%) -0.06s (- 0.93%) 6.18s 6.43s
Total Time 14.90s (± 0.43%) 14.81s (± 0.66%) -0.09s (- 0.60%) 14.65s 15.11s
Compiler-Unions - node (v12.1.0, x64)
Memory used 193,061k (± 0.14%) 193,007k (± 0.13%) -54k (- 0.03%) 192,623k 193,534k
Parse Time 0.84s (± 1.06%) 0.85s (± 1.29%) +0.01s (+ 1.07%) 0.83s 0.88s
Bind Time 0.54s (± 1.20%) 0.54s (± 0.92%) -0.00s (- 0.37%) 0.53s 0.55s
Check Time 7.49s (± 0.55%) 7.52s (± 0.61%) +0.03s (+ 0.36%) 7.40s 7.64s
Emit Time 2.55s (± 1.33%) 2.55s (± 0.98%) -0.01s (- 0.27%) 2.51s 2.61s
Total Time 11.43s (± 0.39%) 11.46s (± 0.58%) +0.02s (+ 0.21%) 11.28s 11.60s
Monaco - node (v12.1.0, x64)
Memory used 326,509k (± 0.07%) 326,473k (± 0.07%) -37k (- 0.01%) 325,818k 326,748k
Parse Time 1.57s (± 0.70%) 1.56s (± 0.85%) -0.00s (- 0.26%) 1.54s 1.60s
Bind Time 0.74s (± 0.78%) 0.75s (± 1.11%) +0.00s (+ 0.40%) 0.73s 0.77s
Check Time 5.60s (± 0.46%) 5.60s (± 0.51%) 0.00s ( 0.00%) 5.52s 5.65s
Emit Time 3.28s (± 1.41%) 3.29s (± 1.69%) +0.01s (+ 0.46%) 3.21s 3.45s
Total Time 11.19s (± 0.52%) 11.21s (± 0.65%) +0.01s (+ 0.12%) 11.04s 11.38s
TFS - node (v12.1.0, x64)
Memory used 289,907k (± 0.01%) 290,026k (± 0.03%) +119k (+ 0.04%) 289,857k 290,284k
Parse Time 1.29s (± 0.82%) 1.31s (± 0.73%) +0.01s (+ 0.85%) 1.28s 1.32s
Bind Time 0.71s (± 0.99%) 0.71s (± 0.70%) -0.00s (- 0.42%) 0.70s 0.72s
Check Time 5.20s (± 0.45%) 5.19s (± 0.37%) -0.02s (- 0.31%) 5.15s 5.24s
Emit Time 3.48s (± 0.71%) 3.50s (± 0.56%) +0.01s (+ 0.37%) 3.46s 3.54s
Total Time 10.69s (± 0.35%) 10.70s (± 0.31%) +0.01s (+ 0.12%) 10.62s 10.76s
material-ui - node (v12.1.0, x64)
Memory used 448,521k (± 0.08%) 455,355k (± 0.01%) +6,834k (+ 1.52%) 455,264k 455,442k
Parse Time 1.81s (± 0.61%) 1.82s (± 0.62%) +0.01s (+ 0.28%) 1.79s 1.84s
Bind Time 0.65s (± 0.85%) 0.64s (± 0.74%) -0.01s (- 1.38%) 0.63s 0.65s
Check Time 13.05s (± 1.05%) 19.07s (± 1.60%) 🔻+6.02s (+46.13%) 18.56s 19.68s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.52s (± 0.91%) 21.54s (± 1.44%) 🔻+6.02s (+38.81%) 21.03s 22.14s
xstate - node (v12.1.0, x64)
Memory used 536,356k (± 0.02%) 536,463k (± 0.01%) +106k (+ 0.02%) 536,363k 536,603k
Parse Time 2.54s (± 0.47%) 2.53s (± 0.44%) -0.01s (- 0.28%) 2.51s 2.56s
Bind Time 1.04s (± 0.46%) 1.04s (± 0.54%) -0.01s (- 0.48%) 1.03s 1.05s
Check Time 1.48s (± 0.63%) 1.48s (± 0.68%) +0.00s (+ 0.13%) 1.46s 1.51s
Emit Time 0.07s (± 0.00%) 0.07s (± 3.14%) +0.00s (+ 1.43%) 0.07s 0.08s
Total Time 5.14s (± 0.41%) 5.12s (± 0.38%) -0.01s (- 0.29%) 5.09s 5.16s
Angular - node (v14.15.1, x64)
Memory used 332,740k (± 0.01%) 332,514k (± 0.01%) -227k (- 0.07%) 332,472k 332,554k
Parse Time 2.04s (± 0.69%) 2.04s (± 0.86%) +0.00s (+ 0.25%) 2.01s 2.10s
Bind Time 0.87s (± 0.42%) 0.87s (± 0.86%) +0.00s (+ 0.12%) 0.86s 0.89s
Check Time 5.63s (± 0.54%) 5.62s (± 0.52%) -0.01s (- 0.23%) 5.57s 5.70s
Emit Time 6.30s (± 0.71%) 6.36s (± 1.09%) +0.06s (+ 0.98%) 6.24s 6.56s
Total Time 14.84s (± 0.54%) 14.89s (± 0.53%) +0.05s (+ 0.36%) 14.77s 15.09s
Compiler-Unions - node (v14.15.1, x64)
Memory used 194,627k (± 0.49%) 193,966k (± 0.61%) -661k (- 0.34%) 191,949k 195,297k
Parse Time 0.86s (± 0.46%) 0.86s (± 0.69%) +0.00s (+ 0.46%) 0.85s 0.88s
Bind Time 0.57s (± 1.76%) 0.57s (± 0.66%) -0.00s (- 0.35%) 0.56s 0.57s
Check Time 7.52s (± 0.57%) 7.55s (± 0.68%) +0.03s (+ 0.40%) 7.45s 7.66s
Emit Time 2.50s (± 0.97%) 2.50s (± 0.93%) +0.01s (+ 0.24%) 2.46s 2.57s
Total Time 11.45s (± 0.51%) 11.48s (± 0.53%) +0.04s (+ 0.31%) 11.33s 11.61s
Monaco - node (v14.15.1, x64)
Memory used 325,421k (± 0.01%) 325,411k (± 0.00%) -10k (- 0.00%) 325,386k 325,458k
Parse Time 1.59s (± 0.70%) 1.58s (± 0.71%) -0.01s (- 0.57%) 1.55s 1.60s
Bind Time 0.78s (± 0.57%) 0.78s (± 0.79%) +0.00s (+ 0.13%) 0.77s 0.79s
Check Time 5.52s (± 0.60%) 5.48s (± 0.48%) -0.04s (- 0.71%) 5.43s 5.54s
Emit Time 3.33s (± 0.92%) 3.30s (± 0.90%) -0.03s (- 0.81%) 3.25s 3.37s
Total Time 11.21s (± 0.47%) 11.14s (± 0.50%) -0.07s (- 0.60%) 11.03s 11.28s
TFS - node (v14.15.1, x64)
Memory used 288,919k (± 0.01%) 288,917k (± 0.01%) -2k (- 0.00%) 288,851k 288,993k
Parse Time 1.33s (± 1.01%) 1.34s (± 1.25%) +0.01s (+ 1.06%) 1.30s 1.38s
Bind Time 0.74s (± 1.01%) 0.74s (± 0.67%) -0.00s (- 0.14%) 0.73s 0.75s
Check Time 5.17s (± 0.41%) 5.16s (± 0.68%) -0.00s (- 0.10%) 5.08s 5.23s
Emit Time 3.54s (± 1.88%) 3.56s (± 1.67%) +0.02s (+ 0.51%) 3.39s 3.66s
Total Time 10.78s (± 0.76%) 10.80s (± 0.88%) +0.02s (+ 0.21%) 10.52s 10.94s
material-ui - node (v14.15.1, x64)
Memory used 447,053k (± 0.01%) 453,656k (± 0.00%) +6,603k (+ 1.48%) 453,607k 453,690k
Parse Time 1.87s (± 0.36%) 1.88s (± 0.53%) +0.01s (+ 0.59%) 1.86s 1.90s
Bind Time 0.71s (± 0.73%) 0.71s (± 1.19%) 0.00s ( 0.00%) 0.69s 0.72s
Check Time 13.16s (± 0.47%) 19.08s (± 1.49%) 🔻+5.92s (+45.01%) 18.58s 19.62s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.73s (± 0.42%) 21.67s (± 1.32%) 🔻+5.94s (+37.73%) 21.16s 22.20s
xstate - node (v14.15.1, x64)
Memory used 534,095k (± 0.01%) 534,101k (± 0.00%) +6k (+ 0.00%) 534,029k 534,150k
Parse Time 2.59s (± 0.56%) 2.60s (± 0.50%) +0.01s (+ 0.39%) 2.56s 2.63s
Bind Time 1.15s (± 0.35%) 1.15s (± 0.87%) +0.00s (+ 0.17%) 1.14s 1.18s
Check Time 1.52s (± 0.55%) 1.52s (± 0.55%) 0.00s ( 0.00%) 1.50s 1.54s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.33s (± 0.31%) 5.34s (± 0.33%) +0.01s (+ 0.11%) 5.30s 5.38s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory3 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v10.16.3, x64)
  • xstate - node (v12.1.0, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 48448 10
Baseline main 10

Developer Information:

Download Benchmark

@weswigham
Copy link
Member

Oof, this is still reporting a 50% check time regression in material-ui.

@erikbrinkman
Copy link
Contributor Author

@typescript-bot perf test this

@erikbrinkman
Copy link
Contributor Author

The "problem" is that material-ui has some types that are large unions ~700 types, and at least some have ~100 optional types. discriminateTypeByDiscriminableItems has complexity O(number of types in union x number of discriminators) and they way I naively implemented this extension required checking all of them. This new version only adds member names that discriminate against different sets of types in the union. This trick doesn't change the worst case complexity of this change, but it should prevent common / degenerate cases like material-ui.

It's not clear to me that the performance could be improved any more, so if it's still bad somehow, or is deemed too complex for the handful of edge cases it helps, I'm cool with abandoning in.

As an implementation side note, I needed a way to hash subsets of the union type, and I did this by forming a string with space delimited indices. This works, but I'm not sure if there's a better way to achieve this same result.

The first attempt at fixing this issue caused a large regression on one
benchmark. This new PR addresses the regression by caching optional
members of union types when they're used for discrimination. This seems
to reduce the performance hit at the cost of some extra memory, but it's
not clear if the fix is worth this regression.
@weswigham
Copy link
Member

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 27, 2022

Heya @weswigham, I've started to run the perf test suite on this PR at cd00855. You can monitor the build here.

Update: The results are in!

@weswigham
Copy link
Member

As an implementation side note, I needed a way to hash subsets of the union type, and I did this by forming a string with space delimited indices. This works, but I'm not sure if there's a better way to achieve this same result.

getTypeListId already exists and has a similar implementation. Slightly more compressed for long runs of consecutive type ids.

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..48448

Metric main 48448 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 356,681k (± 0.02%) 356,443k (± 0.02%) -237k (- 0.07%) 356,321k 356,603k
Parse Time 2.07s (± 0.48%) 2.08s (± 0.40%) +0.00s (+ 0.14%) 2.05s 2.09s
Bind Time 0.87s (± 0.57%) 0.87s (± 0.69%) -0.00s (- 0.12%) 0.86s 0.88s
Check Time 5.81s (± 0.65%) 5.84s (± 0.36%) +0.03s (+ 0.57%) 5.80s 5.89s
Emit Time 6.02s (± 0.64%) 6.04s (± 0.71%) +0.01s (+ 0.23%) 5.95s 6.16s
Total Time 14.77s (± 0.44%) 14.82s (± 0.39%) +0.05s (+ 0.33%) 14.71s 14.97s
Compiler-Unions - node (v10.16.3, x64)
Memory used 205,764k (± 0.03%) 205,820k (± 0.04%) +56k (+ 0.03%) 205,662k 205,965k
Parse Time 0.85s (± 0.76%) 0.85s (± 0.61%) -0.00s (- 0.12%) 0.84s 0.86s
Bind Time 0.52s (± 1.00%) 0.52s (± 1.00%) +0.00s (+ 0.39%) 0.51s 0.53s
Check Time 8.01s (± 0.56%) 8.02s (± 0.54%) +0.00s (+ 0.04%) 7.94s 8.14s
Emit Time 2.54s (± 1.82%) 2.51s (± 0.89%) -0.03s (- 1.22%) 2.47s 2.57s
Total Time 11.93s (± 0.70%) 11.90s (± 0.46%) -0.03s (- 0.23%) 11.77s 12.05s
Monaco - node (v10.16.3, x64)
Memory used 343,686k (± 0.01%) 343,676k (± 0.03%) -10k (- 0.00%) 343,409k 343,847k
Parse Time 1.58s (± 1.11%) 1.59s (± 0.63%) +0.01s (+ 0.76%) 1.57s 1.61s
Bind Time 0.76s (± 0.88%) 0.76s (± 0.85%) +0.00s (+ 0.13%) 0.74s 0.77s
Check Time 5.76s (± 0.33%) 5.76s (± 0.38%) +0.01s (+ 0.10%) 5.71s 5.80s
Emit Time 3.24s (± 0.45%) 3.25s (± 0.87%) +0.01s (+ 0.22%) 3.17s 3.32s
Total Time 11.33s (± 0.36%) 11.35s (± 0.42%) +0.02s (+ 0.19%) 11.25s 11.49s
TFS - node (v10.16.3, x64)
Memory used 305,268k (± 0.02%) 305,330k (± 0.01%) +62k (+ 0.02%) 305,234k 305,393k
Parse Time 1.28s (± 0.47%) 1.29s (± 0.82%) +0.00s (+ 0.08%) 1.27s 1.32s
Bind Time 0.72s (± 0.72%) 0.72s (± 0.66%) -0.00s (- 0.28%) 0.71s 0.73s
Check Time 5.26s (± 0.47%) 5.24s (± 0.51%) -0.02s (- 0.34%) 5.18s 5.28s
Emit Time 3.44s (± 1.91%) 3.44s (± 0.91%) -0.00s (- 0.06%) 3.35s 3.53s
Total Time 10.70s (± 0.65%) 10.69s (± 0.54%) -0.02s (- 0.17%) 10.58s 10.81s
material-ui - node (v10.16.3, x64)
Memory used 469,676k (± 0.01%) 465,079k (± 0.01%) -4,598k (- 0.98%) 464,971k 465,192k
Parse Time 1.82s (± 0.64%) 1.83s (± 0.60%) +0.01s (+ 0.49%) 1.80s 1.85s
Bind Time 0.68s (± 0.77%) 0.68s (± 0.70%) +0.00s (+ 0.59%) 0.67s 0.69s
Check Time 14.28s (± 0.28%) 14.94s (± 0.64%) +0.66s (+ 4.65%) 14.70s 15.16s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.78s (± 0.26%) 17.46s (± 0.59%) +0.67s (+ 4.00%) 17.19s 17.69s
xstate - node (v10.16.3, x64)
Memory used 580,507k (± 1.90%) 571,511k (± 0.02%) -8,996k (- 1.55%) 571,274k 571,894k
Parse Time 2.60s (± 0.39%) 2.59s (± 0.26%) -0.01s (- 0.31%) 2.58s 2.61s
Bind Time 1.01s (± 0.81%) 1.02s (± 0.58%) +0.00s (+ 0.30%) 1.00s 1.03s
Check Time 1.53s (± 0.46%) 1.54s (± 0.82%) +0.01s (+ 0.65%) 1.51s 1.56s
Emit Time 0.07s (± 4.66%) 0.07s (± 3.14%) -0.00s (- 2.74%) 0.07s 0.08s
Total Time 5.21s (± 0.36%) 5.22s (± 0.39%) +0.01s (+ 0.13%) 5.18s 5.26s
Angular - node (v12.1.0, x64)
Memory used 334,491k (± 0.02%) 334,276k (± 0.03%) -215k (- 0.06%) 334,036k 334,420k
Parse Time 2.07s (± 0.28%) 2.07s (± 0.83%) -0.00s (- 0.14%) 2.03s 2.10s
Bind Time 0.83s (± 0.67%) 0.84s (± 1.08%) +0.01s (+ 0.84%) 0.82s 0.86s
Check Time 5.66s (± 0.26%) 5.70s (± 0.44%) +0.04s (+ 0.64%) 5.63s 5.73s
Emit Time 6.33s (± 1.04%) 6.34s (± 1.14%) +0.00s (+ 0.05%) 6.23s 6.56s
Total Time 14.90s (± 0.43%) 14.95s (± 0.58%) +0.05s (+ 0.32%) 14.81s 15.19s
Compiler-Unions - node (v12.1.0, x64)
Memory used 193,061k (± 0.14%) 193,401k (± 0.05%) +340k (+ 0.18%) 193,174k 193,625k
Parse Time 0.84s (± 1.06%) 0.85s (± 1.11%) +0.00s (+ 0.36%) 0.83s 0.87s
Bind Time 0.54s (± 1.20%) 0.54s (± 0.89%) -0.00s (- 0.19%) 0.53s 0.55s
Check Time 7.49s (± 0.55%) 7.54s (± 0.41%) +0.04s (+ 0.57%) 7.47s 7.61s
Emit Time 2.55s (± 1.33%) 2.56s (± 1.42%) +0.01s (+ 0.35%) 2.49s 2.65s
Total Time 11.43s (± 0.39%) 11.49s (± 0.48%) +0.05s (+ 0.48%) 11.35s 11.62s
Monaco - node (v12.1.0, x64)
Memory used 326,509k (± 0.07%) 326,604k (± 0.06%) +95k (+ 0.03%) 325,923k 326,851k
Parse Time 1.57s (± 0.70%) 1.55s (± 0.82%) -0.01s (- 0.96%) 1.52s 1.58s
Bind Time 0.74s (± 0.78%) 0.74s (± 0.66%) +0.00s (+ 0.13%) 0.73s 0.75s
Check Time 5.60s (± 0.46%) 5.62s (± 0.44%) +0.02s (+ 0.39%) 5.55s 5.66s
Emit Time 3.28s (± 1.41%) 3.28s (± 0.64%) +0.00s (+ 0.06%) 3.24s 3.34s
Total Time 11.19s (± 0.52%) 11.21s (± 0.24%) +0.01s (+ 0.13%) 11.14s 11.25s
TFS - node (v12.1.0, x64)
Memory used 289,907k (± 0.01%) 289,911k (± 0.07%) +4k (+ 0.00%) 289,160k 290,093k
Parse Time 1.29s (± 0.82%) 1.29s (± 0.69%) -0.00s (- 0.08%) 1.27s 1.31s
Bind Time 0.71s (± 0.99%) 0.71s (± 0.73%) +0.00s (+ 0.28%) 0.70s 0.72s
Check Time 5.20s (± 0.45%) 5.17s (± 0.66%) -0.03s (- 0.60%) 5.09s 5.25s
Emit Time 3.48s (± 0.71%) 3.48s (± 0.83%) -0.00s (- 0.09%) 3.42s 3.56s
Total Time 10.69s (± 0.35%) 10.66s (± 0.53%) -0.03s (- 0.29%) 10.53s 10.80s
material-ui - node (v12.1.0, x64)
Memory used 448,521k (± 0.08%) 444,144k (± 0.01%) -4,377k (- 0.98%) 444,043k 444,212k
Parse Time 1.81s (± 0.61%) 1.83s (± 0.37%) +0.01s (+ 0.66%) 1.81s 1.84s
Bind Time 0.65s (± 0.85%) 0.65s (± 0.89%) -0.00s (- 0.31%) 0.64s 0.67s
Check Time 13.05s (± 1.05%) 13.40s (± 0.55%) +0.35s (+ 2.67%) 13.24s 13.58s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.52s (± 0.91%) 15.88s (± 0.47%) +0.36s (+ 2.31%) 15.71s 16.03s
xstate - node (v12.1.0, x64)
Memory used 536,356k (± 0.02%) 537,078k (± 0.01%) +722k (+ 0.13%) 536,994k 537,257k
Parse Time 2.54s (± 0.47%) 2.54s (± 0.48%) +0.00s (+ 0.12%) 2.50s 2.56s
Bind Time 1.04s (± 0.46%) 1.05s (± 0.64%) +0.00s (+ 0.38%) 1.04s 1.06s
Check Time 1.48s (± 0.63%) 1.48s (± 0.59%) +0.00s (+ 0.07%) 1.46s 1.50s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.14s (± 0.41%) 5.14s (± 0.29%) +0.00s (+ 0.04%) 5.11s 5.17s
Angular - node (v14.15.1, x64)
Memory used 332,740k (± 0.01%) 332,513k (± 0.01%) -227k (- 0.07%) 332,425k 332,617k
Parse Time 2.04s (± 0.69%) 2.04s (± 0.37%) -0.00s (- 0.25%) 2.02s 2.05s
Bind Time 0.87s (± 0.42%) 0.88s (± 0.68%) +0.01s (+ 1.15%) 0.87s 0.89s
Check Time 5.63s (± 0.54%) 5.66s (± 0.50%) +0.03s (+ 0.48%) 5.62s 5.75s
Emit Time 6.30s (± 0.71%) 6.32s (± 0.61%) +0.02s (+ 0.30%) 6.24s 6.38s
Total Time 14.84s (± 0.54%) 14.89s (± 0.40%) +0.06s (+ 0.37%) 14.76s 15.05s
Compiler-Unions - node (v14.15.1, x64)
Memory used 194,627k (± 0.49%) 194,649k (± 0.50%) +22k (+ 0.01%) 191,951k 195,382k
Parse Time 0.86s (± 0.46%) 0.86s (± 0.69%) +0.00s (+ 0.46%) 0.85s 0.88s
Bind Time 0.57s (± 1.76%) 0.56s (± 1.05%) -0.00s (- 0.53%) 0.55s 0.58s
Check Time 7.52s (± 0.57%) 7.55s (± 0.49%) +0.03s (+ 0.47%) 7.47s 7.63s
Emit Time 2.50s (± 0.97%) 2.51s (± 0.63%) +0.02s (+ 0.60%) 2.47s 2.55s
Total Time 11.45s (± 0.51%) 11.50s (± 0.42%) +0.05s (+ 0.43%) 11.40s 11.60s
Monaco - node (v14.15.1, x64)
Memory used 325,421k (± 0.01%) 325,425k (± 0.01%) +4k (+ 0.00%) 325,383k 325,453k
Parse Time 1.59s (± 0.70%) 1.59s (± 0.57%) +0.00s (+ 0.25%) 1.57s 1.61s
Bind Time 0.78s (± 0.57%) 0.78s (± 0.79%) +0.00s (+ 0.13%) 0.76s 0.79s
Check Time 5.52s (± 0.60%) 5.52s (± 0.53%) +0.00s (+ 0.05%) 5.46s 5.58s
Emit Time 3.33s (± 0.92%) 3.31s (± 0.45%) -0.02s (- 0.48%) 3.29s 3.34s
Total Time 11.21s (± 0.47%) 11.20s (± 0.36%) -0.01s (- 0.06%) 11.12s 11.29s
TFS - node (v14.15.1, x64)
Memory used 288,919k (± 0.01%) 288,914k (± 0.01%) -5k (- 0.00%) 288,864k 288,955k
Parse Time 1.33s (± 1.01%) 1.32s (± 1.26%) -0.00s (- 0.08%) 1.30s 1.37s
Bind Time 0.74s (± 1.01%) 0.74s (± 1.00%) +0.00s (+ 0.41%) 0.73s 0.76s
Check Time 5.17s (± 0.41%) 5.16s (± 0.46%) -0.01s (- 0.15%) 5.10s 5.21s
Emit Time 3.54s (± 1.88%) 3.54s (± 2.12%) -0.00s (- 0.06%) 3.39s 3.69s
Total Time 10.78s (± 0.76%) 10.77s (± 0.76%) -0.01s (- 0.08%) 10.60s 10.96s
material-ui - node (v14.15.1, x64)
Memory used 447,053k (± 0.01%) 442,269k (± 0.08%) -4,784k (- 1.07%) 441,317k 442,567k
Parse Time 1.87s (± 0.36%) 1.87s (± 0.60%) +0.01s (+ 0.27%) 1.85s 1.89s
Bind Time 0.71s (± 0.73%) 0.71s (± 0.87%) 0.00s ( 0.00%) 0.70s 0.72s
Check Time 13.16s (± 0.47%) 13.52s (± 1.15%) +0.36s (+ 2.70%) 13.24s 13.94s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.73s (± 0.42%) 16.10s (± 0.98%) +0.36s (+ 2.30%) 15.79s 16.52s
xstate - node (v14.15.1, x64)
Memory used 534,095k (± 0.01%) 534,794k (± 0.01%) +700k (+ 0.13%) 534,728k 534,848k
Parse Time 2.59s (± 0.56%) 2.59s (± 0.66%) +0.01s (+ 0.39%) 2.56s 2.62s
Bind Time 1.15s (± 0.35%) 1.15s (± 0.66%) +0.00s (+ 0.35%) 1.14s 1.17s
Check Time 1.52s (± 0.55%) 1.53s (± 0.50%) +0.01s (+ 0.92%) 1.52s 1.55s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.33s (± 0.31%) 5.36s (± 0.33%) +0.02s (+ 0.45%) 5.31s 5.39s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory3 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v10.16.3, x64)
  • xstate - node (v12.1.0, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 48448 10
Baseline main 10

Developer Information:

Download Benchmark

@weswigham
Copy link
Member

Well, that's much better, that's for sure. 2-5% worse is a lot better than 30-50% worse.

@erikbrinkman
Copy link
Contributor Author

It was a good thing this was reverted, because the PR is actually unsound, and it was caught from the local serve build. The problem is a few fold, and so I'm writing here to seek guidance. As a general note, I'm unsure how this should function in exactOptionalPropertyTypes land.

The original PR sought to add any discriminable property of any optional property of any element of the union type as a discriminator checking for assignability to undefined. This concept had some performance hits that I was able to address, but there was actually an underlying bug in the test. If an "unrelated" type had optional members, those would still get added to the discriminator list. However in discriminateTypeByDiscriminableItems if a type doesn't have that property, it's immediately marked as not discriminable. This vaguely makes sense, but doesn't actually align with what this was trying to do, which is inherently check for assignability to undefined as a way to check for missing member as part of a discriminated union. I tried defaulting to undefined in discriminateTypeByDiscriminableItems but that created a few other problems.

Either way, this premise of checking for assignability to undefined of optional members is actually flawed. In principle what we actually want to is check for all potential properties in the union type that are missing from the original object for their assignability to missing. Ignoring exactOptionalPropertyTypes this should be possible to do by checking assignability to undefined, but as currently written, this would need a new path discriminateTypeByDiscriminableItems and seems potentially error prone, and potentially very expensive.

As I write this, another idea comes to mind, which seems less "correct" but may still be able to achieve the ideal result: In principle this is still meant to help figure out "discriminated unions", so instead of caching all properties, or all optional properties, we could specifically look for optional discriminated unions, e.g. the property is optional on one type and non-optional on any other members. By itself this will still fail on the union of two disjoint optional discriminated unions (which seems unlikely). But instead of checking all discriminators we check them independently for the set of types they discriminate against. Worst case this could be really bad, but I think in most common usages it might not even require an extra check.

I'm curious as to general thoughts on these different approaches. I realize this isn't very important, so maybe I'm overthinking this.

@erikbrinkman
Copy link
Contributor Author

After some more experimentation, I think the goal of the PR as stated is out of line with the way typescript handles discriminant unions, and the current handling is sufficient. I'll open up an issue (or search to see if one exists) about better union discrimination.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate recent regression on Material UI from contextual discrimination changes
3 participants