Performance: variance check balloons number of instantations and degrades performance

[andreialecu]

Bug Report

The Mongoose project recently introduced their own typescript types, therefore deprecating the DefinitelyTyped types.

However, the new types suffer from some big performance issues, greatly slowing down the VS Code developer experience. I've been investigating the problem and noticed that the issue seems to occur from some extends checks.

Related issues in mongoose repo:
Automattic/mongoose#10349
Automattic/mongoose#10487
Automattic/mongoose#10492

🔎 Search Terms

performance extends instantiations recursiveTypeRelatedTo

🕗 Version & Regression Information

This occurs on TS 4.3.5.

⏯ Playground Link

Unfortunately no playground link, but I made a very simple repro at: https://github.com/andreialecu/typescript-performance-extends

Take a look at the README there.

💻 Code

This diff in mongoose's types seems to completely eradicate the problem:

diff --git a/types/mongoose.d.ts b/types/mongoose.d.ts
index 9e0ad99..2fab5ff 100644
--- a/types/mongoose.d.ts
+++ b/types/mongoose.d.ts
@@ -1232,7 +1232,7 @@ declare module 'mongoose' {
   type PostMiddlewareFunction<ThisType, ResType = any> = (this: ThisType, res: ResType, next: (err?: CallbackError) => void) => void | Promise<void>;
   type ErrorHandlingMiddlewareFunction<ThisType, ResType = any> = (this: ThisType, err: NativeError, res: ResType, next: (err?: CallbackError) => void) => void;
 
-  class Schema<DocType = Document, M extends Model<DocType, any, any> = Model<any, any, any>, SchemaDefinitionType = undefined, TInstanceMethods = ExtractMethods<M>> extends events.EventEmitter {
+  class Schema<DocType = Document, M = Model<any, any, any>, SchemaDefinitionType = undefined, TInstanceMethods = any> extends events.EventEmitter {
     /**
      * Create a new schema
      */

The difference between before and after are:
Before:

Identifiers:                45586
Symbols:                    76446
Types:                      20569
Instantiations:             93884
Memory used:               93011K
Assignability cache size:    5146
Check time:                 0.67s

After:

Identifiers:                45582
Symbols:                    27736
Types:                        710
Instantiations:               818
Memory used:               59482K
Check time:                 0.03s

I realize the code after the diff isn't exactly equivalent, but the problem seems to be related to the extends check. Avoiding extends gets rid of the issue.

Here's a cpu profile:

cpuprofile.zip

The 'before' version shows:

🙁 Actual behavior

Slow type checking performance, especially in VS code.

🙂 Expected behavior

Better performance.

---

I opened this in the hope that there is value in investigating the repro by the TS team, in order to check whether any performance improvements in cases such as this one are possible.

[andreialecu]

Tagging @amcasey as per request in a comment in a similar thread about editor performance issues: #34801 (comment)

[andreialecu]

I stumbled upon https://www.npmjs.com/package/@amcasey/ts-analyze-trace which seems super useful. It should really be mentioned in the wiki page here: https://github.com/microsoft/TypeScript/wiki/Performance-Tracing

Here's the output on the repro above:

Hot Spots
└─ Check file /users/andreialecu/work/contrib/tsc-perf/index.ts (533ms)
   └─ Check variable declaration from (line 11, char 14) to (line 17, char 3) (529ms)
      └─ Check expression from (line 11, char 27) to (line 17, char 3) (529ms)
         └─ Determine variance of type 91 (513ms)
            └─ Compare types 104 and 103 (411ms)
               └─ Compare types 174 and 175 (380ms)
                  └─ Compare types 212 and 209 (378ms)
                     └─ Compare types 212 and 208 (372ms)
                        └─ Compare types 211 and 208 (372ms)
                           └─ Determine variance of type 183 (286ms)
                              └─ {"id":183,"kind":"GenericType","name":"Query","typeArguments":[184,185,186,187],"location":{"path":"/users/andreialecu/work/contrib/tsc-perf/types/mongoose.d.ts","line":2080,"char":3}}
                                 ├─ {"id":184,"kind":"TypeParameter","name":"ResultType","location":{"path":"/users/andreialecu/work/contrib/tsc-perf/types/mongoose.d.ts","line":2080,"char":15}}
                                 ├─ {"id":185,"kind":"TypeParameter","name":"DocType","location":{"path":"/users/andreialecu/work/contrib/tsc-perf/types/mongoose.d.ts","line":2080,"char":27}}
                                 ├─ {"id":186,"kind":"TypeParameter","name":"THelpers","location":{"path":"/users/andreialecu/work/contrib/tsc-perf/types/mongoose.d.ts","line":2080,"char":36}}
                                 └─ {"id":187,"kind":"TypeParameter","name":"RawDocType","location":{"path":"/users/andreialecu/work/contrib/tsc-perf/types/mongoose.d.ts","line":2080,"char":51}}

Seems that the hot spot is: Compare types 104 and 103. Types 104 and 103 are:

{"id":103,"symbolName":"Model","recursionId":77,"instantiatedType":91,"typeArguments":[41,93,94],"firstDeclaration":{"path":"/users/andreialecu/work/contrib/tsc-perf/types/mongoose.d.ts","start":{"line":757,"character":15},"end":{"line":757,"character":33}},"flags":["524288"]},
{"id":104,"symbolName":"Model","recursionId":77,"instantiatedType":91,"typeArguments":[42,93,94],"firstDeclaration":{"path":"/users/andreialecu/work/contrib/tsc-perf/types/mongoose.d.ts","start":{"line":757,"character":15},"end":{"line":757,"character":33}},"flags":["524288"]},

From what I can see the only difference is in the typeArguments, but types 41-43 show no relevant information. They're the only blank ones from types.json:

{"id":41,"flags":["262144"]},
{"id":42,"flags":["262144"]},
{"id":43,"flags":["262144"]},

[andreialecu]

An even simpler way to trigger this seems to be (grab mongoose.d.ts from: https://cdn.jsdelivr.net/npm/[email protected]/index.d.ts):

import { Model } from "mongoose";
class Foo<T extends Model<any>> {} // add/remove the `extends` check and run `yarn tsc --extendedDiagnostics`
export const UserSchema = new Foo<Model<{}>>();

// with: class Foo<T> {}
Types:                         92
Instantiations:                 2

// with: class Foo<T extends Model<any>> {}
Types:                       42259
Instantiations:             291012

[sandersn]

As we discussed on discord, I think the problem is that structural assignability check is expensive, and the variance check is what triggers the structural assignability. It might be possible to skip variance checking in your specific case if the types were treated more nominally. I think the interesting question is whether production code would be able to avoid a variance check.

[andreialecu]

Thank you, @sandersn, for a very insightful discord discussion! 🙂

For reference, here's how the types would be used, idiomatically, using mongoose, as per the documentation: https://mongoosejs.com/docs/typescript.html

It's possible I'm overlooking something but I'm not sure if variance checking is necessary in this case, since the types are nominal here and a base interface for the "schema" would always be defined and used.

[andreialecu]

Hope you don't mind the ping @amcasey (and/or @weswigham)

I noticed this was tagged as planned for TD 4.5 but I was wondering if there's anything you could point us to in the mean time. (This has been an ongoing problem in mongoose and it has been making life painful for everyone since the types got moved off Definitely Typed.)

Improving typescript performance has been a priority for mongoose maintainers but seems that the root cause still can't be found. Any pointers would be hugely appreciated. Thank you!

[Andarist]

It seems like the upcoming variance annotations in the 4.7 would allow mitigating this problem, right?

[andrewbranch]

One would think. I did some quick experimentation and saw some very slight improvement. When I come back to it I’ll do it a bit more scientifically with some tracing. Though if someone on the mongoose side beats me to it, even better.

[andrewbranch]

@andreialecu do you get similarly bad performance with mongoose v6?

[andreialecu]

@andrewbranch, yes, it's still noticeably slow, but I learned to deal with it.

I'm on a Macbook Pro M1 Max FWIW, which should be a powerhouse.

Here's my VSCode experience with mongoose 6.3.9 - notice how it takes a couple of seconds for the errors to update:

Screen.Recording.2022-06-23.at.10.27.38.mov

This capture above was pretty fast, but it can take up to 10 seconds to update at times.

[a3957273]

Hot Spots
└─ Check file /home/ec2-user/git/Bailo/server/models/Deployment.ts (1920ms)
   └─ Check variable declaration from (line 70, char 7) to (line 70, char 74) (1830ms)
      └─ Check expression from (line 70, char 25) to (line 70, char 74) (1829ms)
         └─ Compare types 383 and 426 (1827ms)
            └─ Determine variance of type 141 (1817ms)
               └─ Compare types 431 and 430 (1675ms)
                  └─ Compare types 548 and 547 (1654ms)
                     └─ Compare types 554 and 553 (1652ms)
                        └─ Determine variance of type 398 (1652ms)
                           └─ Compare types 566 and 565 (1596ms)
                              └─ Compare types 567 and 568 (1595ms)
                                 └─ Compare types 567 and 559 (1554ms)
                                    └─ Compare types 563 and 559 (1554ms)
                                       └─ Determine variance of type 105 (1554ms)
                                          └─ Compare types 570 and 569 (1253ms)
                                             └─ Compare types 607 and 606 (1250ms)
                                                └─ Compare types 626 and 624 (1248ms)
                                                   └─ Compare types 626 and 623 (1248ms)
                                                      └─ Compare types 625 and 623 (1247ms)
                                                         └─ Determine variance of type 609 (1090ms)
                                                            └─ Compare types 5236 and 5235 (568ms)
                                                               └─ Compare types 5238 and 5237 (512ms)
                                                                  └─ Compare types 5242 and 5240 (509ms)
                                                                     └─ Compare types 5242 and 5239 (509ms)
                                                                        └─ Compare types 5254 and 5253 (499ms)
                                                                           └─ Compare types 5258 and 5260 (498ms)
                                                                              └─ Compare types 5257 and 5259 (498ms)
                                                                                 └─ Determine variance of type 150 (498ms)
                                                                                    └─ {"id":150,"kind":"GenericType","name":"Model","typeArguments":[151,152,153],"location":{"path":"/home/ec2-user/git/Bailo/node_modules/mongoose/index.d.ts","line":780,"char":16}}
                                                                                       ├─ {"id":151,"kind":"TypeParameter","name":"T","location":{"path":"/home/ec2-user/git/Bailo/node_modules/mongoose/index.d.ts","line":781,"char":19}}
                                                                                       ├─ {"id":152,"kind":"TypeParameter","name":"TQueryHelpers","location":{"path":"/home/ec2-user/git/Bailo/node_modules/mongoose/index.d.ts","line":781,"char":22}}
                                                                                       └─ {"id":153,"kind":"TypeParameter","name":"TMethods","location":{"path":"/home/ec2-user/git/Bailo/node_modules/mongoose/index.d.ts","line":781,"char":42}}

When compiling this file: https://github.com/gchq/Bailo/blob/main/server/models/Deployment.ts

It seems that a fairly simple model with ~10 attributes still seems to take nearly 2 seconds to type check.

[andrewbranch]

I’m not sure there’s anything currently actionable for us here. Automattic/mongoose#10349 has gotten some renewed attention and mongoose maintainers have a PR up with some improvements. Further up in that thread, I described getting some decent improvements by adding variance annotations to a few of the variance hot spots. However, it’s up to the mongoose folks whether they want to split their types with typesVersions in order to give TypeScript 4.7+ users that boost.