Compiler Option to monitor external dependencies

[Jason3S]

🔍 Search Terms

build, incremental, composit, tsbuildinfo, location, symlink, stale

✅ Viability Checklist

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This isn't a request to add a new utility type: https://github.com/microsoft/TypeScript/wiki/No-New-Utility-Types
• This feature would agree with the rest of our Design Goals: https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals

⭐ Suggestion

Related to: #58427

Proposal: A new option compilerOptions.dependencyTracking

Values:

• local -- this is the current behavior
• external -- this would check for both local and changes in node_modules and package.json. Transitive dependencies should be accounted for if possible.

📃 Motivating Example

The motivation is related to #58427, but extends to mono repos.

The current system seems to work well for monolith repos where there is a tsconfig at the root and it coordinates building all the sub-projects. That is not what I am referring to.

I'm talking about a mono repo that is a collection of related workspaces (packages/libraries) that have their own package.json and tsconfig.json files. The package manager controls the order in which workspaces are built, not tsc. tsc is used as part of the build process in an individual workspace. The output directory is always within an individual workspace, because in theory, each of these packages could be published to NPM. Package imports are controlled by package.json#dependencies and exports are listed in package.json#exports. Interdependence between workspaces is generally handled through SymLinks. This is how PNPM and NPM handle it.

Think of a typical morning workflow:

1. git fetch -- grab all the changes made by my teammates.
2. git pull --rebase origin/main -- merge in the changes since I'm working on my own branch.
3. pnpm i -- install any updated packages.
4. pnpm build -- pnpm will run the build script in each of the workspace based upon the dependency graph (NPM can do the same thing, but you need to define the graph yourself).

At this point, any compile errors should be in my branch, since we keep a clean main branch (always compiles and no errors). If it compiles, then all type checking should be good.

But the current tsc --build . does NOT ensure that the build is correct. It will often miss changes to the .d.ts files in the other workspaces. I was certain this was a bug, see #58427.

Typical inter-workspace workflow:

Given WorkspaceA and depends upon WorkspaceB.

repo
|- WorkspaceA/
|  |- package.json#dependencies.WorkspaceB
|  |- tsconfig.json  
|  |- src/
|  |- dist/ # outDir  
|- WorkspaceB/
|  |- package.json#exports dist/index.js
|  |- tsconfig.json
|  |- src/index.ts
|  |- dist/ # outDir  

If I'm working on WorkspaceA and WorkspaceB by adding a new API interface to WorkspaceB/src/index.ts in WorkspaceB, I would expect the following to pick up the changes in B and work.

repo/WorkspaceB $ tsc -b .
repo/WorkspaceB $ cd ../WorkspaceA
repo/WorksapceA $ tsc -b .

But it doesn't pick up the change in WorkspaceB.

Instead I have to remember to use:

repo/WorksapceA $ tsc -b . -f

Unexpected results and errors can arise by needing to use two different build commands based upon the situation.

Strangely, things magically seem to work if I had been running the following before making changes to WorkspaceB:

repo/WorksapceA $ tsc -b . -w

In summary, I am asking for the option to have tsc --build work a bit harder and check the external dependencies by using an option like compilerOptions.dependencyTracking=external.

Please note: I rarely use the --build option in projects because of this build issue. Some times I am forced to because the only way to make something work is by using composite.

💻 Use Cases

1. What do you want to use this for?
   To have the compiler build reliably when there are changes, including external ones.
2. What shortcomings exist with current approaches?
   The current option is to use tsc -b . -f all the time.
3. What workarounds are you using in the meantime?
   Always use force build.

[JasonKleban]

Yes, I believe this keeps biting me - local builds work until I clean out all the tsconfig.tsbuildinfo (forcing rebuild) and THEN I see the errors my gated builds caught. This seems to make incremental builds kind of useless for any project large enough to really benefit from them. Maybe I'm misunderstanding the scope of the problem though.

Is it true that incremental builds only track local source files, but not changes to dependencies - whether "workspace:*" or otherwise? Then yeah, I'll need to disable incremental builds. But maybe there's a nuance here that a corrected setup can work as expected?