High CPU when using --watch on 2000+ files (roughly 40megs)

[eyalellenbogen]

Code

tsc -w

TypeScript version:
2.6.1 (reproduces in previous versions as well)

Expected behavior:
To not hog CPU when watching (not while compiling).

Actual behavior:
tsc node process occupies 30-40% CPU while idle in watch.

[DanielRosenwasser]

Do you mean even while tsc is just waiting for changes?

[mhegazy]

Can you share the project? we would be happy to sign any required NDAs.

[DanielRosenwasser]

Also, can you give us more information about your machines? e.g. Node version, OS, architecture, CPU count, etc.?

[ip]

I'm also experiencing this on macOS 10.13.1, with Node v8.9.0. Although it's usually 3%, but it should be almost 0, and occasionally it occupied all 100% which was gone after the restart.

[mhegazy]

@ip can you please share a repro project?

[eyalellenbogen]

Sorry for the late response guys. Unfortunately, I can't share the repo yet. Will get back to you about the NDA this week. It's happening on Win10 1511 to 1709 with node 6 and 8. Yes, it's hogging the CPU while simply watching for changes (before I save anything). The machine is i7 with 16gig ram and an SSD. The project is an angular 1.5 typescript project using the old school name spaces and references. I'll try and get more info about it and maybe somehow share the project this coming week.

[aszczepanek]

I'm also experiencing this issue.
Machine i7 3632qm 12GB ram, SSD.

Project has almost 2000 ts files, and compiling in watch mode uses about 6-7% CPU constantly when waiting on changes:
18:59:26 - Compilation complete. Watching for file changes.
Unfortunately I cannot share project source, because it's closed software of my company.

I've seen similar issue when using grunt-watch task, which uses pooling to detect file changes. It's good for small projects but it's starting to stress cpu just watching hundreds of files. We switched to our custom implementation of watch using node fs.watch, which uses "native" host os watching for changes feature which is much more efficient with cpu than pooling (and much faster).
Isn't that the case? I couldn't find info about what typescipt compiler uses internally for watching changes.

// Edit
Actually I started to investigating this issue and confirmed it is related to pooling watcher mode. I found that you can set TSC_NONPOLLING_WATCHER env var to 1, and then tsc -w uses non pooling watcher. But there is another issue - we cannot set environment variables using built int tsc --watch build action in VSCode.

[mhegazy]

@OlekNg, what OS are you using and what is the version of node?

I found that you can set TSC_NONPOLLING_WATCHER env var to 1, and then tsc -w uses non pooling watcher.

So you do not see the issue if you set TSC_NONPOLLING_WATCHER to 1?

[aszczepanek]

@mhegazy
I'm on Win 10 Pro (1709) at home (1709) and Win 10 Premium at my company (also 1709 update).
Both running latest 6.x node version.

Setting TSC_NONPOLLING_WATCHER to 1 solved high CPU usage problem (it went down to 0%).
I'm not sure if this is recommended solution because I found it in tsc source - I couldn't find it in any official documentation.

From my previous post I also found that I can set env var for whole VSCode using setting "terminal.integrated.env.windows" which allowed me to run tsc compiler from VSCode with TSC_NONPOLLING_WATCHER flag set to 1.

[mhegazy]

How many files do you have in your project (tsc --listfiles --p tsconfig.josn). also what version of TS are you using?

[aszczepanek]

tsc --listfiles --p tsconfig.json reports 1854 files.

[eyalellenbogen]

TSC_NONPOLLING_WATCHER=1 solves all my problems!
@OlekNg thank you so much for this tip! CPU went from 15-40% to 0.5%. I can definitely live with that.

[SimonSchick]

Same problem, --list-files reports 596 files.
After some time the process takes roughly ~10% of one core all the time.

[virus2016]

I'm still experiencing high cpu usage when idle (using docker). TSC_NONPOLLING_WATCHER=1 stops watching the ts files for me.

[sheetalkamat]

PR #21243 has multiple options enabled using environment variables TSC_WATCHFILE and TSC_WATCHDIRECTORY. The drop is built and shared at localBuilt.zip

The different options are explained in #21243 (comment) . Experience and feedback about these options is appreciated and would help us make decision on which strategy to go as default.