high cpu utilization of webpack watch when installs the deps through yarn

[bsr203]

ver: v0.15.1
OS: OSX 10.9.5
node: v6.8.0

I raised this at SO, and could narrow down to yarn usage. My node_modules are excluded from webpack for js build.

---

My node app (running through webpack dev server) is taking up 60% cpu consistently. From the cpu profile, looks like call to _proc_listpidspath is the culprit. Any hints on what is this function call for, and how to find which node library calls this.

thanks.

See the profile below.

Code move event for unknown code: 0x19cac90226c0
Code move event for unknown code: 0x19cac90b3360
Code move event for unknown code: 0x19caca55d3a0
Statistical profiling result from isolate-0x102004c00-v8.log, (209273 ticks, 2592 unaccounted, 0 excluded).

 [Shared libraries]:
   ticks  total  nonlib   name
   6896    3.3%          /usr/lib/system/libsystem_c.dylib
    285    0.1%          /usr/lib/system/libsystem_pthread.dylib

 [JavaScript]:
   ticks  total  nonlib   name
    332    0.2%    0.2%  KeyedLoadIC: A keyed load IC from the snapshot {1}
    287    0.1%    0.1%  Stub: LoadICStub
    ....
    ....
  [Summary]:
   ticks  total  nonlib   name
   5271    2.5%    2.6%  JavaScript
  194229   92.8%   96.1%  C++
   1481    0.7%    0.7%  GC
   7181    3.4%          Shared libraries
   2592    1.2%          Unaccounted
[C++]:
  ticks  total  nonlib   name
 162560   77.7%   80.4%  _proc_listpidspath
  4417    2.1%    2.2%  _uv_timer_stop
  1683    0.8%    0.8%  _heap_node_swap
  1274    0.6%    0.6%  _timer_cb
  1006    0.5%    0.5%  _hex
   762    0.4%    0.4%  _uv_fs_stat
   704    0.3%    0.3%  _mach_vm_map
   677    0.3%    0.3%  _szone_check_all
   654    0.3%    0.3%  _pthread_join$NOCANCEL
   637    0.3%    0.3%  __pthread_exit
   583    0.3%    0.3%  _uv__run_timers
   583    0.3%    0.3%  _dec
   ......
   ......

[Bottom up (heavy) profile]:
 Note: percentage shows a share of a particular caller in the total
 amount of its parent calls.
 Callers occupying less than 2.0% are not shown.

  ticks parent  name
 162560   77.7%  _proc_listpidspath

  6896    3.3%  /usr/lib/system/libsystem_c.dylib

  4417    2.1%  _uv_timer_stop

Edit:

I could isolate this to when it happens. I started using yarn yesterday. Now, if I remove my node_modules and run npm -i to install all the deps through npm client, I see the file count inside node_modules as

node_modules ❯❯❯ find . -print | wc -l                                                                                                                                   master
   43058

If I remove, node_modules and ran ~/.yarn/bin/yarn.js, then

node_modules ❯❯❯ find . -print | wc -l                                                                                                                                 ⏎ master
   32138

so, there is a difference in files installed by both cli, and yarn installs less. But, the cpu with yarn is 80% in idle (webpack server in watch mode), and 1% with npm.!!

[darcyadams]

seeing a very similar issue on MacOS 10.10 with Node v4.4.

I'm using compiler.watch directly through gulp.

I also see the CPU drop down <10% after reverting to a clean npm install

I do watch node_modules, but excluding that folder seemed to have no impact in CPU usage.

[TheLarkInn]

If this does turn out to be a webpack related problem would you mind leaving the webpack version this was reported in, and os so I can get that information copied over to a new issue in our repo once we find out more information?

[bsr203]

@TheLarkInn webpack related ver

    "sass-loader": "^4.0.2",
    "style-loader": "^0.13.1",
    "url-loader": "^0.5.7",
    "webpack": "^2.1.0-beta.25",
    "webpack-dev-middleware": "^1.8.4",
    "webpack-hot-middleware": "^2.13.0",
    "webpack-shell-plugin": "^0.4.3"

[TheLarkInn]

@SpaceK33z do you think you could check the versions above ^^ and see if there is anything funky that could go on in regards to deps?

[SpaceK33z]

Yeah this is because installing the native watch dependency fails, which causes chokidar to use inefficient polling. This is an issue in yarn. Also see #629 (comment)

[SpaceK33z]

So this can be closed as a dupe of #629, the issue described here is really a side effect of #629.

[TheLarkInn]

Thanks a bunch! Hope this answers your questions @bsr203 @darcyadams

[bsr203]

ok.. thanks. will follow #629