Have an e2e test against a node version that uses v8 very close to head

[aalexand]

As CPU or heap profiler in v8 get any changes, we should continuously make sure those do not carry surprises for us (e.g. there is https://chromium-review.googlesource.com/c/v8/v8/+/1039825, which shouldn't break us but only green test gives confidence). To do that we need an automated e2e test run against a version of node that uses v8 very close to head. @ofrobots any recommendations on how to organize this? My naive hope would be that we could do something like "nvm install head" and the head gets v8 updated regularly (automatically?).

@nolanmar511 FYI

[ofrobots]

This is a good idea. Having feedback from potentially breaking changes in V8 as early as possible would be great. I am not sure whether the tests we have in the Node project are as extensive for the profiling/diagnostics use-cases, and this project might well be one of the most advanced users of these V8 APIs.

In the node project we run v8-canary builds which are supposed to be Node.js master integrated with a recent (within O(days) of V8 Tip-of-Tree) version of V8. Available here. We can download and unpack the tarball and go from there. I think this would be sufficient for our purposes.

If we want a build that is within O(hours) up to date, we can use the tarballs from the V8 integration build: https://uberchromegw.corp.google.com/i/client.v8.fyi/builders/V8%20Linux64%20-%20node.js%20integration. This build integrates each V8-roll (small number of commits) into a semi-recent Node.js snapshot. The built binaries are available for download as a tarball from that site too. This will provide semi-immediate feedback if a V8 commit ends up breaking us.

[aalexand]

O(days) is good but O(hours) is better so I'd go for that.

[nolanmar511]

Installing the v8-canary builds can be done in one line:

NVM_NODEJS_ORG_MIRROR=https://nodejs.org/download/v8-canary/ nvm i node

Installing the v8 integration build would be harder. There's some javascript code which does this, but it first retrieves retrieves a list of builds, then finds the latest successful build, then gets the build git revision and commit and then downloads the build. I don't think we'll need to get the revision and commit (those seem to be used to in reporting).
I think we'd need maybe ~100 lines of additional code to use the integration build, which isn't huge. Would we want to duplicate the javascript code and call it from the start-up script? Translate it to go?

[aalexand]

@nolanmar511 Enabling v8-canary build is probably a good first step. Empirically, how fresh v8 is in these builds?

[nolanmar511]

When I run node-gyp rebuild with the v8-canary version of node, I get an error:

gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@11.0.0-v8-canary20180622e30b7c112d | linux | x64
gyp http GET https://nodejs.org/download/v8-canary/v11.0.0-v8-canary20180622e30b7c112d/node-v11.0.0-v8-canary20180622e30b7c112d-headers.tar.gz
gyp http 404 https://nodejs.org/download/v8-canary/v11.0.0-v8-canary20180622e30b7c112d/node-v11.0.0-v8-canary20180622e30b7c112d-headers.tar.gz
gyp WARN install got an error, rolling back install
gyp ERR! configure error 
gyp ERR! stack Error: 404 response downloading https://nodejs.org/download/v8-canary/v11.0.0-v8-canary20180622e30b7c112d/node-v11.0.0-v8-canary20180622e30b7c112d-headers.tar.gz
gyp ERR! stack     at Request.<anonymous> (/usr/lib/node_modules/node-gyp/lib/install.js:210:14)
gyp ERR! stack     at Request.emit (events.js:187:15)
gyp ERR! stack     at Request.onRequestResponse (/usr/lib/node_modules/node-gyp/node_modules/request/request.js:1074:10)
gyp ERR! stack     at ClientRequest.emit (events.js:182:13)
gyp ERR! stack     at HTTPParser.parserOnIncomingClient (_http_client.js:554:21)
gyp ERR! stack     at HTTPParser.parserOnHeadersComplete (_http_common.js:109:17)
gyp ERR! stack     at TLSSocket.socketOnData (_http_client.js:440:20)
gyp ERR! stack     at TLSSocket.emit (events.js:182:13)
gyp ERR! stack     at addChunk (_stream_readable.js:283:12)
gyp ERR! stack     at readableAddChunk (_stream_readable.js:264:11)
gyp ERR! System Linux 4.9.0-6-amd64
gyp ERR! command "/usr/local/google/home/nolanmar/.nvm/versions/node/v11.0.0-v8-canary20180622e30b7c112d/bin/node" "/usr/bin/node-gyp" "rebuild"
gyp ERR! cwd /usr/local/google/home/nolanmar/cloud-profiler-nodejs-working/cloud-profiler-nodejs
gyp ERR! node -v v11.0.0-v8-canary20180622e30b7c112d
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 

Any recommendations?
Would I have this problem using the lkgr build?

[aalexand]

@nolanmar511 Did you mean to tag the person your question was addressed to?

[nolanmar511]

I asked ofrobots in meeting.
It will be necessary to clone https://github.com/nodejs/node-v8, and use the --nodedir flag to point to that directory.

Since we'll need to identify what the commit being used is, it's probably necessary to use v8-lkgr.

[aalexand]

@ofrobots @nolanmar511 Based on discussion today looks like we'll use the LKGR build. Still need to figure out how get the matching V8 headers.

[aalexand]

I would like to re-open this as I think using the Node.js canary build may be simple enough.

While verifying recent CPU profiler memory usage improvements, I used this command to install the canary build:

NVM_NODEJS_ORG_MIRROR=https://nodejs.org/download/v8-canary nvm install node

With that I indeed had issues installing the profiler module:

$ npm install --save @google-cloud/profiler

> @google-cloud/profiler@0.2.0 install /tmp/testprof/node_modules/@google-cloud/profiler
> node-gyp rebuild

gyp WARN install got an error, rolling back install
gyp ERR! configure error 
gyp ERR! stack Error: 404 response downloading https://nodejs.org/download/v8-canary/v11.0.0-v8-canary201807140f69779e03/node-v11.0.0-v8-canary201807140f69779e03-headers.tar.gz
gyp ERR! stack     at Request.<anonymous> (/usr/local/google/home/aalexand/.nvm/versions/node/v11.0.0-v8-canary201807140f69779e03/lib/node_modules/npm/node_modules/node-gyp/lib/install.js:210:14)
gyp ERR! stack     at Request.emit (events.js:187:15)
gyp ERR! stack     at Request.onRequestResponse (/usr/local/google/home/aalexand/.nvm/versions/node/v11.0.0-v8-canary201807140f69779e03/lib/node_modules/npm/node_modules/request/request.js:1062:10)
gyp ERR! stack     at ClientRequest.emit (events.js:182:13)
gyp ERR! stack     at HTTPParser.parserOnIncomingClient (_http_client.js:554:21)
gyp ERR! stack     at HTTPParser.parserOnHeadersComplete (_http_common.js:109:17)
gyp ERR! stack     at TLSSocket.socketOnData (_http_client.js:440:20)
gyp ERR! stack     at TLSSocket.emit (events.js:182:13)
gyp ERR! stack     at addChunk (_stream_readable.js:283:12)
gyp ERR! stack     at readableAddChunk (_stream_readable.js:264:11)
gyp ERR! System Linux 4.9.0-6-amd64
gyp ERR! command "/usr/local/google/home/aalexand/.nvm/versions/node/v11.0.0-v8-canary201807140f69779e03/bin/node" "/usr/local/google/home/aalexand/.nvm/versions/node/v11.0.0-v8-canary201807140f69779e03/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /tmp/testprof/node_modules/@google-cloud/profiler
gyp ERR! node -v v11.0.0-v8-canary201807140f69779e03
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 
npm WARN enoent ENOENT: no such file or directory, open '/tmp/testprof/package.json'
npm WARN testprof No description
npm WARN testprof No repository field.
npm WARN testprof No README data
npm WARN testprof No license field.
npm WARN You are using a pre-release version of node and things may not work as expected

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @google-cloud/profiler@0.2.0 install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @google-cloud/profiler@0.2.0 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /usr/local/google/home/aalexand/.npm/_logs/2018-07-20T17_55_23_203Z-debug.log

But using this trick makes things work:

$ npm install --save --nodedir=$(dirname $(dirname $(which node))) @google-cloud/profiler

> @google-cloud/profiler@0.2.0 install /tmp/testprof/node_modules/@google-cloud/profiler
> node-gyp rebuild

make: Entering directory '/tmp/testprof/node_modules/@google-cloud/profiler/build'
  CXX(target) Release/obj.target/sampling_heap_profiler/bindings/sampling-heap-profiler.o
  SOLINK_MODULE(target) Release/obj.target/sampling_heap_profiler.node
  COPY Release/sampling_heap_profiler.node
  CXX(target) Release/obj.target/time_profiler/bindings/time-profiler.o
../bindings/time-profiler.cc: In function ‘Nan::NAN_METHOD_RETURN_TYPE SetIdle(Nan::NAN_METHOD_ARGS_TYPE)’:
../bindings/time-profiler.cc:92:31: warning: ‘void v8::CpuProfiler::SetIdle(bool)’ is deprecated: Use Isolate::SetIdle(bool) instead. [-Wdeprecated-declarations]
   cpuProfiler->SetIdle(is_idle);
                               ^
In file included from /usr/local/google/home/aalexand/.nvm/versions/node/v11.0.0-v8-canary201807140f69779e03/include/node/v8.h:26:0,
                 from /usr/local/google/home/aalexand/.nvm/versions/node/v11.0.0-v8-canary201807140f69779e03/include/node/v8-profiler.h:10,
                 from ../bindings/time-profiler.cc:17:
/usr/local/google/home/aalexand/.nvm/versions/node/v11.0.0-v8-canary201807140f69779e03/include/node/v8-profiler.h:360:22: note: declared here
                 void SetIdle(bool is_idle));
                      ^
/usr/local/google/home/aalexand/.nvm/versions/node/v11.0.0-v8-canary201807140f69779e03/include/node/v8config.h:324:3: note: in definition of macro ‘V8_DEPRECATED’
   declarator __attribute__((deprecated(message)))
   ^~~~~~~~~~
  SOLINK_MODULE(target) Release/obj.target/time_profiler.node
  COPY Release/time_profiler.node
make: Leaving directory '/tmp/testprof/node_modules/@google-cloud/profiler/build'

> protobufjs@6.8.8 postinstall /tmp/testprof/node_modules/protobufjs
> node scripts/postinstall

npm WARN saveError ENOENT: no such file or directory, open '/tmp/testprof/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/tmp/testprof/package.json'
npm WARN testprof No description
npm WARN testprof No repository field.
npm WARN testprof No README data
npm WARN testprof No license field.
npm WARN You are using a pre-release version of node and things may not work as expected

+ @google-cloud/profiler@0.2.0
added 117 packages from 155 contributors and audited 269 packages in 3.317s
found 0 vulnerabilities

And the headers in that location seems recent enough, at least they do include the recent line mode changes:

$ cat $(dirname $(dirname $(which node)))/include/node/v8-profiler.h | grep -C5 kCaller
  // (from the root to a leaf) will have line numbers that point to the start
  // line of the function, rather than the line of the callsite of the child.
  kLeafNodeLineNumbers,
  // In the resulting CpuProfile tree, nodes are separated based on the line
  // number of their callsite in their parent.
  kCallerLineNumbers,
};

/**
 * Interface for controlling CPU profiling. Instance of the
 * profiler can be created using v8::CpuProfiler::New method.

With that, adding testing against the canary build should be trivial?

[nolanmar511]

I tried using --nodedir where the profiler is installed (so retry npm install --nodedir=/.nvm/versions/node/v11.0.0-v8-canary201807247c08774a29 /cloud-profiler-nodejs/google-cloud-profiler-0.2.0.tgz) and got a segmentation fault immediately after trying to collect a Wall profile.

Here's the complete output of the test run:
out.txt

[aalexand]

@nolanmar511 Does it reproduce locally? Try to reproduce it locally, generate a core file and get the stack of the crash from the core file using gbd, something along these lines. That should give some clues about what is going on.