Segfault when running napajs on CentOS 7

[mnahkies]

Impact:
Napajs is unusable on CentOS 7, and pressumbly some other RHEL distributions. I have tried both the pre-build binary automatically installed, and by forcing a rebuild using

npm install --save napajs --fetch=false --build=true

Reproduction:
On a Linux system with a working docker installation create the following two files side by side. Then execute reproduce.sh.

This will build a docker environment based on centos 7 that has the required tools (cmake, nodejs, g++, etc), clone the napajs repository, and build it.

The docker run command then attempts to run the napajs test suite against the newly build image. This fails with a segfault. This happens when I try to include napajs in my project as well.

node-v8-centos7-napajs.dockerfile

FROM centos:7

RUN yum -y install mercurial git which gcc-c++ make ruby ruby-devel zip unzip bzip2 fontconfig \
        && yum upgrade \
        && yum clean all

RUN curl -L -o node-v8.11.1-linux-x64.tar.gz https://nodejs.org/dist/v8.11.1/node-v8.11.1-linux-x64.tar.gz

RUN tar --strip-components 1 -xzvf node-v8.11.1-linux-x64.tar.gz -C /usr node-v8.11.1-linux-x64/ && rm node-v8.11.1-linux-x64.tar.gz

RUN node -v && npm -v

RUN npm install -g cmake-js

RUN git clone https://github.com/Microsoft/napajs.git /working

RUN curl -L -o cmake-3.11.1-Linux-x86_64.tar.gz https://cmake.org/files/v3.11/cmake-3.11.1-Linux-x86_64.tar.gz

RUN tar --strip-components 1 -xzvf cmake-3.11.1-Linux-x86_64.tar.gz -C /usr cmake-3.11.1-Linux-x86_64/ && rm cmake-3.11.1-Linux-x86_64.tar.gz

RUN yum -y install centos-release-scl
RUN yum -y install devtoolset-7-gcc*

WORKDIR /working

RUN scl enable devtoolset-7 "bash -c 'npm install && npm run prepare && npm run rebuild'"

reproduce.sh

#!/usr/bin/env bash
docker build --no-cache -t nappajs/centos-issue:latest -f node-v8-centos7-napajs.dockerfile .
docker run -it nappajs/centos-issue /bin/bash -c 'npm test'

Output:

> napajs@0.2.2 test /working
> mocha test -g "^((?!napajs/timers).)*$" --recursive && mocha test -g "^napajs/timers"

WARN napajs binding Thanks for using Napa.js.
WARN napajs binding There is a compatibility issue on Node v8.5.0 and above.
WARN napajs binding The flag "--noincremental-marking" is set to disable V8 incremental marking as a workaround.
WARN napajs binding We are working with Node.js team on a fix in newer Node versions.
*** Error in `/usr/bin/node': munmap_chunk(): invalid pointer: 0x00007f46c8057180 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7ab54)[0x7f46e922bb54]
/usr/bin/node(_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE21_M_insert_unique_nodeEmmPNS1_10_Hash_nodeISsLb1EEE+0x12c)[0x1249cfc]
/working/bin/libnapa.so(_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE10_M_emplaceIJRPKcEEESt4pairINS1_14_Node_iteratorISsLb1ELb1EEEbESt17integral_constantIbLb1EEDpOT_+0x111)[0x7f46e6cd0791]
/working/bin/libnapa.so(+0xa6afc)[0x7f46e6cd3afc]
/working/bin/libnapa.so(+0xa2b15)[0x7f46e6ccfb15]
/working/bin/libnapa.so(+0xa2d01)[0x7f46e6ccfd01]
/working/bin/libnapa.so(+0xdaeb8)[0x7f46e6d07eb8]
/working/bin/libnapa.so(+0xdef0f)[0x7f46e6d0bf0f]
/lib64/libpthread.so.0(+0x7e25)[0x7f46e957be25]
/lib64/libc.so.6(clone+0x6d)[0x7f46e92a934d]
======= Memory map: ========
00400000-01f56000 r-xp 00000000 00:26 53                                 /usr/bin/node
02156000-02170000 rw-p 01b56000 00:26 53                                 /usr/bin/node
02170000-02188000 rw-p 00000000 00:00 0 
037df000-03a1e000 rw-p 00000000 00:00 0                                  [heap]
2146e89a000-2146e89c000 ---p 00000000 00:00 0 
2146e89c000-2146e8a4000 rw-p 00000000 00:00 0 
2146e8a4000-2146e8a6000 ---p 00000000 00:00 0 
2d03832d000-2d038330000 ---p 00000000 00:00 0 
2d038330000-2d038338000 rw-p 00000000 00:00 0 
2d038338000-2d038339000 ---p 00000000 00:00 0 
65edd800000-65edd84f000 rw-p 00000000 00:00 0 
72507300000-72507380000 rw-p 00000000 00:00 0 
7eff5600000-7eff5680000 rw-p 00000000 00:00 0 
90058080000-90058100000 rw-p 00000000 00:00 0 
c1828c80000-c1828d00000 rw-p 00000000 00:00 0 
db827500000-db827580000 rw-p 00000000 00:00 0 
dde2e800000-dde2e880000 rw-p 00000000 00:00 0 
e15b5800000-e15b5805000 rw-p 00000000 00:00 0 
fc59a303000-fc59a304000 ---p 00000000 00:00 0 
fc59a304000-fc59a30c000 *** Error in `rw-p 00000000 00:00 0 
fc59a30c000-fc59a30f000 ---p 00000000 00:00 0 
fece7680000-fece7700000 rw-p 00000000 00:00 0 
129f10200000-129f1024f000 rw-p 00000000 00:00 0 
13c9e9600000-13c9e9680000 rw-p 00000000 00:00 0 
144b0bc80000-144b0bd00000 rw-p 00000000 00:00 0 
17ca9c500000-17ca9c580000 rw-p 00000000 00:00 0 
1899bc780000-1899bc800000 rw-p 00000000 00:00 0 
19c996e80000-19c996f00000 rw-p 00000000 00:00 0 
1a070ca80000-1a070cb00000 rw-p 00000000 00:00 0 
1a1cf7e80000-1a1cf7f00000 rw-p 00000000 00:00 0 
1be39e580000-1be39e600000 rw-p 00000000 00:00 0 
1bfe60c00000-1bfe60c80000 rw-p 00000000 00:00 0 
1db9ebb15000-1db9ebb80000 ---p 00000000 00:00 0 
1db9ebb80000-1db9ebb83000 rw-p 00000000 00:00 0 
1db9ebb83000-1db9ebb84000 ---p 00000000 00:00 0 
1db9ebb84000-1db9ebb85000 rwxp 00000000 00:00 0 
1db9ebb85000-1db9ebc00000 ---p 00000000 00:00 0 
1db9ebc00000-1db9ebc03000 rw-p 00000000 00:00 0 
1db9ebc03000-1db9ebc04000 ---p 00000000 00:00 0 
1db9ebc04000-1db9ebc05000 rwxp 00000000 00:00 0 
1db9ebc05000-1db9ebc80000 ---p 00000000 00:00 0 
1db9ebc80000-1db9ebc83000 rw-p 00000000 00:00 0 
1db9ebc83000-1db9ebc84000 ---p 00000000 00:00 0 
1db9ebc84000-1db9ebc85000 rwxp 00000000 00:00 0 
1db9ebc85000-1db9ebd00000 ---p 00000000 00:00 0 
1db9ebd00000-1db9ebd03000 rw-p 00000000 00:00 0 
1db9ebd03000-1db9ebd04000 ---p 00000000 00:00 0 
1db9ebd04000-1db9ebd7f000 rwxp 00000000 00:00 0 
1db9ebd7f000-1db9ebd80000 ---p 00000000 00:00 0 
1db9ebd80000-1db9ebd83000 rw-p 00000000 00:00 0 
1db9ebd83000-1db9ebd84000 ---p 00000000 00:00 0 
1db9ebd84000-1db9ebdff000 rwxp 00000000 00:00 0 
1db9ebdff000-1db9ebe00000 ---p 00000000 00:00 0 
1db9ebe00000-1db9ebe03000 rw-p 00000000 00:00 0 
1db9ebe03000-1db9ebe04000 ---p 00000000 00:00 0 
1db9ebe04000-1db9ebe7f000 rwxp 00000000 00:00 0 
1db9ebe7f000-1dba0bb15000 ---p 00000000 00:00 0 
216ba5500000-216ba5580000 rw-p 00000000 00:00 0 
22375f600000-22375f680000 rw-p 00000000 00:00 0 
225b12e00000-225b12e80000 rw-p 00000000 00:00 0 
229d38700000-229d38705000 rw-p 00000000 00:00 0 
232beb700000-232beb780000 rw-p 00000000 00:00 0 
239292c00000-239292c80000 rw-p 00000000 00:00 0 
2483e8680000-2483e8700000 rw-p 00000000 00:00 0 
27b5bde80000-27b5bdecf000 rw-p 00000000 00:00 0 
27ec64700000-27ec64780000 rw-p 00000000 00:00 0 
284ad7400000-284ad7480000 rw-p 00000000 00:00 0 
29d251f00000-29d251f80000 rw-p 00000000 00:00 0 
29ef5eb80000-29ef5ec00000 rw-p 00000000 00:00 0 
2bd7a4080000-2bd7a4100000 rw-p 00000000 00:00 0 
2dc2f8280000-2dc2f8300000 rw-p 00000000 00:00 0 
2dd405673000-2dd405680000 ---p 00000000 00:00 0 
2dd405680000-2dd405683000 rw-p 00000000 00:00 0 
2dd405683000-2dd405684000 ---p 00000000 00:00 0 
2dd405684000-2dd4056ad000 rwxp 00000000 00:00 0 
2dd4056ad000-2dd405700000 ---p 00000000 00:00 0 
2dd405700000-2dd405703000 rw-p 00000000 00:00 0 
2dd405703000-2dd405704000 ---p 00000000 00:00 0 
2dd405704000-2dd40572d000 rwxp 00000000 00:00 0 
2dd40572d000-2dd405780000 ---p 00000000 00:00 0 
2dd405780000-2dd405783000 rw-p 00000000 00:00 0 
2dd405783000-2dd405784000 ---p 00000000 00:00 0 
2dd405784000-2dd4057ad000 rwxp 00000000 00:00 0 
2dd4057ad000-2dd405800000 ---p 00000000 00:00 0 
2dd405800000-2dd405803000 rw-p 00000000 00:00 0 
2dd405803000-2dd405804000 ---p 00000000 00:00 0 
2dd405804000-2dd40587f000 rwxp 00000000 00:00 0 
2dd40587f000-2dd405880000 ---p 00000000 00:00 0 
2dd405880000-2dd405883000 rw-p 00000000 00:00 0 
2dd405883000-2dd405884000 ---p 00000000 00:00 0 
2dd405884000-2dd4058ff000 rwxp 00000000 00:00 0 
2dd4058ff000-2dd405900000 ---p 00000000 00:00 0 
2dd405900000-2dd405903000 rw-p 00000000 00:00 0 
2dd405903000-2dd405904000 ---p 00000000 00:00 0 
2dd405904000-2dd40597f000 rwxp 00000000 00:00 0 
2dd40597f000-2dd405980000 ---p 00000000 00:00 0 
2dd405980000-2dd405983000 rw-p 00000000 00:00 0 
2dd405983000-2dd405984000 ---p 00000000 00:00 0 
2dd405984000-2dd4059ff000 rwxp 00000000 00:00 0 
2dd4059ff000-2dd425673000 ---p 00000000 00:00 0 
2ef27d300000-2ef27d305000 rw-p 00000000 00:00 0 
2f8a5b980000-2f8a5ba00000 rw-p 00000000 00:00 0 
2fb35f300000-2fb35f380000 rw-p 00000000 00:00 0 
326b8ee000/usr/bin/node': 00-326b8ee80000 rw-p 00000000 00:00 0 
329369a00000-329369a80000 rw-p 00000000 00:00 0 
350b90f80000-350b91000000 rw-p 00000000 00:00 0 
3a40f3c9b000-3a40f3d00000 ---p 00000000 00:00 0 
3a40f3d00000-3a40f3d03000 rw-p 00000000 00:00 0 
3a40f3d03000-3a40f3d04000 ---p 00000000 00:00 0 
3a40f3d04000-3a40f3d05000 rwxp 00000000 00:00 0 
3a40f3d05000-3a40f3d80000 ---p 00000000 00:00 0 
3a40f3d80000-3a40f3d83000 rw-p 00000000 00:00 0 
3a40f3d83000-3a40f3d84000 ---p 00000000 00:00 0 
3a40f3d84000-3a40f3d85000 rwxp 00000000 00:00 0 
3a40f3d85000-3a40f3e00000 ---p 00000000 00:00 0 
3a40f3e00000-3a40f3e03000 rw-p 00000000 00:00 0 
3a40f3e03000-3a40f3e04000 ---p 00000000 00:00 0 
3a40f3e04000-3a40f3e05000 rwxp 00000000 00:00 0 
3a40f3e05000-3a40f3e80000 ---p 00000000 00:00 0 
3a40f3e80000-3a40f3e83000 rw-p 00000000 00:00 0 
3a40f3e83000-3a40f3e84000 ---p 00000000 00:00 0 
3a40f3e84000-3a40f3eff000 rwxp 00000000 00:00 0 
3a40f3eff000-3a40f3f00000 ---p 00000000 00:00 0 
3a40f3f00000-3a40f3f03000 rw-p 00000000 00:00 0 
3a40fsh: line 1:    17 Aborted                 (core dumped) mocha test -g "^((?!napajs/timers).)*$" --recursive
npm ERR! Test failed.  See above for more details.

[fs-eire]

The crash stacktrace looks like the known issue here - nodejs/node#17817

Please have a try if the crash happens on Node 8.10.0/8.9.4. If the answer is Yes on 8.10.0 and No on 8.9.4. That should be the same issue.

[mnahkies]

@fs-eire it worked (didn't crash) on 8.9.4, haven't had a chance to test 8.10.0 yet

[fs-eire]

I did some investigation. Thanks for the stack trace the root cause is identified:

( From the dump above)

======= Backtrace: =========
/lib64/libc.so.6(+0x7ab54)[0x7f46e922bb54]
/usr/bin/node(_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE21_M_insert_unique_nodeEmmPNS1_10_Hash_nodeISsLb1EEE+0x12c)[0x1249cfc]
/working/bin/libnapa.so(_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE10_M_emplaceIJRPKcEEESt4pairINS1_14_Node_iteratorISsLb1ELb1EEEbESt17integral_constantIbLb1EEDpOT_+0x111)[0x7f46e6cd0791]
...

Both node and libnapa.so expose the symbol _ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE21_M_insert_unique_nodeEmmPNS1_10_Hash_nodeISsLb1EEE, and the wrong one is selected in runtime, thus causes the crash.

See nodejs/node#17817 to check more information.

[fs-eire]

Should have been fixed in 0.2.3

@mnahkies could you please have a try on the latest version?

[mnahkies]

This appears fixed now, thanks @fs-eire!