ORA-24550 SIGSEGV when reinstantiating oracledb and reconnecting

[rodrigosalinas]

1. The versions I have used

5.0.1 and 5.1.0

process.platform: linux
process.version: v11.15.0
process.arch: x64
require('oracledb').versionString: 5.1.0
require('oracledb').oracleClientVersionString: 19.3.0.0.0

2. The problem

I have an api with koa.js. From the api I launch a Worker (node worker_thread) each time an endpoint is reached. That worker is a script that connects to an oracle database using oracledb. With the same connection it executes some queries. At the end the script closes the connection and sends a message back to the api so the api terminates the worker. Everything works just fine up to this point. The problem occurs when the next request arrives: the api instantiates a new Worker, the worker opens a new connection, and before any command is executed, the whole api program crashes and reboots (I use pm2 to run my api) with the following error that cannot be handled:

ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_int=15822326] [si_ptr=0xf16df6] [si_addr=0x3ab96b0]
kpedbg_dmp_stack()+394
<-kpeDbgCrash()+204
<-kpeDbgSignalHandler()+113
<-skgesig_sigactionHandler()+258<-__sighandler()<-dpiVar__initBuffer()+1415<-dpiVar__allocate()+408<-dpiConn_newVar()+160<-njsVariable_createBuffer()+180<-njsConnection_processExecuteBinds()+341<-njsConnection_execute()+310
<-_ZN6v8impl12_GLOBAL__N_123FunctionCallbackWrapper6InvokeERKN2v820FunctionCallbackInfoINS2_5ValueEEE()+133
<-_ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb0EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS()
<-_ZN2v88internal21Builtin_HandleApiCallEiPPNS0_6ObjectEPNS0_7IsolateE()+185
<-0x0000269912DCFC5D<-0x0000269912D8E458<-0x0000269912D8E458<-0x0000269912D876A6<-0x0000269912D8E458<-0x0000269912E778DB<-0x0000269912D89E72<-0x0000269912E07A20<-0x0000269912D8E458<-0x0000269912D8E458<-0x0000269912E778DB

The si_signo=11 means SIGSEGV, which means Invalid memory reference. Once the api reboots, the next request works just fine.

When I use segfault-handler I get the following detail:

node /root/apps/bi-etl/api.js(_ZN2v88internal21Builtin_HandleApiCallEiPPNS0_6ObjectEPNS0_7IsolateE+0xb9)[0xbc78d9]
[0x72baa24fc5d]
PID 25342 received SIGSEGV for address: 0x78970
/root/apps/bi-etl/node_modules/segfault-handler/build/Release/segfault-handler.node(+0x2cf1)[0x7f68e40dacf1]
/lib64/libpthread.so.0(+0xf630)[0x7f68e7ec1630]
/root/apps/bi-etl/node_modules/oracledb/build/Release/oracledb-5.1.0-linux-x64.node(+0x5d207)[0x7f68ceddd207]
 /root/apps/bi-etl/node_modules/oracledb/build/Release/oracledb-5.1.0-linux-x64.node(dpiVar__allocate+0x198)[0x7f68ceddd7f8]
/root/apps/bi-etl/node_modules/oracledb/build/Release/oracledb-5.1.0-linux-x64.node(dpiConn_newVar+0xa0)[0x7f68cedb4af0]
/root/apps/bi-etl/node_modules/oracledb/build/Release/oracledb-5.1.0-linux-x64.node(njsVariable_createBuffer+0xb8)[0x7f68cedae928]
/root/apps/bi-etl/node_modules/oracledb/build/Release/oracledb-5.1.0-linux-x64.node(+0x1b948)[0x7f68ced9b948]
/root/apps/bi-etl/node_modules/oracledb/build/Release/oracledb-5.1.0-linux-x64.node(+0x1c6ed)[0x7f68ced9c6ed]
node /root/apps/bi-etl/api.js[0x916965]
node /root/apps/bi-etl/api.js[0xbc6d2a]
node /root/apps/bi-etl/api.js(_ZN2v88internal21Builtin_HandleApiCallEiPPNS0_6ObjectEPNS0_7IsolateE+0xb9)[0xbc78d9]
[0x2cd8bcbcfc5d]

Please help. Why is this happening? Both the connection and the worker I close and terminate correctly before I try to reinstantiate the worker and reconnect to the database.

This is a simplified version of my function in my api where I instantiate the worker:

function run(scriptPath) {
    return new Promise((resolve, reject) => {
        try {
            let worker = new Worker(scriptPath, { workerData: { params: someParams, connectionData: connData } })
            worker.on('message', async(message) => {
                await worker.terminate()
            })
            worker.on('exit', exitCode => {
                resolve()
            })
        } catch (err) {
            reject(err)
        }
    })
}

And this is a simplified version of my Worker:

const { parentPort, workerData } = require('worker_threads')
const oracledb = require('oracledb')

(async () => {
    const conn = await oracledb.getConnection(workerData.connectionData)
    await conn.execute(cmd, workerData.params)
    await conn.execute(cmd, workerData.params)
    await conn.close()
    parentPort.postMessage('Finished')
})()

Thank you very very much in advance.

[cjbj]

• Can you give us a reproducible testcase?
• Have you reviewed this and this?
• Can you update Node.js? And it's always worth updating the Oracle client version.

[cjbj]

@rodrigosalinas any updates on this?

[rodrigosalinas2]

Hi @cjbj, I'm the same guy that created this issue. I had an auth problem with my older account so I had to create a new one. Regarding the issue, I did what you told me: I updated Node and Oracle client. That solved my problem apparently because it hasn't happened again.
But when I updated your library to 5.1 some bizarre issue started to happen randomly with my queries: for string fields it started to retrieve with null characters attached, and in some cases even characters that should not be there, as if the database retrieved strings attached from other places. Like for example, when I run a simple SELECT over the USER_TAB_COLUMNS table, it retrieved me sometimes things like the following:

{
    COLUMN_NAME: 'SALDO_FINAL',
    DATA_TYPE: 'NUMBER�\x15\x00\x00\x02\x00\\\x00\rIW8EBCDIC424\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00�\x15\x00\x00\x02\x00]\x00\x0ETR8EBCDIC1026\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00�\x13\x00\x00\x02\x00^\x00\rWE8EBCDNUMB',
    NULLABLE: 'Y'
}

Sometimes it even cropped the column name. I had to come back to 4.2 version so that wouldn't happen. Any thoughts on that? Thank you very very much in advance.

[cjbj]

I haven't seen any reports of this. Is the data like that immediately after being fetched, or is getting munged when it gets to the higher API?

Have you got a simple test case (non koa; using async/await not promises) that can reproduce it - even if not consistently?

[rodrigosalinas2]

The data is like that immediately after being fetched. It happens one of three times always. With 5.1 version, not with 4.2.

[rodrigosalinas2]

I'm using Node 15.12.0 and Oracle Instantclient 19.8 over Centos 7.

[cjbj]

I'd to understand what's going on but have nothing else to add; we'd need a test case. The use of workers is an obvious investigation point that you could look at, and eliminate or confirm as a cause.

[cjbj]

@rodrigosalinas is there a runnable script you can share to show us the problem with node-oracledb 5.1?

[cjbj]

@rodrigosalinas similar to #1369, it would be interesting to know if you see your problem with 5.2.0-dev.

[rodrigosalinas2]

I'll try. Thanks @cjbj

[cjbj]

@rodrigosalinas any news?

[cjbj]

I'm pre-emptively marking this as a duplicate of #1369.

[cjbj]

Now we have released node-oracledb 5.2, I'm expecting this to have been resolved.