EFAULT, EPIPE errors on Big Sur on Apple Silicon (M1)

[andreialecu]

• Version: 15.5.0
• Platform: Darwin Kernel Version 20.2.0: Wed Dec 2 20:40:21 PST 2020; root:xnu-7195.60.75~1/RELEASE_ARM64_T8101 arm64
• Subsystem: streams(?)

What steps will reproduce the bug?

Using node 15.5.0 compiled for arm64.

➜ node -p process.arch
arm64

I'm baffled that I couldn't find anything related to this yet, and I'm not sure whether it's a nodejs bug or something else.

While running nodejs apps that use network or file streams it seems that they very frequently break with errors like EPIPE or EFAULT.

For example, a recent crash:

Error: read EFAULT
    at Pipe.onStreamRead (node:internal/stream_base_commons:213:20)
    at Pipe.callbackTrampoline (node:internal/async_hooks:131:14)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:188:8)
    at emitErrorCloseNT (node:internal/streams/destroy:153:3)
    at processTicksAndRejections (node:internal/process/task_queues:80:21) {
  errno: -14,
  code: 'EFAULT',
  syscall: 'read'
}

This also happens quite a lot in the nodejs subsystem used in VSCode: microsoft/vscode#113410 (see various screenshots attached there)

I haven't verified if it happens in Rosetta emulation.

How often does it reproduce? Is there a required condition?

Very often, at least once every 10-60 minutes.

I don't have a specific reproduction, but I've seen it happen when something like tsc --watch reinitiates a compilation based on file system changes. It also happens randomly in simple expressjs servers that don't watch the file system, but just keep a connection to a mongodb server.

What is the expected behavior?

More stability.

What do you see instead?

Frequent errors.

Additional information

If I can run with some sort of flag that produces an useful crash dump, let me know of the instructions and I'll do it asap.

[andreialecu]

Only other mention of this I could find is here: desktop/desktop#11258 which is also from an Electron app.

Electron uses node ~12 though. I don't believe it to be isolated to 15.

[andreialecu]

Similar error when a file change event restarts tsc compilation (using tsc-watch):

(using yarn v2 here, hence the unusual stack trace)

TypeError: Cannot read property 'map' of undefined
    at Stream.<anonymous> (/Users/andreialecu/Work/x/x-monorepo/.yarn/$$virtual/tsc-watch-virtual-c03da97c37/0/cache/tsc-watch-npm-4.2.9-f898a4f291-b1c2a2d295.zip/node_modules/tsc-watch/lib/killer.js:24:69)
    at Stream.emit (node:events:376:20)
    at Socket.reemit (/Users/andreialecu/Work/x/x-monorepo/.yarn/cache/duplexer-npm-0.1.2-952c810235-5c2ccea7c8.zip/node_modules/duplexer/index.js:85:16)
    at Socket.emit (node:events:376:20)
    at emitErrorNT (node:internal/streams/destroy:188:8)
    at emitErrorCloseNT (node:internal/streams/destroy:153:3)
    at processTicksAndRejections (node:internal/process/task_queues:80:21)
The terminal process "/bin/zsh '-c', 'yarn run apigql:dev'" terminated with exit code: 1.

Seems to be the same underlying problem.

[andreialecu]

Possibly also related, via the official eslint extension in vscode (I'm not sure if this runs via Electron and its node 12.18.3 or the locally installed node 15.5):

ERR Request textDocument/codeAction failed with message: Cannot read .eslintignore file: /Users/andreialecu/.../.eslintignore
Error: EFAULT: bad address in system call argument, read: Error: Request textDocument/codeAction failed with message: Cannot read .eslintignore file: /Users/andreialecu/.../.eslintignore
Error: EFAULT: bad address in system call argument, read
	at /Users/andreialecu/.vscode-insiders/extensions/dbaeumer.vscode-eslint-2.1.14/client/out/extension.js:1:52031
	at /Users/andreialecu/.vscode-insiders/extensions/dbaeumer.vscode-eslint-2.1.14/client/out/extension.js:1:52325
	at Immediate.<anonymous> (/Users/andreialecu/.vscode-insiders/extensions/dbaeumer.vscode-eslint-2.1.14/client/out/extension.js:1:52690)
	at processImmediate (internal/timers.js:456:21)

[andreialecu]

(friendly ping @aduh95, noticed some relevant nodejs macOS arm64 commits)

[aduh95]

Sorry I can't really help you there, I haven't experience any of the problems described above 🤷‍♂️

[JvManuel]

I also experienced this error. I am using M1 macbook pro right now and I haven't experience this kind of error before when I am using Macbook pro 2015 and Linux. I still don't know how to fix this issue tho. When my program stopped, I just rerun my program.

[ashveen]

Had to switch to pm2 for automatic restart on my development M1 Mac mini because the EFAULT happens so often.
It is still very annoying to have these micro outage during development.

Might even go with cluster mode to ensure there's always and instance up.

[andreialecu]

In my initial post I mentioned:

If I can run with some sort of flag that produces an useful crash dump, let me know of the instructions and I'll do it asap.

Perhaps we could help figure it out because it's fairly reproducible. It's possible it's a libuv issue though.

[andreialecu]

One other possibly relevant piece of information. I've seen relatively frequent crashes of various other executables with SIGILL. Especially VSCode arm64's extension host process crashes frequently with that code. More about that here: microsoft/vscode#113410

It seems that the crashes occur more frequently while memory pressure is high and swapping is occuring.

I'm not sure if SIGILL is related to these EFAULT based crashes, but there's a report that describes SIGILL occuring during page faults here:

https://openradar.appspot.com/FB8922558

Also potentially relevant is this issue on the Go issue tracker: golang/go#42774

They were able to find a solution for it in golang/go@7f688d1

[andreialecu]

Update: I replaced my 8GB of RAM Mac Mini M1 with a 16GB one and I haven't seen any single crash in a couple of days. In particular I haven't seen any SIGILL crashes in VSCode, and mostly no EFAULT errors either by nodejs apps (maybe one in two days, vs several every hour on the 8GB RAM machine).

The issue reported in my comment above mentions:

In the wild, the bug occurs most readily on a system under heavy load (particularly, memory pressure).

Seems like a macOS bug, but it appears that node could implement a workaround if Apple doesn't fix it: libuv/libuv#3095 (comment)

[tobspr]

+1 to this issue, I'm facing this issue in nodemon which makes the process crash every 1-3 restarts, as well as some external libraries like imagemin-pngquant. Is there anything we can do to help tracking down the issue? (MacBook Pro M1, 8GB ram)

[karlhorky]

I'm also having a seemingly related issue, with running this program in the background (on an M1 MacBook Pro - 16GB):

https://github.com/karlhorky/do-not-disturb-during-zoom-macos

Here's the error I get:

node:events:353
      throw er; // Unhandled 'error' event
      ^

Error: read EFAULT
    at Pipe.onStreamRead (node:internal/stream_base_commons:213:20)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:188:8)
    at emitErrorCloseNT (node:internal/streams/destroy:153:3)
    at processTicksAndRejections (node:internal/process/task_queues:80:21) {
  errno: -14,
  code: 'EFAULT',
  syscall: 'read'
}

I find that it often has happened during sleep / wake cycles, but that could also just be coincidence.

[karlhorky]

Wonder if the Big Sur 11.2 update that just appeared will help. Just installed now.

[andreialecu]

@karlhorky you're going to have a bad time with 11.2: #37061

[karlhorky]

Ooh looks fancy, thanks for the link! Hopefully don't run into this! In any case, worth it to have Bluetooth working reliably again.

[maximilianschmid]

Same here, long running node script produces this error on M1 Mac mini, Big Sur 11.1 and 11.2 and NodeJS 15.x.
Same script on x86 and same NodeJS version does not produce this error.

Recent crash:

Tue Feb 02 2021 08:51:37 GMT+0100 (Mitteleuropäische Normalzeit): no updates detected.
node:events:356
throw er; // Unhandled 'error' event
^

Error: read EFAULT
    at Pipe.onStreamRead (node:internal/stream_base_commons:211:20)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:188:8)
    at emitErrorCloseNT (node:internal/streams/destroy:153:3)
    at processTicksAndRejections (node:internal/process/task_queues:81:21) {
  errno: -14,
  code: 'EFAULT',
  syscall: 'read'
}
npm ERR! code 1
npm ERR! path /Users/cypress1/planfred3/testapp
npm ERR! command failed
npm ERR! command sh -c node run.js

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/cypress1/.npm/_logs/2021-02-02T07_51_44_152Z-debug.log

[andreialecu]

I haven't recently seen this error at all, but I upgraded my M1 from an initial 8GB of RAM one, to 16.

I have also been using early 11.2 betas, back before nodejs got broken completely in recent RCs (#37061)

There was a bug reported in macOS 11.1 on M1 chips that occurs when memory pressure is high, which is much more likely on the 8GB versions. See discussion in: libuv/libuv#3095 (comment)

So I think it is relevant to mention which version of the M1 everyone is using:

8GB or 16GB?

[karlhorky]

16GB. Updated my post above: #36826 (comment)

[Biboswan]

We are facing the same issue. A meteor application after some time is giving the same EFAULT error... being discussed here.
Device is Macbook Pro M1 8gb RAM

[maximilianschmid]

8GB mac mini which produces this error.

[zvictor]

We are facing the same issue. A meteor application after some time is giving the same EFAULT error... being discussed here.
Device is Macbook Pro M1 8gb RAM

Increasing the memory limit has helped us immensely. Could it be that Rosetta uses much more memory to do its work?

If so, this should help you all for now:

#!/bin/bash

if [ "$(sysctl -in sysctl.proc_translated)" = "1" ]; then
    echo "Meteor: Rosetta2: Limitting memory to 2gb"
    export NODE_OPTIONS="--max_old_space_size=2048"
else
    echo "Meteor: Native-intel: Limitting memory to 512mb"
    export NODE_OPTIONS="--max_old_space_size=512"
fi

your-cmd-here

[JvManuel]

Is there someone who was able to fix this issue?

[timmywheels]

Is there someone who was able to fix this issue?

I was able to get around this issue by adding --max_old_space_size=2048 flag to my npm run start script per @zvictor's comment

[AltamimiYasser]

Having the same issue, Macbook Air, m1 ship, 16gp ram.

[ChALkeR]

I can't reproduce this on macOS 11.4 anymore on 8GB/256GB with the isolated testcase that I got above in #36826 (comment), which reproduced this reliably on macOS 11.2.3.
Also, I can't reproduce the original issue with ungit from #36826 (comment) anymore after the update.

Unless there are or will be reports with this issue occuring on 11.4 setups, I think it's safe to assume that this was fixed in macOS.

[ChALkeR]

I'm going to close this shortly unless anyone objects.

Will reopen if there would be any new cases reported with macOS 11.4 or newer.

[GautierT]

Hi @ChALkeR, I spent the last 2 days with 11.4 and no EFAULT anymore ! 🥳 🥳

[ianmartorell]

I've been on 11.4 for a couple weeks and haven't had the issue 😄

[andreialecu]

Awesome! I've been on 11.4 for a while also.

I'll close this myself. Thanks everyone!

[vashisth00]

I am still getting the issue on MacOS 11.4 ...

[vashisth00]

events.js:377js
throw er; // Unhandled 'error' event
^

Error: read EFAULT
at Pipe.onStreamRead (internal/stream_base_commons.js:209:20)
Emitted 'error' event on Socket instance at:
at emitErrorNT (internal/streams/destroy.js:106:8)
at emitErrorCloseNT (internal/streams/destroy.js:74:3)
at processTicksAndRejections (internal/process/task_queues.js:82:21) {
errno: -14,

[maximelafarie]

Same issue with Node:current (16.x.x) on similar Mac configuration. Several memory pressure tests on a route using createReadStream seem to make it crash:

Error: read EFAULT
    at Pipe.onStreamRead (node:internal/stream_base_commons:211:20)
    at Pipe.callbackTrampoline (node:internal/async_hooks:130:17)
[ERROR] 17:45:21 Error: read EFAULT

Additional note: it seems it crashes only few minutes after the pressure test passed (without errors during the test).

[ChALkeR]

@vashisth00 @maximelafarie What do you use to reproduce this?
The script from #36826 (comment) or something else?

[maximelafarie]

@vashisth00 @maximelafarie What do you use to reproduce this? The script from #36826 (comment) or something else?

I'm creating files massively with fs.writeStream, but I will run your scripts anyway to be sure.

[marcotas]

I'm having this issue with M1 Pro Chip since I got the new MacBook Pro 16". Anyone else having the same issue? Did anyone solve it for M1 Pro?

Error: write EPIPE
    at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (internal/streams/destroy.js:106:8)
    at emitErrorCloseNT (internal/streams/destroy.js:74:3)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  errno: -32,
  code: 'EPIPE',
  syscall: 'write'
}
Exited with code: 1

Node version: 14.18.1 (but happens with any version).

[ChALkeR]

@marcotas

1. What's the macOS version?
2. Could you reproduce using the testcase above? EFAULT, EPIPE errors on Big Sur on Apple Silicon (M1) #36826 (comment)
3. If not, could you share a testcase?

[andreialecu]

@ChALkeR I'm not sure if this is directly related, but just going to add it here.

I was recently troubleshooting a node crash inside docker builds on my M1 machine.
There is a repro here:
https://github.com/andreialecu/yarn-docker-crash-repro

Run it with: docker build --tag repro --no-cache --progress=plain --file packages/backend/Dockerfile .

Basically when Docker is configured to the default on Mac (which is maximum 2GB of memory), the yarn install step crashes a majority of the time with exit code 1 but no error output at all. Sometimes it goes through, but most times it crashes on the Fetch step.

If I increase Docker memory to 4GB it builds properly every time.

I believe this may be caused by memory pressure. I wasn't able to reproduce it with --max-old-space-size at all however.

It could also be a docker bug of some sort, I'm not sure. I tried node 16 and 17 and it occurs on both.

There was more discussion on this topic on the Yarn discord here: https://discord.com/channels/226791405589233664/910944274953175082/911221057732636672

[micaiah-effiong]

I'm having this issue with MacBook Pro M1, 2020, 8Gb ram, 2020.

events.js:377
      throw er; // Unhandled 'error' event
      ^

Error: read EFAULT
    at Pipe.onStreamRead (internal/stream_base_commons.js:209:20)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (internal/streams/destroy.js:106:8)
    at emitErrorCloseNT (internal/streams/destroy.js:74:3)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  errno: -14,
  code: 'EFAULT',
  syscall: 'read'
}
error Command failed with exit code 1.

Node version v14.18.0

[ChALkeR]

@micaiah-effiong could you please share a testcase?
And macOS version.

Also see #36826 (comment)

[marcotas]

@ChALkeR I'm using Monterrey 12.0.1. Tried node v17.1.0, 16.13.0 and 14.18.1 with no success. The application I'm running is meteor framework and they launched a recent version 2.5.1 compatible with M1 chips, but didn't work anyways too. It doesn't seem to be a framework problem (I suppose).

I have no idea how to solve this and it seems to be related to memory usage? Anyways I'm following up with this issue to see if someone shows up with the same issue. I also tried to use nvm and didn't solve my problem too.

[ChALkeR]

@marcotas Thanks for the OS info! Could you try to reproduce the issue on #36826 (comment)?
That's a stand-alone script with no deps.

If it doesn't reproduce, please try adding memory pressure (e.g. via the bottom part of that comment).

Also: does the setup with Meteor spawn child processes at all or not?

[micaiah-effiong]

@micaiah-effiong could you please share a testcase? And macOS version.

Also see #36826 (comment)

I using a macOS Big Sur version 11.1.
I am running an express server in a dev environment am I use nodemon to restart the server. After a couple of restarts, it crashes with the error

events.js:377
      throw er; // Unhandled 'error' event
      ^

Error: read EFAULT
    at Pipe.onStreamRead (internal/stream_base_commons.js:209:20)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (internal/streams/destroy.js:106:8)
    at emitErrorCloseNT (internal/streams/destroy.js:74:3)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  errno: -14,
  code: 'EFAULT',
  syscall: 'read'
}
error Command failed with exit code 1.

[ChALkeR]

@micaiah-effiong This was known on 11.1 due to a macOS issue and it was previously thought that it was resolved in macOS 11.4.

But there are new reports on 12 above, though I wasn't yet able to reproduce.
It would be great if someone would be able to either reproduce with the testcase from #36826 (comment) on any macOS version >= 11.4 or share a testcase that reproduces this on a macOS version >= 11.4.

[marcotas]

@ChALkeR I wasn't able to reproduce the error with your comment you suggested. Nothing happened at all. It seems to me it's something related to websockets though. Meteor framework uses websockets a LOT so I think this might be the case. I also tried this solution in Safari but it still happening, also tried to use only chrome but didn't change anything. Unfortunately it's not something I'm able to reproduce, because I'm just facing this in a meteor framework application. I'll give any more updates if I figure out something more.

[ChALkeR]

@marcotas
Thanks! If you could provide at least some way to reproduce (e.g. a repo/setup) I could try to work from there to build a minimal testcase.

A repo which has some meteor setup triggering this might also help.

Preferably without docker or any other extra parts.

[marcotas]

@ChALkeR after updating my MacOS to 12.1 it just stop with this error. I'm not facing this error anymore. It seems it was something on Apple's end.

[IlyaEremin]

@marcotas to me upgrade to 12.1 fixed damn error for a while and then I started to get it again. Problem goes away after I restart my mac. So probably upgrade to 12.1 has nothing to do with problem. It's rather reboot made a difference for you and me.

[ChALkeR]

Hmm. I still couldn't reproduce this, but i didn't try keeping mac active for a while.

@IlyaEremin A few qustions that might help:

1. Is it the reboot or does logout/relogin help?
2. Does just closing all other apps help?
3. Is this perhaps memory load related?
4. Is there a way to trigger this quicker after a reboot?

[marcotas]

@marcotas to me upgrade to 12.1 fixed damn error for a while and then I started to get it again. Problem goes away after I restart my mac. So probably upgrade to 12.1 has nothing to do with problem. It's rather reboot made a difference for you and me.

@IlyaEremin I was facing the same issue sometimes until I figured out that the problem was with the wkhtmltopdf package not installed on my machine using homebrew. So I just installed using homebrew and I haven't gotten this error anymore.

[IlyaEremin]

@marcotas man.... this is a miracle. I'm using wkhtmltopdf in the project I work on and this was exactly the problem. After I installed one problem disappeared. Thank you a lot!

[Sun-Woo-Kim]

try this

watchman watch-del-all