ShareDB/ws occasionally crashes Node when "readyState 2 (CLOSING)" #275
Comments
|
There seems to be a race condition between the stream and the socket closing, when the connection is terminated by the client. I'm not sure, if it can be fixed, however, the error can definitely be handled. To do that, switch to @teamwork/websocket-json-stream, which is a drop-in replacement for websocket-json-stream but reports errors correctly. Then you'd just need to listen for the @ericyhwang Should README.md be updated to recommend |
|
This is brilliant. Just to clarify, when you say "listen for error events",
what does that look like in practice?
…On Mon, Mar 4, 2019 at 7:34 AM Greg Kubisa ***@***.***> wrote:
There seems to be a race condition between the stream and the socket
closing, when the connection is terminated by the client. I'm not sure, if
it can be fixed, however, the error can definitely be handled. To do that,
switch to @teamwork/websocket-json-stream
***@***.***/websocket-json-stream>, which is
a drop-in replacement for websocket-json-stream
<https://www.npmjs.com/package/websocket-json-stream> but reports errors
correctly. Then you'd just need to listen for the error events on the
stream and ignore those related to writing to a closed or closing socket.
We've been using this approach in production for months without problems.
@ericyhwang <https://github.com/ericyhwang> Should README.md be updated
to recommend @teamwork/websocket-json-stream and advice about handling
the above mentioned errors?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#275 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AADwh5dFC8O5Xvl5NFM0hfqaiaSDdjAkks5vTL53gaJpZM4bbQRw>
.
--
http://reganmian.net/blog -- Random Stuff that Matters
|
|
Yeah, we wound up also rewriting our own version of export default class WebSocketJsonStream extends Duplex {
private readonly webSocket: WebSocket;
public constructor(webSocket: WebSocket) {
super({ objectMode: true });
this.webSocket = webSocket;
this.registerWebSocketListeners();
this.registerStreamListeners();
}
public _read() {}
public _write(message: any, encoding: any, callback: (error: Error) => void) {
this.webSocket.send(JSON.stringify(message), callback);
}
private registerWebSocketListeners() {
this.webSocket.on('message', (message: string) => {
this.receiveMessage(message);
});
this.webSocket.on('close', () => {
this.closeStreams();
});
}
private registerStreamListeners() {
this.on('error', (error: any) => {
logger.debug('Websocket JSON stream error', { error });
this.closeSocket();
});
this.on('end', () => {
this.closeSocket();
});
}
private receiveMessage(message: string) {
this.push(JSON.parse(message));
}
private closeStreams() {
this.push(null);
this.end();
this.emit('close');
this.emit('end');
}
private closeSocket() {
this.webSocket.close();
}
} |
stream.on('error', error => {
if (error.message.startsWith('WebSocket is not open')) {
// No point reporting this error, as it happens often and is harmless.
return
}
// Log the error, etc.
}) |
|
Just to be 100% sure, does this look good to you?
|
|
Yes, that should stop the server crashes related to stream errors. |
Some time last year, I believe @nateps mentioned he wanted to switch all of Share's examples over to use the Teamwork version of websocket-json-stream? Did he talk to you about that? I'm not sure how far he got. I can check with him on that, but in the meantime, if you make a PR for the README change, I can get that merged in, especially since you've been using it in production for a while now. (Nate and I are using the browserchannel adapter in production, so for the websocket adapter, it makes sense to leverage your production experience there.) |
|
PR for the README change: #276. I've just improved
Yes, he mentioned that last year but I haven't heard anything more about it since.
Well, we've recently switched over to SockJS to be able to work over corporate proxies, however, |
|
@gkubisa I'm evaluating SockJS and I'm curious, how does it look like to use SockJS with ShareDB? Are there any examples of how to set up the backend connection? |
|
Hi @curran , on the server-side just wrap SockJS in a stream and then pass it to the On the client-side SockJS exposes a WebSocket interface, so you can simply use it instead of the native WebSocket. |
|
Thank you so much for sharing that snippet! It may prove useful one day. |
|
No problem, it can certainly be improved but it's a good start. |
|
FWIW I'm working with Webpack Dev Server, and I'm frequently encountering the error with the following message: The suggested workaround helped a lot. Here's what I'm doing, which appears to replace crashes with a benign new WebSocket.Server({server}).on('connection', ws => {
const stream = new JSONStream(ws);
// Prevent server crashes on errors.
stream.on('error', error => {
console.log(error.message); // Tends to print "WebSocket CLOSING or CLOSED."
});
share.listen(stream);
}); |
|
Closed by #291 |
Uh oh!
There was an error while loading. Please reload this page.
We expose the ShareDB server within the same Node process that runs our Meteor app (separate websockets from Meteor). It's quite important for us that this main Node process should never crash, since that would negatively impact all connected clients (even if the restart time is quick). Therefore we use try...catch for things that could possibly fail, and deal with them internally.
However, I see a few crashes in the log due to ShareDB's use of Websocket (we use ws on the backend, and reconnecting-websocket on the frontend). I see that ShareDB does check whether the websocket is closed in this snippet
(lib/agent.js:165)
But that seems to not be enough, maybe the websocket changes status between line 1 and line 3 above, or maybe this.closed doesn't capture readyState 2?
It seems that attaching a callback function on this.stream.write function also avoids throwing, but I'm not sure if it's safe for ShareDB to just disregard (which it seems to be, since it justs returns if this.closed)?
Versions:
ws 6.1.3
sharedb 1.0.0-beta.18
The text was updated successfully, but these errors were encountered: