Getting ERR_STREAM_WRITE_AFTER_END after #4139

[tmcw]

• Review the documentation: https://docs.sentry.io/
• Search for existing issues: https://github.com/getsentry/sentry-javascript/issues
• Use the latest release: https://github.com/getsentry/sentry-javascript/releases
• Provide a link to the affected event from your Sentry account

Package + Version

• @sentry/next-js

Version:

6.14.3

Description

Hi! I'm using Sentry with a Blitzjs stack, which is a mildly abstracted version of Next.js. Generally, the Next.js sentry module works very well, but after 6.14.3, I experienced crashes on any request in prod:

Nov 11 04:11:01 PM  Error [ERR_STREAM_WRITE_AFTER_END] [ERR_STREAM_WRITE_AFTER_END]: write after end
Nov 11 04:11:01 PM      at writeAfterEnd (_http_outgoing.js:694:15)
Nov 11 04:11:01 PM      at write_ (_http_outgoing.js:706:5)
Nov 11 04:11:01 PM      at ServerResponse.write (_http_outgoing.js:687:15)
Nov 11 04:11:01 PM      at ServerResponse.write (/opt/render/project/src/node_modules/next/dist/compiled/compression/index.js:1:4006)
Nov 11 04:11:01 PM      at Request.ondata (internal/streams/readable.js:745:22)
Nov 11 04:11:01 PM      at Request.emit (events.js:376:20)
Nov 11 04:11:01 PM      at Request.emit (domain.js:532:15)
Nov 11 04:11:01 PM      at addChunk (internal/streams/readable.js:309:12)
Nov 11 04:11:01 PM      at readableAddChunk (internal/streams/readable.js:284:9)
Nov 11 04:11:01 PM      at Request.Readable.push (internal/streams/readable.js:223:10)

This is noted in the PR itself, and appears to not affect Next.js in some configurations, but clearly affects my relatively vanilla setup of Blitz running on Render (and the bug also appears locally). I'll try to figure out if this is purely a Blitz-level occurrence or something that manifests in Render on serverful non-Lambda environments as well. I've set up the externalResolver option as suggested to avoid the error message that this PR attempts to fix systematically.

[AbhiPrasad]

Hey, thank you for reporting. This might be Blitz configuring the request in a special way, since we are reaching in pretty heavily into the internals to access request data. To confirm that the issue is caused by #4139, could you check if 6.14.1 works as expected?

A reproduction of some kind would be a huge help for us to get a head start on debugging - but no worries if you can't share it, we can take a look.

@lobsterkatie ping to get your thoughts here as well.

[tmcw]

Yep, I'm trying to get a reduced example for you to look at. I've swapped through the versions of @sentry/nextjs and confirm that versions <= 6.14.1 work as expected and all versions > 6.14.1 trigger a hard crash.

[tmcw]

Okay, I've figured out the cause. One of my API routes had this code in it:

res.json(pullResponse);
res.end();

Which is redundant, because res.json ends the request, but was not a problem - you can call .end on a HttpResponse multiple times with no ill effects. The changes in 6.14.3 changes this from a redundancy to a hard crash, because the .json method ends the request, then the overridden end method flips the finished flag back to false, Node tries to end the request again, and chaos ensues.

That said: I can just yank the second line from ^ that example, and it works fine. So this is more of a bug that turns non-ideal code from symptomless to very symptomful, depending on how you think about the guarantees around res.end(). Fwiw, I'd consider it still a bug because it changes the behavior of a core node api.

[AbhiPrasad]

We should aim to either address this, or document as a limitation.

Thanks for investigating @tmcw!

[bunea]

I just want to add that I got this error as well on an api endpoints that pipes a file response. Basically I just request a file from a url and pipe that response to the NextApiResponse. I guess this might happen because sentry ends the response before it's finished streaming and then .pipe complains that it has already ended.

[yousefcisco]

We're getting the same error with a next.js API route that adds authentication headers and proxies the request to a third party api endpoint. We were able to resolve the issue by removing withSentry from that API route (which is concerning). It only occurs on next.js version >12.0.5 but works fine in v12.0.4. Offending code below

import { withSentry } from '@sentry/nextjs';
import httpProxy from 'http-proxy';
import { NextApiRequest, NextApiResponse } from 'next';

/**
 * @see https://www.npmjs.com/package/http-proxy
 */
const proxy: httpProxy = httpProxy.createProxy();

async function handler(
  req: NextApiRequest,
  res: NextApiResponse,
): Promise<void> {
  if (req.method !== 'POST') {
    return res.status(405).send('405: Method not allowed');
  }

  let authorization: string;

  if (req.headers.authorization) {
    authorization = req.headers.authorization;
  } else {
    const authorization = await getServerSideUserSessionAccessToken({
      req,
      res,
    });
  }

  return new Promise((resolve, reject) => {
    req.url = req.url?.replace(
      pathRegex,
      `/graphql`,
    );

    proxy
      .once('proxyRes', resolve)
      .once('error', reject)
      .web(req, res, {
        target: 'https://third-party.com',
        headers: {
          authorization,
          cookie: '',
        },
        changeOrigin: true,
      });
  });
}

export const config = {
  api: {
    bodyParser: false,
    externalResolver: true,
  },
};

export default withSentry(handler);

[joggienl]

Stumbled on this today.

Found this interesting monologue in the sentry source:

sentry-javascript/packages/nextjs/src/utils/withSentry.ts

Line 82 in fb66a78

// Temporarily mark the response as finished, as a hack to get nextjs to not complain that we're coming back

I have some cases where I call .end() after .json().. removing the .end() "fixes" the issue. But just like @bunea I also pipe a file through NextApiResponse. Not sure how to tackle this yet, removing withSentry seems to be the quickfix but is not what I would like to do 🤔