Tests logging after teardown do not fail tests in non-watch mode

[NMinhNguyen]

🐛 Bug Report

#10728 sets the exit code to 1 when tests log after teardown. This correctly terminates the process if run in non-watch mode. However, if you do jest --watch then all tests are reported as passing, despite there being a warning:

Cannot log after tests are done. Did you forget to wait for something async in your test?

Additionally, I noticed that console.warn and console.error don't cause failures - is that intended? I'm also happy to open a dedicated issue if you prefer keeping things isolated.

To Reproduce

Steps to reproduce the behavior:

1. Given this test:

   // sum.test.js
   
   const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
   
   test('should fail in watch mode', () => {
     async function foo() {
       await delay(100);
   
       console.log('this should fail tests in watch mode too');
     }
   
     foo();
     expect(true).toBe(true);
   });

2. Run yarn jest --watchAll (or --watch if running inside a git repo)

3. Observe that Jest reports a passing test, although it does log a warning:

   

4. Now if you just run yarn jest, you'll see the process exits with exit code 1.

Expected behavior

Jest should report a test failure.

Link to repl or repo (highly encouraged)

https://repl.it/@NMinhNguyen/jest-playground#sum.test.js

Note: you'll need to run it in watch mode: yarn jest --watchAll (--watchAll since it's not inside a git repo).

envinfo

npx envinfo --preset jest
npx: installed 1 in 3.502s

  System:
    OS: Linux 5.4 Debian GNU/Linux 9 (stretch) 9 (stretch)
    CPU: (4) x64 Intel(R) Xeon(R) CPU @ 2.30GHz
  Binaries:
    Node: 12.18.3 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.6 - /usr/local/bin/npm
  npmPackages:
    jest: 27.0.0-next.3 => 27.0.0-next.3 

[snitin315]

I will take a look and send a PR for this in near future.

[SimenB]

Thanks @snitin315!

[noobyogi0010]

@SimenB Can I take this if no one's working on it already?

[itsmingjie]

Hi! I'm one of the program coordinators of a nonprofit program called CodeDay Init, and we partner with colleges to help CS students make their first open-source contribution.

We'll claim this issue for now and a group of students will be on it soon with mentorship! If this issue is no longer relevant, please let me know.

[SimenB]

AFAIK this is still relevant - can't remember merging anything that should affect it at least 🙂

[regevbr]

I'm experiencing the same issue - when running on my machine, I get the Cannot log after tests are done. Did you forget to wait for something async in your test? errors but Jest still finishes with exit code 0. When running the same jest command, but with --runInBand Jest does exit with code 1 like it should. I took me hours to figure out why Jest failed in my CI machine and not locally :-(

[chrisgarber]

I also spent many hours trying to debug this. IMO the user experience of having no tests fail, but having the process exit with a non-0 status is extremely confusing. Is there any way to correlate logging with the test that may have issued the promise (or whatever) that caused the logging after cleanup? The other problem that I experienced is that it was quite difficult to definitively find which test was causing the issue.