Make test.only work correctly with --concurrency flag.

[jamestalmage]

Right now, if you use test.only, it affects your entire test suite.

Add a test.only to one file, and it suppresses tests from being run in other files.

Unfortunately, this currently doesn't work when using the --concurrency flag. It also introduces quite a bit of extra code into our smart watcher.

While I think the original behavior is probably ideal, I wonder if it's not worth reducing it's power a bit, just to make our implementations easier.

Specifically, if we made test.only suppress only other tests in the same file, we could make a sensible concurrency the default behavior (right now the default behavior is unlimited concurrency). Thus far, we have had a very positive response on the --concurrency flag, and I think making it the default behavior would probably benefit most users.

• What does everyone think?
• How important is the current behavior of test.only to you?

A few things to remember:

• If you are using smart watcher and change a test file, it's the only test that will be rerun anyways, so adding test.only to a test will cause only that file to run.
• You can always use the CLI to focus in on just a few files, this can give you behavior similar to how test.only currently behaves, but it's definitely a little less convenient.

[novemberborn]

I think the code in the watcher is manageable. We need to take a stab at implementing it for --concurrency though. If that proves overly complicated then I'd be OK with reducing the scope of .only.

I really like the predictability of .only though, no matter how you run AVA (experimental features like --concurrency aside).

[sindresorhus]

How important is the current behavior of test.only to you?

I use it a lot with test suits of multiple files. It would be an inconvenience having to manually specify the path to the test file.

If that proves overly complicated then I'd be OK with reducing the scope of .only.

I think we should try first.

[arosenthal-r7]

I have a question @jamestalmage, you said:

If you are using smart watcher and change a test file, it's the only test that will be rerun anyways, so adding test.only to a test will cause only that file to run.

That is not the behavior I experience in watch mode. If I modify a source file then it only reruns the corresponding test file, but if I'm working on adding tests or fixing tests on any change it re-runs the whole test suite. This is the AVA config I'm using:

"ava": {
    "babel": "inherit",
    "concurrency": 10,
    "failFast": true,
    "files": [
        "./test/**/*.js",
        "!./test/helpers/**",
        "!./test/fakeData.js",
        "!./test/testDate.js"
    ],
    "source": [
        "./src/**/*.js"
    ],
    "require": [
      "babel-register",
      "babel-polyfill",
      "./test/helpers/setup-browser-environment.js"
    ],
    "verbose": true
  },

Am I missing something?

[novemberborn]

Am I missing something?

Have a look at the Debugging section in the watch mode recipe. AVA probably picked up on file changes it could not trace back to a specific test, so it reruns all.

[arosenthal-r7]

ok so I ran in DEBUG mode, and it looks like you're right, I see this:
ava:watcher Sources remain that cannot be traced to specific tests. Rerunning all tests +18ms

is there any way to see which sources AVA could not trace back to tests? there are certainly parts of our codebase that do not yet (or might never) have tests and we use external modules inside some of our test files also (could that be the problem?) Without being able to use the .only (because we have concurrency set) it makes it very tedious to write tests when they get all re-run every time. If anyone has any tips or tricks (other than the obvious of setting the specific path to the file I'm testing), please let me know. Thanks

[novemberborn]

@arosenthal-r7 it should have output the files that changed. This is off-topic for this issue though, I'm in Gitter at the moment, perhaps chat there? https://gitter.im/avajs/ava

[novemberborn]

Closing in favor of #1472.