Store snapshots by test declaration order

[sramam]

Description

Please consider not sorting snapshots when they are recorded - especially to the markdown version.

My use case

I use ava with snapshots for testing "pure-ish" functions that are composed into larger workflows.
Meaning while the functions have side effects, they are testable with input/output expectations.
Snapshots are ideal for this.

However, since these are functions with side-effects, for proper testing, they have to be run in serial order to simulate the workflow. This allows the functions to be tested in isolation, while
building upon previous steps. The cost of setup of partial workflows is significant enough that it's not practical to think of think of running each function as a test suite.

Snapshots are a great way to capture the expectations and track deviations over time.
However in snapshot-manager.js these recordings are being sorted in title-order.

Now my ability to manually inspect the output is severely impacted, requiring all kinds of
up-and-down scrolling. Some of these are API calls with 10s or even 100s of lines of data.
Sometimes, out of sheer frustration, I put numbers in the test titles, but that turns out to cause
other problems in complex refactoring/requirement-change situations.

Options I can think of:

1. Not sorting - however, this seems to be exactly counter of this comment on #2311
2. Sorting only the snap-files, but not the markdown files
3. Sorting parallel tests, but not serial tests.
4. Providing a flag/option to not sort the markdown files.
5. Storing snapshots in order they are encountered in the test files.

Strikes me that option 5 would seem to solve all problems (including #2311?), allow for side-by-side scrolling of test file and markdown, without requiring any configuration.

Really unsure of the implementation complexity though.

[novemberborn]

5. Storing snapshots in order they are encountered in the test files.

Strikes me that option 5 would seem to solve all problems (including #2311?), allow for side-by-side scrolling of test file and markdown, without requiring any configuration.

Yup, we should do this. Thanks for bringing it up.

It should be possible to assign incremental IDs as tests are declared, and use those to sort the snapshots.

[sramam]

ooh - that's so interesting that you mention incremental ids.
Perhaps I should open a new issue for this, but can't resist asking:

Could that also be used to order the verbose reports on the CLI?
That'd be so valuable when one has a large number of tests
(200+ in my case - this is the final integration module, made up of many smaller modules)

[novemberborn]

Perhaps I should open a new issue for this, but can't resist asking:

Could that also be used to order the verbose reports on the CLI?
That'd be so valuable when one has a large number of tests
(200+ in my case - this is the final integration module, made up of many smaller modules)

Yea, but we need to rewrite the reporters anyhow.

[sramam]

Yea, but we need to rewrite the reporters anyhow.

as part of this issue?

I'd be interested understanding the thinking behind changes to reporters. A quick search didn't reveal anything obvious.

[novemberborn]

as part of this issue?

No.

I'd be interested understanding the thinking behind changes to reporters. A quick search didn't reveal anything obvious.

See #2217.