Skip to content

feat: new Aggressive Reporting settings #344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Apr 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 45 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,9 @@ Assuming you are using the same pattern for your test files as [Jest by default]
};
```

#### ESLint Cascading and Hierachy
#### ESLint Cascading and Hierarchy

Another approach for customizing ESLint config by paths is through [ESLint Cascading and Hierachy](https://eslint.org/docs/user-guide/configuring/configuration-files#cascading-and-hierarchy). This is useful if all your tests are placed under the same folder, so you can place there another `.eslintrc` where you enable `eslint-plugin-testing-library` for applying it only to the files under such folder, rather than enabling it on your global `.eslintrc` which would apply to your whole project.
Another approach for customizing ESLint config by paths is through [ESLint Cascading and Hierarchy](https://eslint.org/docs/user-guide/configuring/configuration-files#cascading-and-hierarchy). This is useful if all your tests are placed under the same folder, so you can place there another `.eslintrc` where you enable `eslint-plugin-testing-library` for applying it only to the files under such folder, rather than enabling it on your global `.eslintrc` which would apply to your whole project.

## Shareable configurations

Expand Down Expand Up @@ -228,17 +228,17 @@ To enable this configuration use the `extends` property in your

In v4 this plugin introduced a new feature called "Aggressive Reporting", which intends to detect Testing Library utils usages even if they don't come directly from a Testing Library package (i.e. [using a custom utility file to re-export everything from Testing Library](https://testing-library.com/docs/react-testing-library/setup/#custom-render)). You can [read more about this feature here](docs/migrating-to-v4-guide.md#aggressive-reporting).

If you are looking to restricting this feature, please refer to the [Shared Settings section](#shared-settings) to do so. It's not possible to switch this mechanism entirely off yet, but there will be a new option in the Shared Settings in the future to be able to achieve this.
If you are looking to restricting or switching off this feature, please refer to the [Shared Settings section](#shared-settings) to do so.

## Shared Settings

There are some configuration options available that will be shared across all the plugin rules. This is achieved using [ESLint Shared Settings](https://eslint.org/docs/user-guide/configuring/configuration-files#adding-shared-settings). These Shared Settings are meant to be used if you need to restrict the Aggressive Reporting mechanism, which is an out of the box advanced feature to lint Testing Library usages in a simpler way for most of the users. **So please before configuring any of these settings**, read more about [the advantages of `eslint-plugin-testing-library` Aggressive Reporting mechanism](docs/migrating-to-v4-guide.md#aggressive-reporting), and [how it's affected by these settings](docs/migrating-to-v4-guide.md#shared-settings).
There are some configuration options available that will be shared across all the plugin rules. This is achieved using [ESLint Shared Settings](https://eslint.org/docs/user-guide/configuring/configuration-files#adding-shared-settings). These Shared Settings are meant to be used if you need to restrict or switch off the Aggressive Reporting, which is an out of the box advanced feature to lint Testing Library usages in a simpler way for most of the users. **So please before configuring any of these settings**, read more about [the advantages of `eslint-plugin-testing-library` Aggressive Reporting feature](docs/migrating-to-v4-guide.md#aggressive-reporting), and [how it's affected by these settings](docs/migrating-to-v4-guide.md#shared-settings).

If you are sure about configuring the settings, these are the options available:

### `testing-library/utils-module`

The name of your custom utility file from where you re-export everything from Testing Library package.
The name of your custom utility file from where you re-export everything from the Testing Library package, or `"off"` to switch related Aggressive Reporting mechanism off. Relates to [Aggressive Imports Reporting](docs/migrating-to-v4-guide.md#imports).

```json
// .eslintrc
Expand All @@ -249,11 +249,11 @@ The name of your custom utility file from where you re-export everything from Te
}
```

[You can find more details here](docs/migrating-to-v4-guide.md#testing-libraryutils-module).
[You can find more details about the `utils-module` setting here](docs/migrating-to-v4-guide.md#testing-libraryutils-module).

### `testing-library/custom-renders`

A list of function names that are valid as Testing Library custom renders. Relates to [Aggressive Reporting - Renders](docs/migrating-to-v4-guide.md#renders)
A list of function names that are valid as Testing Library custom renders, or `"off"` to switch related Aggressive Reporting mechanism off. Relates to [Aggressive Renders Reporting](docs/migrating-to-v4-guide.md#renders).

```json
// .eslintrc
Expand All @@ -264,26 +264,60 @@ A list of function names that are valid as Testing Library custom renders. Relat
}
```

[You can find more details here](docs/migrating-to-v4-guide.md#testing-librarycustom-renders).
[You can find more details about the `custom-renders` setting here](docs/migrating-to-v4-guide.md#testing-librarycustom-renders).

### `testing-library/custom-queries`

A list of query names/patterns that are valid as Testing Library custom queries, or `"off"` to switch related Aggressive Reporting mechanism off. Relates to [Aggressive Reporting - Queries](docs/migrating-to-v4-guide.md#queries)

```json
// .eslintrc
{
"settings": {
"testing-library/custom-queries": ["ByIcon", "getByComplexText"]
}
}
```

[You can find more details about the `custom-queries` setting here](docs/migrating-to-v4-guide.md#testing-librarycustom-queries).

### Switching all Aggressive Reporting mechanisms off

Since each Shared Setting is related to one Aggressive Reporting mechanism, and they accept `"off"` to opt out of that mechanism, you can switch the entire feature off by doing:

```json
// .eslintrc
{
"settings": {
"testing-library/utils-module": "off",
"testing-library/custom-renders": "off",
"testing-library/custom-queries": "off"
}
}
```

## Troubleshooting

### There are errors reported in non-testing files
### Errors reported in non-testing files

If you find ESLint errors related to `eslint-plugin-testing-library` in files other than testing, this could be caused by [Aggressive Reporting](#aggressive-reporting).

You can avoid this by:

1. [running `eslint-plugin-testing-library` only against testing files](#run-the-plugin-only-against-test-files)
2. [limiting the scope of Aggressive Reporting through Shared Settings](#shared-settings)
3. [switching Aggressive Reporting feature off](#switching-all-aggressive-reporting-mechanisms-off)

If you think the error you are getting is not related to this at all, please [fill a new issue](https://github.com/testing-library/eslint-plugin-testing-library/issues/new/choose) with as many details as possible.

### There are false positives in testing files
### False positives in testing files

If you are getting false positive ESLint errors in your testing files, this could be caused by [Aggressive Reporting](#aggressive-reporting).

You can avoid this by [limiting the scope of Aggressive Reporting through Shared Settings](#shared-settings)
You can avoid this by:

1. [limiting the scope of Aggressive Reporting through Shared Settings](#shared-settings)
2. [switching Aggressive Reporting feature off](#switching-all-aggressive-reporting-mechanisms-off)

If you think the error you are getting is not related to this at all, please [fill a new issue](https://github.com/testing-library/eslint-plugin-testing-library/issues/new/choose) with as many details as possible.

Expand Down
107 changes: 95 additions & 12 deletions docs/migrating-to-v4-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ So what is this Aggressive Reporting introduced on v4? Until v3, `eslint-plugin-
- users can [re-export Testing Library utils from a custom module](https://testing-library.com/docs/react-testing-library/setup/#configuring-jest-with-test-utils), so they won't be imported from a Testing Library package but a custom one.
- users can [add their own Custom Queries](https://testing-library.com/docs/react-testing-library/setup/#add-custom-queries), so it's possible to use other queries than built-in ones.

These customization mechanisms make impossible for `eslint-plugin-testing-library` to figure out if some utils are related to Testing Library or not. Here you have some examples illustrating it:
These customization possibilities make it impossible for `eslint-plugin-testing-library` to figure out if some utils are related to Testing Library or not. Here you have some examples illustrating it:

```javascript
import { render, screen } from '@testing-library/react';
Expand Down Expand Up @@ -129,7 +129,7 @@ const el = findByIcon('profile');

How can the `eslint-plugin-testing-library` be aware of this? Until v3, the plugin offered some options to indicate some of these custom things, so the plugin would check them when reporting usages. This can lead to false negatives though since the users might not be aware of the necessity of indicating such custom utils or just forget about doing so.

Instead, in `eslint-plugin-testing-library` v4 we have opted-in a more **aggressive reporting** mechanism which, by default, will assume any method named following the same patterns as Testing Library utils has to be reported too:
Instead, in `eslint-plugin-testing-library` v4 we have opted-in into a more **aggressive reporting** feature which, by default, will assume any method named following the same patterns as Testing Library utils has to be reported too:

```javascript
// importing from Custom Module
Expand All @@ -145,39 +145,42 @@ const wrapper = renderWithRedux(<Component />);
const el = findByIcon('profile');
```

There are 3 behaviors then that can be aggressively reported: imports, renders, and queries. This new Aggressive Reporting mechanism will just work fine out of the box and won't create false positives for most of the users. However, it's possible to do some tweaks to disable some of these behaviors using the new [Shared Settings](#shared-settings). We recommend you to keep reading this section to know more about these Aggressive Reporting behaviors and then check the Shared Settings if you think you'd still need it for some particular reason.
There are 3 mechanisms that can be aggressively reported: imports, renders, and queries. This new Aggressive Reporting feature will work fine out of the box and won't create false positives for most of the users. However, it's possible to restrict or switch off these mechanisms using [Shared Settings](#shared-settings).
We recommend you to keep reading this section to know more about these Aggressive Reporting mechanisms and afterwards check the Shared Settings if you think you'd still need them for some particular reason.

_You can find the motivation behind this behavior on [this issue comment](https://github.com/testing-library/eslint-plugin-testing-library/issues/222#issuecomment-679592434)._

### Imports

By default, `eslint-plugin-testing-library` v4 won't check from which module are the utils imported. This means it doesn't matter if you are importing the utils from `@testing-library/*`, `test-utils` or `whatever`.
By default, `eslint-plugin-testing-library` v4 won't check from which module the utils are imported. This means it doesn't matter if you are importing the utils from `@testing-library/*`, `test-utils` or `whatever`: they'll be assumed as Testing Library related if matching Testing Library utils patterns.

There is a new Shared Setting to restrict this scope though: [`utils-module`](#testing-libraryutils-module). By using this setting, only utils imported from `@testing-library/*` packages, or the custom one indicated in this setting would be reported.
There is a Shared Setting property to restrict or switch off this mechanism though: [`utils-module`](#testing-libraryutils-module). By using this setting, only utils imported from `@testing-library/*` packages + those indicated in this setting (if any) will be reported.

### Renders

By default, `eslint-plugin-testing-library` v4 will assume that all methods which names contain "render" should be reported. This means it doesn't matter if you are rendering your elements for testing using `render`, `customRender` or `renderWithRedux`.

There is a new Shared Setting to restrict this scope though: [`custom-renders`](#testing-librarycustom-renders). By using this setting, only methods strictly named `render` or as one of the indicated Custom Renders would be reported.
There is a Shared Setting property to restrict or switch off this mechanism though: [`custom-renders`](#testing-librarycustom-renders). By using this setting, only methods strictly named `render` + those indicated in this setting (if any) will be reported.

### Queries

`eslint-plugin-testing-library` v4 will assume that all methods named following the pattern `get(All)By*`, `query(All)By*`, or `find(All)By*` are queries to be reported. This means it doesn't matter if you are using a built-in query (`getByText`), or a custom one (`getByIcon`): if it matches this pattern, it will be assumed as a potential query to be reported.

There is no way to restrict this behavior for now.
There is a Shared Setting property to restrict or switch off this mechanism though: [`custom-queries`](#testing-librarycustom-queries). By using this setting, only [built-in queries](https://testing-library.com/docs/queries/about) + those indicated in this setting (if any) will be reported.

## Shared Settings

ESLint has a setting feature which allows configuring data that must be shared across all its rules: [Shared Settings](https://eslint.org/docs/user-guide/configuring/configuration-files#adding-shared-settings). Since `eslint-plugin-testing-library` v4 we are using this Shared Settings to config global things for the plugin.
ESLint provides a way of configuring data that must be shared across all its rules: [Shared Settings](https://eslint.org/docs/user-guide/configuring/configuration-files#adding-shared-settings). Since `eslint-plugin-testing-library` v4 we are using this Shared Settings to config global things for the plugin.

To avoid collision with settings from other ESLint plugins, all the properties for this one are prefixed with `testing-library/`.

⚠️ **Please be aware of using these settings will disable part of [Aggressive Reporting](#aggressive-reporting).**

### `testing-library/utils-module`

The name of your custom utility file from where you re-export everything from Testing Library package. Relates to [Aggressive Reporting - Imports](#imports).
Relates to the [Aggressive Imports Reporting mechanism](#imports). This setting accepts any string value.

If you pass a string other than `"off"` to this option, it will represent your custom utility file from where you re-export everything from Testing Library package.

```json
// .eslintrc
Expand All @@ -188,7 +191,7 @@ The name of your custom utility file from where you re-export everything from Te
}
```

Enabling this setting, you'll restrict the errors reported by the plugin to only those utils being imported from this custom utility file, or some `@testing-library/*` package. The previous setting example would cause:
Configuring this setting like that, you'll restrict the errors reported by the plugin to only those utils being imported from this custom utility file, or some `@testing-library/*` package. The previous setting example would cause:

```javascript
import { waitFor } from '@testing-library/react';
Expand Down Expand Up @@ -220,9 +223,22 @@ test('testing-library/utils-module setting example', () => {
});
```

You can also set this setting to `"off"` to entirely opt-out Aggressive Imports Reporting mechanism, so only utils coming from Testing Library packages are reported.

```json
// .eslintrc
{
"settings": {
"testing-library/utils-module": "off"
}
}
```

### `testing-library/custom-renders`

A list of function names that are valid as Testing Library custom renders. Relates to [Aggressive Reporting - Renders](#renders)
Relates to the [Aggressive Renders Reporting mechanism](#renders). This setting accepts an array of strings or `"off"`.

If you pass an array of strings to this option, it will represent a list of function names that are valid as Testing Library custom renders.

```json
// .eslintrc
Expand All @@ -233,7 +249,7 @@ A list of function names that are valid as Testing Library custom renders. Relat
}
```

Enabling this setting, you'll restrict the errors reported by the plugin related to `render` somehow to only those functions sharing a name with one of the elements of that list, or built-in `render`. The previous setting example would cause:
Configuring this setting like that, you'll restrict the errors reported by the plugin related to `render` somehow to only those functions sharing a name with one of the elements of that list, or built-in `render`. The previous setting example would cause:

```javascript
import {
Expand Down Expand Up @@ -269,3 +285,70 @@ test('testing-library/custom-renders setting example', () => {
const invalidUsage = setupB(<Component />);
});
```

You can also set this setting to `"off"` to entirely opt-out Aggressive Renders Reporting mechanism, so only methods named `render` are reported as Testing Library render util.

```json
// .eslintrc
{
"settings": {
"testing-library/custom-renders": "off"
}
}
```

### `testing-library/custom-queries`

Relates to the [Aggressive Queries Reporting mechanism](#queries). This setting accepts an array of strings or `"off"`.

If you pass an array of strings to this option, it will represent a list of query names/variants that are the only valid Testing Library custom queries.

Each string passed to this list of custom queries can be:

- **pattern query (recommended)**: a custom query variant (suffix starting with "By") to be reported, so all query combinations around it are reported. For instance: `"ByIcon"` would report all `getByIcon()`, `getAllByIcon()`, `queryByIcon()` and `findByIcon()`.
- **strict query**: a specific custom query name to be reported, so only that very exact query would be reported but not any related variant. For instance: `"getByIcon"` would make the plugin to report `getByIcon()` but not `getAllByIcon()`, `queryByIcon()` or `findByIcon()`.

```json
// .eslintrc
{
"settings": {
"testing-library/custom-queries": ["ByIcon", "getByComplexText"]
}
}
```

Configuring this setting like that, you'll restrict the errors reported by the plugin related to the queries to only those custom queries matching name or pattern from that list, or [built-in queries](https://testing-library.com/docs/queries/about). The previous setting example would cause:

```javascript
// ✅ this would be reported since `getByText` is a built-in Testing Library query
getByText('foo');

// ✅ this would be reported since `findAllByRole` is a built-in Testing Library query
findAllByRole('foo');

// ✅ this would be reported since `getByIcon` is a custom query matching "ByIcon" setting
getByIcon('foo');

// ✅ this would be reported since `findAllByIcon` is a custom query matching "ByIcon" setting
findAllByIcon('foo');

// ✅ this would be reported since `getByComplexText` is a custom query matching "getByComplexText" etting
getByComplexText('foo');

// ❌ this would NOT be reported since `getAllByComplexText` is a custom query but not matching any setting
getAllByComplexText('foo');

// ❌ this would NOT be reported since `findBySomethingElse` is a custom query but not matching any setting
findBySomethingElse('foo');
```

You can also set this setting to `"off"` to entirely opt-out Aggressive Queries Reporting mechanism, so only built-in queries are reported.

```json
// .eslintrc
{
"settings": {
"testing-library/custom-queries": "off"
}
}
```
Loading