Cleaning up the issue backlog

[obestwalter]

We have more than 800 open issues, quite a few of them very old and with no proper follow-up and quite a few might be good first issues, if provided with a bit more context.

We had a small discussion already with @The-Compiler and @webknjaz - the rough plan is going through issues with these things in mind:

1. If the issue is really old and it is not quick/easily to reproduce it, ask for follow-up and mark with needs: information - we already have automation that those are auto-closed then after a month
2. if it is easy to reproduce: reproduce it, add a comment that verifies the reproduction and update the original issue comment with that info and a link to the reproducing comment, so that it is clear right away that this might be an old issue but it is still valid
3. If an issue is obviously already solved or invalid, close it right away with the proper explanation
4. If something is not yet labelled properly, add the proper label (D'UH!)
5. If something looks like a good first issue, add the label (and maybe add a bit more context for people to get started)
6. Review the help wanted, as basically we want help with everything, so maybe relabel the ones that are suitable to good first issue and simply remove the label in the other issues.

When looking at the labels with @The-Compiler, we also had a few thoughts about improving the labelling a bit, which boils down to:

• rename type:docs to topic:docs as this fits better there
• rename type:infrastructure to topic:infrastructure same reason as above
• rename type:feature-branch to type:feature - we do not really have feature-branches as such anymore
• relabel type:enhancements to type:feature and delete type:enhancements

[obestwalter]

@nicoddemus, @Zac-HD, @bluetech, @RonnyPfannschmidt, @flub - would be nice if you could give some feedback if there is anything catastrophically wrong / missing. I want to start with this then later in the week. This is something that can be done in smaller chunks after the sprint, so not really time critical, but would be nice if we can get consensus about this while everybody is in sprint mode and also ideally already get started with this.

[Zac-HD]

This sounds good to me! I'd add:

• close issues which could feasibly be implemented as plugins outside this repo
• automatically close feature requests which are more than one year old.
  • I think this is a good reason to keep separate feature and enhancement labels, or some other way to distinguish "improving an existing feature" vs "adding a new feature".
• a docs page explaining the labels and criteria to close an issue would be helpful for both maintainers and contributors (both people opening issues, and people opening PRs)

Random other thoughts:

• I'm pretty dubious about whether we have any "good first issue"s; they tend to be things that looked easy but often turn into a classic yak-shaving adventure which is not a good experience for new OSS contributors.
• It would be nice if the bug issues were prioritized for fixing, but 🤷 if the volunteers don't want to, I'm not telling them to.

[flub]

sounds reasonable. I think labelling features is probably better than closing as they can more easily be referenced as already existing. Otherwise it is harder to remove duplicates.

[flub]

or just closing is probably more useful at the size of the issue tracker...

I also can't help but note that this issue adds one more open issue :P

[obestwalter]

I also can't help but note that this issue adds one more open issue :P

:D - oh trust me: I'll be closing enough to balance that out during the sprint.

[obestwalter]

Ok, here is the refined plan (if you want to just check the diff: obestwalter/pytest-sprint-2024@68f06c7)

We have more than 800 open issues, quite a few of them very old and with no proper follow-up and quite a few might be good first issues, if provided with a bit more context.

We had a small discussion already with @The-Compiler and @webknjaz - the rough plan is going through issues with these things in mind:

1. If the issue is really old and it is not quick/easily to reproduce it, ask for follow-up and mark with needs: information - we already have automation that those are auto-closed then after a month
2. if it is easy to reproduce: reproduce it, add a comment that verifies the reproduction and update the original issue comment with that info and a link to the reproducing comment, so that it is clear right away that this might be an old issue but it is still valid
3. If an issue is obviously already solved or invalid, close it right away with the proper explanation
4. If something is not yet labelled properly, add the proper label (D'UH!)
5. If something looks like a good first issue, add the label - there are doubts that there are any good first issues, but it's worth a try. This might help:
   • Add a bit more context for people to get started, if more experienced devs can give some pointers already, this might turn it into a doable task
   • if it is a bug, and it does not have a reproducer already: write one marked as XFAIL
6. Review issues labelled help wanted as basically we want help with everything. We should the ones that are suitable as good first issue and simply remove the label in the other issues.
7. Close issues which could feasibly be implemented as plugins outside this repo
8. (Maybe) prioritize the bugs (volunteers can fix or not fix wahtever they want, but maybe some pick higher prio bugs if they know they would have a higher impact?)

Other things to do

• Add automation to close feature requests which are more than one year old (this will likely be controversial, but the thinking here is, that if it is a good idea it will come up again later, or someone will dig it out of the closed issues - more ruthlessly closing feature requests that are not being picked up is the better alternative than clogging the backlog with potentially stale ideas).
• Improve the labels:
  • rename type:docs to topic:docs as this fits better there
  • rename type:infrastructure to topic:infrastructure same reason as above
  • rename type:feature-branch to type:feature - we do not really have feature-branches as such anymore
• Document the labels:
  • e.g. clarify the difference between type:enhancements (improving and existing feature) and type:feature adding a completely new feature
• Document how we (want to) handle issues and PRs (mainly what are the criteria for closing) with the aim of being helpful for contributors and people opening issues