Fix: e2e test for re-reconcile on catalog change is not valid

[joelanford]

The TestClusterExtensionInstallReResolvesWhenNewCatalog test is not valid because it sets up a cluster extenstion that fails to resolve. This results in an exponential backoff where retries will occur whether or not a new catalog appears.

To correct this test, we need to:

1. Create a catalog containing foo.v1.0.0, with a pollInterval set to 2s (or something very short)
2. Create a cluster extension for foo with no channel or version specifications
3. Wait until foo.v1.0.0 successfully installs.
4. Update the catalog to contain foo.v2.0.0 with an upgrade edge from v1, and push the update to the same tag.
5. After step (4) the following should happen all automatically:
   • Catalogd polls and sees a new SHA for the digest, pulls it and makes it available
   • Operator-controller sees the new catalog and re-reconciles all ClusterExtensions
   • The foo cluster extension re-resolves, finds foo.v2.0.0 and installs it.

[LalatenduMohanty]

/assign

[LalatenduMohanty]

/unassign

[yashoza19]

/assign

[yashoza19]

@joelanford I have some concern about this test. We are currently already creating a catalog and pushing it to a local registry during e2e setup by loading all the versions of a particular bundle here.
The catalog creation is already been done here before the e2e is ran. There seems to be no way to update the catalog after the initial catalog has been build and pushed to the registry.
I though about creating 2 different catalog but that would not fit this use case as we need to update the same catalog tag for this test case.
Please let me know your suggestions on how to proceed with this issue.

[joelanford]

During the e2e setup, we could push the catalog images with different tags, which would mean that step (4) becomes "move the tag from one image to another". And then maybe we could implement that directly in the test code?

cc @everettraven. Any other ideas?