"Catalog represents a store of bundles which OLM can use to install Operators config map update triggers registry pod rollout"

[timflannagan]

The "Catalog represents a store of bundles which OLM can use to install Operators [It] config map update triggers registry pod rollout" test case occasionally flakes. Link to an action run that exhibits this failing test: https://github.com/operator-framework/operator-lifecycle-manager/runs/4154202639?check_suite_focus=true.

• Failure [14.194 seconds]
Catalog represents a store of bundles which OLM can use to install Operators
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/catalog_e2e_test.go:35
  config map update triggers registry pod rollout [It]
  /home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/catalog_e2e_test.go:203

  Expected
      <[]v1.Pod | len:2, cap:2>: [
          {
              TypeMeta: {Kind: "", APIVersion: ""},
              ObjectMeta: {
                  Name: "mock-ocs-main-b6d4v-8wnkh",
                  GenerateName: "mock-ocs-main-b6d4v-",
                  Namespace: "operators",
                  SelfLink: "",
                  UID: "92196036-9762-4d65-a161-bc68b880c188",
                  ResourceVersion: "1112",
                  Generation: 0,
                  CreationTimestamp: {
                      Time: 2021-11-09T15:42:09Z,
                  },
                  DeletionTimestamp: nil,
                  DeletionGracePeriodSeconds: nil,
                  Labels: {
                      "olm.catalogSource": "mock-ocs-main-b6d4v",
                      "olm.configMapResourceVersion": "1013",
                      "olm.pod-spec-hash": "584dbfcb7b",
                  },
                  Annotations: nil,
...
  to have length 1

  /home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/catalog_e2e_test.go:254

  Full Stack Trace
  github.com/operator-framework/operator-lifecycle-manager/test/e2e.glob..func2.5()
  	/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/catalog_e2e_test.go:254 +0x1025
  github.com/operator-framework/operator-lifecycle-manager/test/e2e.TestEndToEnd(0xc000af7200)
  	/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/e2e_test.go:59 +0x296
  testing.tRunner(0xc000af7200, 0x343c030)
  	/opt/hostedtoolcache/go/1.16.10/x64/src/testing/testing.go:1193 +0xef
  created by testing.(*T).Run
  	/opt/hostedtoolcache/go/1.16.10/x64/src/testing/testing.go:1238 +0x2b3

[timflannagan]

Poking around that linked action's container logs, it's not immediately clear why that test case failed as we don't collect any additional test fixtures for debugging (e.g. configmaps in the operators namespace in this case). Looking at the test code, it's possible we have an issue with test pollution in the installplan e2e suite as well, where we create resources in shared namespace, typically in the operators namespace. This can lead to poor testing assumptions and failed assertions.

[timflannagan]

Ran into this error in #2438.

[akihikokuroda]

This #2444 may fix this issue.