flaky tests: dataproc Python 3.11

[holtskinner]

• Discovered in chore: Ran nox -s blacken on samples #10231 the following dataproc samples are failing in Python 3.11 tests with the errors listed below.

• update_cluster_test.test_update_cluster

• https://source.cloud.google.com/results/invocations/e135e63a-d1fd-417f-96c2-60970a7030f7/targets/github%2Fpython-docs-samples%2Fdataproc%2Fsnippets/tests

Traceback (most recent call last):
  File "/workspace/dataproc/snippets/update_cluster_test.py", line 107, in test_update_cluster
    assert response.status.state == ClusterStatus.State.RUNNING
AssertionError: assert <State.ERROR: 3> == <State.RUNNING: 2>
 +  where <State.ERROR: 3> = state: ERROR\nstate_start_time {\n  seconds: 1686695517\n  nanos: 758965000\n}\n.state
 +    where state: ERROR\nstate_start_time {\n  seconds: 1686695517\n  nanos: 758965000\n}\n = project_id: "python-docs-samples-tests"\ncluster_name: "py-cc-test-6b2c93b5-45e8-4be9-989d-3b5890d442b8"\nconfig {\n  config_bucket: "dataproc-28f7d075-c5ac-4938-a5ad-65eafd8032d3-us-central1"\n  temp_bucket: "dataproc-temp-us-central1-1012616486416-3kbzlm1c"\n  gce_cluster_config {\n    zone_uri: "https://www.googleapis.com/compute/v1/projects/python-docs-samples-tests/zones/us-central1-f"\n    network_uri: "https://www.googleapis.com/compute/v1/projects/python-docs-samples-tests/global/networks/default"\n    service_account_scopes: "https://www.googleapis.com/auth/bigquery"\n    service_account_scopes: "https://www.googleapis.com/auth/bigtable.admin.table"\n    service_account_scopes: "https://www.googleapis.com/auth/bigtable.data"\n    service_account_scopes: "https://www.googleapis.com/auth/cloud.useraccounts.readonly"\n    service_account_scopes: "https://www.googleapis.com/auth/devstorage.full_control"\n    service_account_scopes: "https://www.googleapis.com/auth/devstorage.read_write"\n    service_account_scopes: "https://www.googleapis.com/auth/logging.write"\n  }\n  master_config {\n    num_instances: 1\n    instance_names: "py-cc-test-6b2c93b5-45e8-4be9-989d-3b5890d442b8-m"\n    image_uri...   properties {\n      key: "distcp:mapreduce.map.memory.mb"\n      value: "768"\n    }\n    properties {\n      key: "distcp:mapreduce.map.java.opts"\n      value: "-Xmx576m"\n    }\n    properties {\n      key: "core:hadoop.ssl.enabled.protocols"\n      value: "TLSv1,TLSv1.1,TLSv1.2"\n    }\n    properties {\n      key: "core:fs.gs.metadata.cache.enable"\n      value: "false"\n    }\n    properties {\n      key: "core:fs.gs.block.size"\n      value: "134217728"\n    }\n    properties {\n      key: "capacity-scheduler:yarn.scheduler.capacity.root.default.ordering-policy"\n      value: "fair"\n    }\n  }\n  endpoint_config {\n  }\n}\nlabels {\n  key: "goog-dataproc-location"\n  value: "us-central1"\n}\nlabels {\n  key: "goog-dataproc-cluster-uuid"\n  value: "b9757788-8742-4e45-ab2f-0353992b94e4"\n}\nlabels {\n  key: "goog-dataproc-cluster-name"\n  value: "py-cc-test-6b2c93b5-45e8-4be9-989d-3b5890d442b8"\n}\nlabels {\n  key: "goog-dataproc-autozone"\n  value: "enabled"\n}\nstatus {\n  state: ERROR\n  state_start_time {\n    seconds: 1686695517\n    nanos: 758965000\n  }\n}\nstatus_history {\n  state: CREATING\n  state_start_time {\n    seconds: 1686695515\n    nanos: 155519000\n  }\n}\ncluster_uuid: "b9757788-8742-4e45-ab2f-0353992b94e4"\n.status
 +  and   <State.RUNNING: 2> = <enum 'State'>.RUNNING
 +    where <enum 'State'> = ClusterStatus.State

• submit_job_test.test_submit_job

Traceback (most recent call last):
  File "/workspace/dataproc/snippets/submit_job_test.py", line 113, in test_submit_job
    assert response.status.state == dataproc.ClusterStatus.State.RUNNING
AssertionError: assert <State.ERROR: 3> == <State.RUNNING: 2>
 +  where <State.ERROR: 3> = state: ERROR\nstate_start_time {\n  seconds: 1686695517\n  nanos: 53576000\n}\n.state
 +    where state: ERROR\nstate_start_time {\n  seconds: 1686695517\n  nanos: 53576000\n}\n = project_id: "python-docs-samples-tests"\ncluster_name: "py-sj-test-42a91083-9128-4b28-babf-c30d3a2f8532"\nconfig {\n  config_bucket: "dataproc-28f7d075-c5ac-4938-a5ad-65eafd8032d3-us-central1"\n  temp_bucket: "dataproc-temp-us-central1-1012616486416-3kbzlm1c"\n  gce_cluster_config {\n    zone_uri: "https://www.googleapis.com/compute/v1/projects/python-docs-samples-tests/zones/us-central1-f"\n    network_uri: "https://www.googleapis.com/compute/v1/projects/python-docs-samples-tests/global/networks/default"\n    service_account_scopes: "https://www.googleapis.com/auth/bigquery"\n    service_account_scopes: "https://www.googleapis.com/auth/bigtable.admin.table"\n    service_account_scopes: "https://www.googleapis.com/auth/bigtable.data"\n    service_account_scopes: "https://www.googleapis.com/auth/cloud.useraccounts.readonly"\n    service_account_scopes: "https://www.googleapis.com/auth/devstorage.full_control"\n    service_account_scopes: "https://www.googleapis.com/auth/devstorage.read_write"\n    service_account_scopes: "https://www.googleapis.com/auth/logging.write"\n  }\n  master_config {\n    num_instances: 1\n    instance_names: "py-sj-test-42a91083-9128-4b28-babf-c30d3a2f8532-m"\n    image_uri...    properties {\n      key: "distcp:mapreduce.map.memory.mb"\n      value: "768"\n    }\n    properties {\n      key: "distcp:mapreduce.map.java.opts"\n      value: "-Xmx576m"\n    }\n    properties {\n      key: "core:hadoop.ssl.enabled.protocols"\n      value: "TLSv1,TLSv1.1,TLSv1.2"\n    }\n    properties {\n      key: "core:fs.gs.metadata.cache.enable"\n      value: "false"\n    }\n    properties {\n      key: "core:fs.gs.block.size"\n      value: "134217728"\n    }\n    properties {\n      key: "capacity-scheduler:yarn.scheduler.capacity.root.default.ordering-policy"\n      value: "fair"\n    }\n  }\n  endpoint_config {\n  }\n}\nlabels {\n  key: "goog-dataproc-location"\n  value: "us-central1"\n}\nlabels {\n  key: "goog-dataproc-cluster-uuid"\n  value: "5c509dfa-2e70-414c-b0de-3bb15e2ff1c3"\n}\nlabels {\n  key: "goog-dataproc-cluster-name"\n  value: "py-sj-test-42a91083-9128-4b28-babf-c30d3a2f8532"\n}\nlabels {\n  key: "goog-dataproc-autozone"\n  value: "enabled"\n}\nstatus {\n  state: ERROR\n  state_start_time {\n    seconds: 1686695517\n    nanos: 53576000\n  }\n}\nstatus_history {\n  state: CREATING\n  state_start_time {\n    seconds: 1686695514\n    nanos: 396972000\n  }\n}\ncluster_uuid: "5c509dfa-2e70-414c-b0de-3bb15e2ff1c3"\n.status
 +  and   <State.RUNNING: 2> = <enum 'State'>.RUNNING
 +    where <enum 'State'> = <class 'google.cloud.dataproc_v1.types.clusters.ClusterStatus'>.State
 +      where <class 'google.cloud.dataproc_v1.types.clusters.ClusterStatus'> = dataproc.ClusterStatus

[leahecole]

I believe this is a duplicate of known issues #9445 and #9513 - and they are related to internal bug 285632121