From e29f381cb2022ffe6c5790a6ddff02b6bf0a5b7f Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Mon, 14 Aug 2023 13:44:50 -0700 Subject: [PATCH 01/15] DRIVERS-2578 Drivers use polling SDAM on AWS Lambda Disable streaming SDAM by default on AWS Lambda and similar FaaS platforms. Introduce the sdamMode=stream/poll/auto URI option. --- .../faas-automated-testing.rst | 5 + .../server-monitoring.rst | 92 +++++++++++-- .../tests/unified/sdamMode.json | 129 ++++++++++++++++++ .../tests/unified/sdamMode.yml | 67 +++++++++ source/uri-options/tests/sdam-options.json | 46 +++++++ source/uri-options/tests/sdam-options.yml | 35 +++++ source/uri-options/uri-options.rst | 11 +- 7 files changed, 374 insertions(+), 11 deletions(-) create mode 100644 source/server-discovery-and-monitoring/tests/unified/sdamMode.json create mode 100644 source/server-discovery-and-monitoring/tests/unified/sdamMode.yml create mode 100644 source/uri-options/tests/sdam-options.json create mode 100644 source/uri-options/tests/sdam-options.yml diff --git a/source/faas-automated-testing/faas-automated-testing.rst b/source/faas-automated-testing/faas-automated-testing.rst index df5c3a7408..ef9be1a0b2 100644 --- a/source/faas-automated-testing/faas-automated-testing.rst +++ b/source/faas-automated-testing/faas-automated-testing.rst @@ -243,6 +243,8 @@ the function implementation the driver MUST: - Drivers MUST record the durations and counts of the heartbeats, the durations of the commands, as well as keep track of the number of open connections, and report this information in the function response as JSON. +- Drivers MUST assert no ServerHeartbeat events contain the ``awaited=True`` flag to + confirm that the streaming protocol is disabled (`DRIVERS-2578`_). Running in Continuous Integration @@ -368,6 +370,9 @@ Description of the behaviour of run-deployed-lambda-aws-tests.sh: Changelog ========= +:2023-08-21: Drivers MUST assert that the streaming protocol is disabled in the Lambda function. :2023-08-17: Fixed URI typo, added host note, increase assume role duration. :2023-06-22: Updated evergreen configuration to use task groups. :2023-04-14: Added list of supported variants, added additional template config. + +.. _DRIVERS-2578: https://jira.mongodb.org/browse/DRIVERS-2578 diff --git a/source/server-discovery-and-monitoring/server-monitoring.rst b/source/server-discovery-and-monitoring/server-monitoring.rst index 940366a9a7..2a94bdbcb5 100644 --- a/source/server-discovery-and-monitoring/server-monitoring.rst +++ b/source/server-discovery-and-monitoring/server-monitoring.rst @@ -91,6 +91,28 @@ Round trip time. The client's measurement of the duration of one hello or legacy The RTT is used to support `localThresholdMS`_ from the Server Selection spec and `timeoutMS`_ from the `Client Side Operations Timeout Spec`_. +FaaS +```` + +A Function-as-a-Service (FaaS) environment like AWS Lambda. + +sdamMode +```````` + +The sdamMode option configures which server monitoring protocol to use. Valid modes are +"stream", "poll", or "auto". The default value MUST be "auto": + +- With "stream" mode, the client MUST use the streaming protocol when the server supports + it or fall back to the polling protocol otherwise. +- With "poll" mode, the client MUST use the polling protocol. +- With "auto" mode, the client MUST behave the same as "poll" mode when running on a FaaS + platform or the same as "stream" mode otherwise. The client detects that it's + running on a FaaS platform via the same rules for generating the ``client.env`` + handshake metadata field in the `MongoDB Handshake spec`_. + +Multi-threaded or asynchronous drivers MUST implement this option. +See `Why disable the streaming protocol on FaaS platforms like AWS Lambda?`_ and +`Why introduce a knob for sdamMode?`_ Monitoring '''''''''' @@ -203,7 +225,7 @@ Clients use the streaming protocol when supported ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When a monitor discovers that the server supports the streamable hello or legacy hello -command, it MUST use the `streaming protocol`_. +command and the client does not have `streaming disabled`_, it MUST use the `streaming protocol`_. Single-threaded monitoring `````````````````````````` @@ -491,6 +513,22 @@ connected to a server that supports the awaitable hello or legacy hello commands This protocol requires an extra thread and an extra socket for each monitor to perform RTT calculations. +.. _streaming is disabled: + +Streaming disabled +`````````````````` + +The streaming protocol MUST be disabled when either: + +- the client is configured with sdamMode=poll, or +- the client is configured with sdamMode=auto and a FaaS platform is detected, or +- the server does not support streaming (eg MongoDB <4.4). + +When the streaming protocol is disabled the client MUST use the `polling protocol`_ +and MUST NOT start an extra thread or connection for `Measuring RTT`_. + +See `Why disable the streaming protocol on FaaS platforms like AWS Lambda?`_. + Streaming hello or legacy hello ``````````````````````````````` @@ -584,8 +622,8 @@ current monitoring connection. (See `Drivers cancel in-progress monitor checks`_ Polling Protocol '''''''''''''''' -The polling protocol is used to monitor MongoDB <= 4.4 servers. The client -`checks`_ a server with a hello or legacy hello command and then sleeps for +The polling protocol is used to monitor MongoDB <= 4.4 servers or when `streaming is disabled`_. +The client `checks`_ a server with a hello or legacy hello command and then sleeps for heartbeatFrequencyMS before running another check. Marking the connection pool as ready (CMAP only) @@ -661,6 +699,12 @@ The event API here is assumed to be like the standard `Python Event heartbeatFrequencyMS = heartbeatFrequencyMS minHeartbeatFrequencyMS = 500 stableApi = stableApi + if sdamMode == "stream": + streamingEnabled = True + elif sdamMode == "poll": + streamingEnabled = False + else: # sdamMode == "auto" + streamingEnabled = not isFaas() # Internal Monitor state: connection = Null @@ -671,8 +715,6 @@ The event API here is assumed to be like the standard `Python Event rttMonitor = RttMonitor(serverAddress, stableApi) def run(): - # Start the RttMonitor. - rttMonitor.run() while this monitor is not stopped: previousDescription = description try: @@ -700,7 +742,10 @@ The event API here is assumed to be like the standard `Python Event serverSupportsStreaming = description.type != Unknown and description.topologyVersion != Null connectionIsStreaming = connection != Null and connection.moreToCome transitionedWithNetworkError = isNetworkError(description.error) and previousDescription.type != Unknown - if serverSupportsStreaming or connectionIsStreaming or transitionedWithNetworkError: + if streamingEnabled and serverSupportsStreaming and not rttMonitor.started: + # Start the RttMonitor. + rttMonitor.run() + if (streamingEnabled and (serverSupportsStreaming or connectionIsStreaming)) or transitionedWithNetworkError: continue wait() @@ -733,13 +778,13 @@ The event API here is assumed to be like the standard `Python Event response = connection.handshakeResponse elif connection.moreToCome: response = read next helloCommand exhaust response - elif previousDescription.topologyVersion: + elif streamingEnabled and previousDescription.topologyVersion: # Initiate streaming hello or legacy hello if connectTimeoutMS != 0: set connection timeout to connectTimeoutMS+heartbeatFrequencyMS response = call {helloCommand: 1, helloOk: True, topologyVersion: previousDescription.topologyVersion, maxAwaitTimeMS: heartbeatFrequencyMS} else: - # The server does not support topologyVersion. + # The server does not support topologyVersion or streamingEnabled=False. response = call {helloCommand: 1, helloOk: True} # If the server supports hello, then response.helloOk will be true @@ -1140,6 +1185,32 @@ the "awaited" field on server heartbeat events so that applications can differentiate a slow heartbeat in the polling protocol from a normal awaitable hello or legacy hello heartbeat in the new protocol. +Why disable the streaming protocol on FaaS platforms like AWS Lambda? +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +The streaming protocol requires an extra connection and thread per monitored +server which is expensive on platforms like AWS Lambda. The extra connection +is particularly inefficient when thousands of AWS instances and thus +thousands of clients are used. + +Additionally, the streaming protocol relies on the assumption that the client +can read the server's heartbeat responses in a timely manner, otherwise the +client will be acting on stale information. In many FaaS platforms, like AWS +Lambda, host applications will be suspended and resumed many minutes later. +This behavior causes a build up of heartbeat responses and the client can end +up spending a long time in a catch up phase processing outdated responses. +This problem was discovered in `DRIVERS-2246`_. + +We decided to make polling the default behavior when running on FaaS platforms +like AWS Lambda to improve scalability, performance, and reliability. + +Why introduce a knob for sdamMode? +'''''''''''''''''''''''''''''''''' + +The sdamMode knob provides an workaround in cases where the polling +protocol would be a better choice but the driver is not running on a FaaS +platform. It also provides a workaround in case the FaaS detection +logic becomes outdated or inaccurate. Changelog --------- @@ -1159,6 +1230,7 @@ Changelog :2022-04-05: Preemptively cancel in progress operations when SDAM heartbeats timeout. :2022-10-05: Remove spec front matter reformat changelog. :2022-11-17: Add minimum RTT tracking and remove 90th percentile RTT. +:2023-08-21: Add sdamMode and default to the Polling Protocol on FaaS. ---- @@ -1183,4 +1255,6 @@ Changelog .. _Why synchronize clearing a server's pool with updating the topology?: server-discovery-and-monitoring.rst#why-synchronize-clearing-a-server-s-pool-with-updating-the-topology? .. _Client Side Operations Timeout Spec: /source/client-side-operations-timeout/client-side-operations-timeout.rst .. _timeoutMS: /source/client-side-operations-timeout/client-side-operations-timeout.rst#timeoutMS -.. _Why does the pool need to support closing in use connections as part of its clear logic?: /source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst#Why-does-the-pool-need-to-support-closing-in-use-connections-as-part-of-its-clear-logic? \ No newline at end of file +.. _Why does the pool need to support closing in use connections as part of its clear logic?: /source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst#Why-does-the-pool-need-to-support-closing-in-use-connections-as-part-of-its-clear-logic? +.. _DRIVERS-2246: https://jira.mongodb.org/browse/DRIVERS-2246 +.. _MongoDB Handshake spec: /source/mongodb-handshake/handshake.rst#client-env diff --git a/source/server-discovery-and-monitoring/tests/unified/sdamMode.json b/source/server-discovery-and-monitoring/tests/unified/sdamMode.json new file mode 100644 index 0000000000..0df841788d --- /dev/null +++ b/source/server-discovery-and-monitoring/tests/unified/sdamMode.json @@ -0,0 +1,129 @@ +{ + "description": "sdamMode", + "schemaVersion": "1.3", + "tests": [ + { + "description": "connect with sdamMode=auto", + "operations": [ + { + "name": "createEntities", + "object": "testRunner", + "arguments": { + "entities": [ + { + "client": { + "id": "client0", + "uriOptions": { + "sdamMode": "auto" + } + } + }, + { + "database": { + "id": "dbSdamModeAuto", + "client": "client0", + "databaseName": "sdam-tests" + } + } + ] + } + }, + { + "name": "runCommand", + "object": "dbSdamModeAuto", + "arguments": { + "commandName": "ping", + "command": { + "ping": 1 + } + }, + "expectResult": { + "ok": 1 + } + } + ] + }, + { + "description": "connect with sdamMode=stream", + "operations": [ + { + "name": "createEntities", + "object": "testRunner", + "arguments": { + "entities": [ + { + "client": { + "id": "client1", + "uriOptions": { + "sdamMode": "stream" + } + } + }, + { + "database": { + "id": "dbSdamModeStream", + "client": "client1", + "databaseName": "sdam-tests" + } + } + ] + } + }, + { + "name": "runCommand", + "object": "dbSdamModeStream", + "arguments": { + "commandName": "ping", + "command": { + "ping": 1 + } + }, + "expectResult": { + "ok": 1 + } + } + ] + }, + { + "description": "connect with sdamMode=poll", + "operations": [ + { + "name": "createEntities", + "object": "testRunner", + "arguments": { + "entities": [ + { + "client": { + "id": "client2", + "uriOptions": { + "sdamMode": "poll" + } + } + }, + { + "database": { + "id": "dbSdamModePoll", + "client": "client2", + "databaseName": "sdam-tests" + } + } + ] + } + }, + { + "name": "runCommand", + "object": "dbSdamModePoll", + "arguments": { + "commandName": "ping", + "command": { + "ping": 1 + } + }, + "expectResult": { + "ok": 1 + } + } + ] + } + ] +} diff --git a/source/server-discovery-and-monitoring/tests/unified/sdamMode.yml b/source/server-discovery-and-monitoring/tests/unified/sdamMode.yml new file mode 100644 index 0000000000..84d2a33648 --- /dev/null +++ b/source/server-discovery-and-monitoring/tests/unified/sdamMode.yml @@ -0,0 +1,67 @@ +description: sdamMode + +schemaVersion: "1.3" + +tests: + - description: "connect with sdamMode=auto" + operations: + - name: createEntities + object: testRunner + arguments: + entities: + - client: + id: &client0 client0 + uriOptions: + sdamMode: "auto" + - database: + id: &dbSdamModeAuto dbSdamModeAuto + client: *client0 + databaseName: sdam-tests + - name: runCommand + object: *dbSdamModeAuto + arguments: + commandName: ping + command: { ping: 1 } + expectResult: { ok: 1 } + + - description: "connect with sdamMode=stream" + operations: + - name: createEntities + object: testRunner + arguments: + entities: + - client: + id: &client1 client1 + uriOptions: + sdamMode: "stream" + - database: + id: &dbSdamModeStream dbSdamModeStream + client: *client1 + databaseName: sdam-tests + - name: runCommand + object: *dbSdamModeStream + arguments: + commandName: ping + command: { ping: 1 } + expectResult: { ok: 1 } + + - description: "connect with sdamMode=poll" + operations: + - name: createEntities + object: testRunner + arguments: + entities: + - client: + id: &client2 client2 + uriOptions: + sdamMode: "poll" + - database: + id: &dbSdamModePoll dbSdamModePoll + client: *client2 + databaseName: sdam-tests + - name: runCommand + object: *dbSdamModePoll + arguments: + commandName: ping + command: { ping: 1 } + expectResult: { ok: 1 } diff --git a/source/uri-options/tests/sdam-options.json b/source/uri-options/tests/sdam-options.json new file mode 100644 index 0000000000..afc39ca3f5 --- /dev/null +++ b/source/uri-options/tests/sdam-options.json @@ -0,0 +1,46 @@ +{ + "tests": [ + { + "description": "sdamMode=auto", + "uri": "mongodb://example.com/?sdamMode=auto", + "valid": true, + "warning": false, + "hosts": null, + "auth": null, + "options": { + "sdamMode": "auto" + } + }, + { + "description": "sdamMode=stream", + "uri": "mongodb://example.com/?sdamMode=stream", + "valid": true, + "warning": false, + "hosts": null, + "auth": null, + "options": { + "sdamMode": "stream" + } + }, + { + "description": "sdamMode=poll", + "uri": "mongodb://example.com/?sdamMode=poll", + "valid": true, + "warning": false, + "hosts": null, + "auth": null, + "options": { + "sdamMode": "poll" + } + }, + { + "description": "invalid sdamMode", + "uri": "mongodb://example.com/?sdamMode=invalid", + "valid": true, + "warning": true, + "hosts": null, + "auth": null, + "options": {} + } + ] +} diff --git a/source/uri-options/tests/sdam-options.yml b/source/uri-options/tests/sdam-options.yml new file mode 100644 index 0000000000..735bf0aed7 --- /dev/null +++ b/source/uri-options/tests/sdam-options.yml @@ -0,0 +1,35 @@ +tests: + - description: "sdamMode=auto" + uri: "mongodb://example.com/?sdamMode=auto" + valid: true + warning: false + hosts: ~ + auth: ~ + options: + sdamMode: "auto" + + - description: "sdamMode=stream" + uri: "mongodb://example.com/?sdamMode=stream" + valid: true + warning: false + hosts: ~ + auth: ~ + options: + sdamMode: "stream" + + - description: "sdamMode=poll" + uri: "mongodb://example.com/?sdamMode=poll" + valid: true + warning: false + hosts: ~ + auth: ~ + options: + sdamMode: "poll" + + - description: "invalid sdamMode" + uri: "mongodb://example.com/?sdamMode=invalid" + valid: true + warning: true + hosts: ~ + auth: ~ + options: {} diff --git a/source/uri-options/uri-options.rst b/source/uri-options/uri-options.rst index 035e7ddec5..3f4589b4a5 100644 --- a/source/uri-options/uri-options.rst +++ b/source/uri-options/uri-options.rst @@ -282,17 +282,23 @@ pertaining to URI options apply here. - The name of the replica set to connect to * - retryReads - - "true" or "false + - "true" or "false" - defined in `retryable reads spec `_ - no - Enables retryable reads on server 3.6+ * - retryWrites - - "true" or "false + - "true" or "false" - defined in `retryable writes spec `_ - no - Enables retryable writes on server 3.6+ + * - sdamMode + - "stream", "poll", or "auto" + - defined in `SDAM spec `__ + - required for multi-threaded or asynchronous drivers + - Configures which server monitoring protocol to use. + * - serverSelectionTimeoutMS - positive integer; a driver may also accept 0 to be used for a special case, provided that it documents the meaning - defined in `server selection spec `__ @@ -525,6 +531,7 @@ this specification MUST be updated to reflect those changes. Changelog --------- +:2023-08-21: Add sdamMode option. :2022-10-05: Remove spec front matter and reformat changelog. :2022-01-19: Add the timeoutMS option and deprecate some existing timeout options :2021-12-14: Add SOCKS5 options From 2445f626ae67b4e8ce582165f152984a1d4987cf Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Tue, 22 Aug 2023 12:42:22 -0700 Subject: [PATCH 02/15] DRIVERS-2578 Rename sdamMode to serverMonitoringMode --- .../server-monitoring.rst | 26 +++++++++---------- .../tests/unified/sdamMode.json | 14 +++++----- .../tests/unified/sdamMode.yml | 14 +++++----- source/uri-options/tests/sdam-options.json | 22 ++++++++-------- source/uri-options/tests/sdam-options.yml | 22 ++++++++-------- source/uri-options/uri-options.rst | 6 ++--- 6 files changed, 52 insertions(+), 52 deletions(-) diff --git a/source/server-discovery-and-monitoring/server-monitoring.rst b/source/server-discovery-and-monitoring/server-monitoring.rst index 2a94bdbcb5..28a2deb761 100644 --- a/source/server-discovery-and-monitoring/server-monitoring.rst +++ b/source/server-discovery-and-monitoring/server-monitoring.rst @@ -96,10 +96,10 @@ FaaS A Function-as-a-Service (FaaS) environment like AWS Lambda. -sdamMode -```````` +serverMonitoringMode +```````````````````` -The sdamMode option configures which server monitoring protocol to use. Valid modes are +The serverMonitoringMode option configures which server monitoring protocol to use. Valid modes are "stream", "poll", or "auto". The default value MUST be "auto": - With "stream" mode, the client MUST use the streaming protocol when the server supports @@ -112,7 +112,7 @@ The sdamMode option configures which server monitoring protocol to use. Valid mo Multi-threaded or asynchronous drivers MUST implement this option. See `Why disable the streaming protocol on FaaS platforms like AWS Lambda?`_ and -`Why introduce a knob for sdamMode?`_ +`Why introduce a knob for serverMonitoringMode?`_ Monitoring '''''''''' @@ -520,8 +520,8 @@ Streaming disabled The streaming protocol MUST be disabled when either: -- the client is configured with sdamMode=poll, or -- the client is configured with sdamMode=auto and a FaaS platform is detected, or +- the client is configured with serverMonitoringMode=poll, or +- the client is configured with serverMonitoringMode=auto and a FaaS platform is detected, or - the server does not support streaming (eg MongoDB <4.4). When the streaming protocol is disabled the client MUST use the `polling protocol`_ @@ -699,11 +699,11 @@ The event API here is assumed to be like the standard `Python Event heartbeatFrequencyMS = heartbeatFrequencyMS minHeartbeatFrequencyMS = 500 stableApi = stableApi - if sdamMode == "stream": + if serverMonitoringMode == "stream": streamingEnabled = True - elif sdamMode == "poll": + elif serverMonitoringMode == "poll": streamingEnabled = False - else: # sdamMode == "auto" + else: # serverMonitoringMode == "auto" streamingEnabled = not isFaas() # Internal Monitor state: @@ -1204,10 +1204,10 @@ This problem was discovered in `DRIVERS-2246`_. We decided to make polling the default behavior when running on FaaS platforms like AWS Lambda to improve scalability, performance, and reliability. -Why introduce a knob for sdamMode? -'''''''''''''''''''''''''''''''''' +Why introduce a knob for serverMonitoringMode? +'''''''''''''''''''''''''''''''''''''''''''''' -The sdamMode knob provides an workaround in cases where the polling +The serverMonitoringMode knob provides an workaround in cases where the polling protocol would be a better choice but the driver is not running on a FaaS platform. It also provides a workaround in case the FaaS detection logic becomes outdated or inaccurate. @@ -1230,7 +1230,7 @@ Changelog :2022-04-05: Preemptively cancel in progress operations when SDAM heartbeats timeout. :2022-10-05: Remove spec front matter reformat changelog. :2022-11-17: Add minimum RTT tracking and remove 90th percentile RTT. -:2023-08-21: Add sdamMode and default to the Polling Protocol on FaaS. +:2023-08-21: Add serverMonitoringMode and default to the Polling Protocol on FaaS. ---- diff --git a/source/server-discovery-and-monitoring/tests/unified/sdamMode.json b/source/server-discovery-and-monitoring/tests/unified/sdamMode.json index 0df841788d..520635ba2c 100644 --- a/source/server-discovery-and-monitoring/tests/unified/sdamMode.json +++ b/source/server-discovery-and-monitoring/tests/unified/sdamMode.json @@ -1,9 +1,9 @@ { - "description": "sdamMode", + "description": "serverMonitoringMode", "schemaVersion": "1.3", "tests": [ { - "description": "connect with sdamMode=auto", + "description": "connect with serverMonitoringMode=auto", "operations": [ { "name": "createEntities", @@ -14,7 +14,7 @@ "client": { "id": "client0", "uriOptions": { - "sdamMode": "auto" + "serverMonitoringMode": "auto" } } }, @@ -44,7 +44,7 @@ ] }, { - "description": "connect with sdamMode=stream", + "description": "connect with serverMonitoringMode=stream", "operations": [ { "name": "createEntities", @@ -55,7 +55,7 @@ "client": { "id": "client1", "uriOptions": { - "sdamMode": "stream" + "serverMonitoringMode": "stream" } } }, @@ -85,7 +85,7 @@ ] }, { - "description": "connect with sdamMode=poll", + "description": "connect with serverMonitoringMode=poll", "operations": [ { "name": "createEntities", @@ -96,7 +96,7 @@ "client": { "id": "client2", "uriOptions": { - "sdamMode": "poll" + "serverMonitoringMode": "poll" } } }, diff --git a/source/server-discovery-and-monitoring/tests/unified/sdamMode.yml b/source/server-discovery-and-monitoring/tests/unified/sdamMode.yml index 84d2a33648..bb2b2053b7 100644 --- a/source/server-discovery-and-monitoring/tests/unified/sdamMode.yml +++ b/source/server-discovery-and-monitoring/tests/unified/sdamMode.yml @@ -1,9 +1,9 @@ -description: sdamMode +description: serverMonitoringMode schemaVersion: "1.3" tests: - - description: "connect with sdamMode=auto" + - description: "connect with serverMonitoringMode=auto" operations: - name: createEntities object: testRunner @@ -12,7 +12,7 @@ tests: - client: id: &client0 client0 uriOptions: - sdamMode: "auto" + serverMonitoringMode: "auto" - database: id: &dbSdamModeAuto dbSdamModeAuto client: *client0 @@ -24,7 +24,7 @@ tests: command: { ping: 1 } expectResult: { ok: 1 } - - description: "connect with sdamMode=stream" + - description: "connect with serverMonitoringMode=stream" operations: - name: createEntities object: testRunner @@ -33,7 +33,7 @@ tests: - client: id: &client1 client1 uriOptions: - sdamMode: "stream" + serverMonitoringMode: "stream" - database: id: &dbSdamModeStream dbSdamModeStream client: *client1 @@ -45,7 +45,7 @@ tests: command: { ping: 1 } expectResult: { ok: 1 } - - description: "connect with sdamMode=poll" + - description: "connect with serverMonitoringMode=poll" operations: - name: createEntities object: testRunner @@ -54,7 +54,7 @@ tests: - client: id: &client2 client2 uriOptions: - sdamMode: "poll" + serverMonitoringMode: "poll" - database: id: &dbSdamModePoll dbSdamModePoll client: *client2 diff --git a/source/uri-options/tests/sdam-options.json b/source/uri-options/tests/sdam-options.json index afc39ca3f5..673f5607ee 100644 --- a/source/uri-options/tests/sdam-options.json +++ b/source/uri-options/tests/sdam-options.json @@ -1,41 +1,41 @@ { "tests": [ { - "description": "sdamMode=auto", - "uri": "mongodb://example.com/?sdamMode=auto", + "description": "serverMonitoringMode=auto", + "uri": "mongodb://example.com/?serverMonitoringMode=auto", "valid": true, "warning": false, "hosts": null, "auth": null, "options": { - "sdamMode": "auto" + "serverMonitoringMode": "auto" } }, { - "description": "sdamMode=stream", - "uri": "mongodb://example.com/?sdamMode=stream", + "description": "serverMonitoringMode=stream", + "uri": "mongodb://example.com/?serverMonitoringMode=stream", "valid": true, "warning": false, "hosts": null, "auth": null, "options": { - "sdamMode": "stream" + "serverMonitoringMode": "stream" } }, { - "description": "sdamMode=poll", - "uri": "mongodb://example.com/?sdamMode=poll", + "description": "serverMonitoringMode=poll", + "uri": "mongodb://example.com/?serverMonitoringMode=poll", "valid": true, "warning": false, "hosts": null, "auth": null, "options": { - "sdamMode": "poll" + "serverMonitoringMode": "poll" } }, { - "description": "invalid sdamMode", - "uri": "mongodb://example.com/?sdamMode=invalid", + "description": "invalid serverMonitoringMode", + "uri": "mongodb://example.com/?serverMonitoringMode=invalid", "valid": true, "warning": true, "hosts": null, diff --git a/source/uri-options/tests/sdam-options.yml b/source/uri-options/tests/sdam-options.yml index 735bf0aed7..8f72ff4098 100644 --- a/source/uri-options/tests/sdam-options.yml +++ b/source/uri-options/tests/sdam-options.yml @@ -1,33 +1,33 @@ tests: - - description: "sdamMode=auto" - uri: "mongodb://example.com/?sdamMode=auto" + - description: "serverMonitoringMode=auto" + uri: "mongodb://example.com/?serverMonitoringMode=auto" valid: true warning: false hosts: ~ auth: ~ options: - sdamMode: "auto" + serverMonitoringMode: "auto" - - description: "sdamMode=stream" - uri: "mongodb://example.com/?sdamMode=stream" + - description: "serverMonitoringMode=stream" + uri: "mongodb://example.com/?serverMonitoringMode=stream" valid: true warning: false hosts: ~ auth: ~ options: - sdamMode: "stream" + serverMonitoringMode: "stream" - - description: "sdamMode=poll" - uri: "mongodb://example.com/?sdamMode=poll" + - description: "serverMonitoringMode=poll" + uri: "mongodb://example.com/?serverMonitoringMode=poll" valid: true warning: false hosts: ~ auth: ~ options: - sdamMode: "poll" + serverMonitoringMode: "poll" - - description: "invalid sdamMode" - uri: "mongodb://example.com/?sdamMode=invalid" + - description: "invalid serverMonitoringMode" + uri: "mongodb://example.com/?serverMonitoringMode=invalid" valid: true warning: true hosts: ~ diff --git a/source/uri-options/uri-options.rst b/source/uri-options/uri-options.rst index 3f4589b4a5..8f8f4f0375 100644 --- a/source/uri-options/uri-options.rst +++ b/source/uri-options/uri-options.rst @@ -293,9 +293,9 @@ pertaining to URI options apply here. - no - Enables retryable writes on server 3.6+ - * - sdamMode + * - serverMonitoringMode - "stream", "poll", or "auto" - - defined in `SDAM spec `__ + - defined in `SDAM spec `__ - required for multi-threaded or asynchronous drivers - Configures which server monitoring protocol to use. @@ -531,7 +531,7 @@ this specification MUST be updated to reflect those changes. Changelog --------- -:2023-08-21: Add sdamMode option. +:2023-08-21: Add serverMonitoringMode option. :2022-10-05: Remove spec front matter and reformat changelog. :2022-01-19: Add the timeoutMS option and deprecate some existing timeout options :2021-12-14: Add SOCKS5 options From e676bbbad971f9b42f6d1ca03b15739c4a7ac3ee Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Tue, 22 Aug 2023 12:56:51 -0700 Subject: [PATCH 03/15] DRIVERS-2578 Finish rename --- .../tests/unified/{sdamMode.json => serverMonitoringMode.json} | 0 .../tests/unified/{sdamMode.yml => serverMonitoringMode.yml} | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename source/server-discovery-and-monitoring/tests/unified/{sdamMode.json => serverMonitoringMode.json} (100%) rename source/server-discovery-and-monitoring/tests/unified/{sdamMode.yml => serverMonitoringMode.yml} (100%) diff --git a/source/server-discovery-and-monitoring/tests/unified/sdamMode.json b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json similarity index 100% rename from source/server-discovery-and-monitoring/tests/unified/sdamMode.json rename to source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json diff --git a/source/server-discovery-and-monitoring/tests/unified/sdamMode.yml b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml similarity index 100% rename from source/server-discovery-and-monitoring/tests/unified/sdamMode.yml rename to source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml From e749ada3ef77cbfe85171b1c5f296f9a89b4c1e4 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Fri, 8 Sep 2023 13:29:08 -0700 Subject: [PATCH 04/15] DRIVERS-2578 More sdamMode renaming --- .../tests/unified/serverMonitoringMode.json | 12 ++++++------ .../tests/unified/serverMonitoringMode.yml | 12 ++++++------ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json index 520635ba2c..e07cf1aca1 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json @@ -20,7 +20,7 @@ }, { "database": { - "id": "dbSdamModeAuto", + "id": "dbServerMonitoringModeAuto", "client": "client0", "databaseName": "sdam-tests" } @@ -30,7 +30,7 @@ }, { "name": "runCommand", - "object": "dbSdamModeAuto", + "object": "dbServerMonitoringModeAuto", "arguments": { "commandName": "ping", "command": { @@ -61,7 +61,7 @@ }, { "database": { - "id": "dbSdamModeStream", + "id": "dbServerMonitoringModeStream", "client": "client1", "databaseName": "sdam-tests" } @@ -71,7 +71,7 @@ }, { "name": "runCommand", - "object": "dbSdamModeStream", + "object": "dbServerMonitoringModeStream", "arguments": { "commandName": "ping", "command": { @@ -102,7 +102,7 @@ }, { "database": { - "id": "dbSdamModePoll", + "id": "dbServerMonitoringModePoll", "client": "client2", "databaseName": "sdam-tests" } @@ -112,7 +112,7 @@ }, { "name": "runCommand", - "object": "dbSdamModePoll", + "object": "dbServerMonitoringModePoll", "arguments": { "commandName": "ping", "command": { diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml index bb2b2053b7..66198e6433 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml @@ -14,11 +14,11 @@ tests: uriOptions: serverMonitoringMode: "auto" - database: - id: &dbSdamModeAuto dbSdamModeAuto + id: &dbServerMonitoringModeAuto dbServerMonitoringModeAuto client: *client0 databaseName: sdam-tests - name: runCommand - object: *dbSdamModeAuto + object: *dbServerMonitoringModeAuto arguments: commandName: ping command: { ping: 1 } @@ -35,11 +35,11 @@ tests: uriOptions: serverMonitoringMode: "stream" - database: - id: &dbSdamModeStream dbSdamModeStream + id: &dbServerMonitoringModeStream dbServerMonitoringModeStream client: *client1 databaseName: sdam-tests - name: runCommand - object: *dbSdamModeStream + object: *dbServerMonitoringModeStream arguments: commandName: ping command: { ping: 1 } @@ -56,11 +56,11 @@ tests: uriOptions: serverMonitoringMode: "poll" - database: - id: &dbSdamModePoll dbSdamModePoll + id: &dbServerMonitoringModePoll dbServerMonitoringModePoll client: *client2 databaseName: sdam-tests - name: runCommand - object: *dbSdamModePoll + object: *dbServerMonitoringModePoll arguments: commandName: ping command: { ping: 1 } From c580f0550a46c9eed6a251bd34119be88089f2c3 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Mon, 11 Sep 2023 15:01:36 -0700 Subject: [PATCH 05/15] DRIVERS-2578 Make invalid uri option an error not warning --- source/uri-options/tests/sdam-options.json | 6 +++--- source/uri-options/tests/sdam-options.yml | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/source/uri-options/tests/sdam-options.json b/source/uri-options/tests/sdam-options.json index 673f5607ee..9653f038e9 100644 --- a/source/uri-options/tests/sdam-options.json +++ b/source/uri-options/tests/sdam-options.json @@ -34,10 +34,10 @@ } }, { - "description": "invalid serverMonitoringMode", + "description": "serverMonitoringMode=invalid", "uri": "mongodb://example.com/?serverMonitoringMode=invalid", - "valid": true, - "warning": true, + "valid": false, + "warning": false, "hosts": null, "auth": null, "options": {} diff --git a/source/uri-options/tests/sdam-options.yml b/source/uri-options/tests/sdam-options.yml index 8f72ff4098..780dbc9497 100644 --- a/source/uri-options/tests/sdam-options.yml +++ b/source/uri-options/tests/sdam-options.yml @@ -26,10 +26,10 @@ tests: options: serverMonitoringMode: "poll" - - description: "invalid serverMonitoringMode" + - description: "serverMonitoringMode=invalid" uri: "mongodb://example.com/?serverMonitoringMode=invalid" - valid: true - warning: true + valid: false + warning: false hosts: ~ auth: ~ options: {} From fc6449c379a0ed7b73093ad358dd39427903e891 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Mon, 11 Sep 2023 15:02:29 -0700 Subject: [PATCH 06/15] Revert "DRIVERS-2578 Make invalid uri option an error not warning" This reverts commit 808e999556ba3e016d95b026733593ead04439cf. --- source/uri-options/tests/sdam-options.json | 6 +++--- source/uri-options/tests/sdam-options.yml | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/source/uri-options/tests/sdam-options.json b/source/uri-options/tests/sdam-options.json index 9653f038e9..673f5607ee 100644 --- a/source/uri-options/tests/sdam-options.json +++ b/source/uri-options/tests/sdam-options.json @@ -34,10 +34,10 @@ } }, { - "description": "serverMonitoringMode=invalid", + "description": "invalid serverMonitoringMode", "uri": "mongodb://example.com/?serverMonitoringMode=invalid", - "valid": false, - "warning": false, + "valid": true, + "warning": true, "hosts": null, "auth": null, "options": {} diff --git a/source/uri-options/tests/sdam-options.yml b/source/uri-options/tests/sdam-options.yml index 780dbc9497..8f72ff4098 100644 --- a/source/uri-options/tests/sdam-options.yml +++ b/source/uri-options/tests/sdam-options.yml @@ -26,10 +26,10 @@ tests: options: serverMonitoringMode: "poll" - - description: "serverMonitoringMode=invalid" + - description: "invalid serverMonitoringMode" uri: "mongodb://example.com/?serverMonitoringMode=invalid" - valid: false - warning: false + valid: true + warning: true hosts: ~ auth: ~ options: {} From 934990e743c9ea048996101e5e3bc4029b9a0ffc Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Thu, 21 Sep 2023 11:20:48 -0700 Subject: [PATCH 07/15] DRIVERS-2578 fix polling off by one typo --- source/server-discovery-and-monitoring/server-monitoring.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/server-discovery-and-monitoring/server-monitoring.rst b/source/server-discovery-and-monitoring/server-monitoring.rst index 28a2deb761..c794fbc9c4 100644 --- a/source/server-discovery-and-monitoring/server-monitoring.rst +++ b/source/server-discovery-and-monitoring/server-monitoring.rst @@ -522,7 +522,7 @@ The streaming protocol MUST be disabled when either: - the client is configured with serverMonitoringMode=poll, or - the client is configured with serverMonitoringMode=auto and a FaaS platform is detected, or -- the server does not support streaming (eg MongoDB <4.4). +- the server does not support streaming (eg MongoDB < 4.4). When the streaming protocol is disabled the client MUST use the `polling protocol`_ and MUST NOT start an extra thread or connection for `Measuring RTT`_. @@ -622,7 +622,7 @@ current monitoring connection. (See `Drivers cancel in-progress monitor checks`_ Polling Protocol '''''''''''''''' -The polling protocol is used to monitor MongoDB <= 4.4 servers or when `streaming is disabled`_. +The polling protocol is used to monitor MongoDB < 4.4 servers or when `streaming is disabled`_. The client `checks`_ a server with a hello or legacy hello command and then sleeps for heartbeatFrequencyMS before running another check. From 94b720692505c007378cc6a9f530d1f71e6a22e8 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Wed, 4 Oct 2023 16:25:41 -0700 Subject: [PATCH 08/15] DRIVERS-2578 UTF version 1.17 to add support for server heartbeat events --- .../tests/unified/serverMonitoringMode.json | 105 ++- .../tests/unified/serverMonitoringMode.yml | 54 +- source/unified-test-format/schema-1.17.json | 712 ++++++++++++++++++ .../unified-test-format.rst | 38 +- 4 files changed, 902 insertions(+), 7 deletions(-) create mode 100644 source/unified-test-format/schema-1.17.json diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json index e07cf1aca1..651fb4383e 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json @@ -1,6 +1,16 @@ { "description": "serverMonitoringMode", - "schemaVersion": "1.3", + "schemaVersion": "1.17", + "runOnRequirements": [ + { + "topologies": [ + "single", + "sharded", + "sharded-replicaset" + ], + "serverless": "forbid" + } + ], "tests": [ { "description": "connect with serverMonitoringMode=auto", @@ -15,7 +25,13 @@ "id": "client0", "uriOptions": { "serverMonitoringMode": "auto" - } + }, + "useMultipleMongoses": false, + "observeEvents": [ + "serverHeartbeatStartedEvent", + "serverHeartbeatSucceededEvent", + "serverHeartbeatFailedEvent" + ] } }, { @@ -41,6 +57,29 @@ "ok": 1 } } + ], + "expectEvents": [ + { + "client": "client0", + "eventType": "sdam", + "events": [ + { + "serverHeartbeatStartedEvent": { + "awaited": false + } + }, + { + "serverHeartbeatSucceededEvent": { + "awaited": false + } + }, + { + "serverHeartbeatStartedEvent": { + "awaited": true + } + } + ] + } ] }, { @@ -56,7 +95,13 @@ "id": "client1", "uriOptions": { "serverMonitoringMode": "stream" - } + }, + "useMultipleMongoses": false, + "observeEvents": [ + "serverHeartbeatStartedEvent", + "serverHeartbeatSucceededEvent", + "serverHeartbeatFailedEvent" + ] } }, { @@ -82,6 +127,29 @@ "ok": 1 } } + ], + "expectEvents": [ + { + "client": "client1", + "eventType": "sdam", + "events": [ + { + "serverHeartbeatStartedEvent": { + "awaited": false + } + }, + { + "serverHeartbeatSucceededEvent": { + "awaited": false + } + }, + { + "serverHeartbeatStartedEvent": { + "awaited": true + } + } + ] + } ] }, { @@ -97,7 +165,13 @@ "id": "client2", "uriOptions": { "serverMonitoringMode": "poll" - } + }, + "useMultipleMongoses": false, + "observeEvents": [ + "serverHeartbeatStartedEvent", + "serverHeartbeatSucceededEvent", + "serverHeartbeatFailedEvent" + ] } }, { @@ -123,6 +197,29 @@ "ok": 1 } } + ], + "expectEvents": [ + { + "client": "client2", + "eventType": "sdam", + "events": [ + { + "serverHeartbeatStartedEvent": { + "awaited": false + } + }, + { + "serverHeartbeatSucceededEvent": { + "awaited": false + } + }, + { + "serverHeartbeatStartedEvent": { + "awaited": false + } + } + ] + } ] } ] diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml index 66198e6433..c9cc31b78f 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml @@ -1,7 +1,12 @@ description: serverMonitoringMode -schemaVersion: "1.3" - +schemaVersion: "1.17" +# These tests cannot run on replica sets because the order of the expected +# SDAM events are non-deterministic when monitoring multiple servers. +# They also cannot run on Serverless or load balanced clusters where SDAM is disabled. +runOnRequirements: + - topologies: [single, sharded, sharded-replicaset] + serverless: forbid tests: - description: "connect with serverMonitoringMode=auto" operations: @@ -13,6 +18,11 @@ tests: id: &client0 client0 uriOptions: serverMonitoringMode: "auto" + useMultipleMongoses: false + observeEvents: + - serverHeartbeatStartedEvent + - serverHeartbeatSucceededEvent + - serverHeartbeatFailedEvent - database: id: &dbServerMonitoringModeAuto dbServerMonitoringModeAuto client: *client0 @@ -23,6 +33,16 @@ tests: commandName: ping command: { ping: 1 } expectResult: { ok: 1 } + expectEvents: + - client: *client0 + eventType: sdam + events: + - serverHeartbeatStartedEvent: + awaited: False + - serverHeartbeatSucceededEvent: + awaited: False + - serverHeartbeatStartedEvent: + awaited: True - description: "connect with serverMonitoringMode=stream" operations: @@ -34,6 +54,11 @@ tests: id: &client1 client1 uriOptions: serverMonitoringMode: "stream" + useMultipleMongoses: false + observeEvents: + - serverHeartbeatStartedEvent + - serverHeartbeatSucceededEvent + - serverHeartbeatFailedEvent - database: id: &dbServerMonitoringModeStream dbServerMonitoringModeStream client: *client1 @@ -44,6 +69,16 @@ tests: commandName: ping command: { ping: 1 } expectResult: { ok: 1 } + expectEvents: + - client: *client1 + eventType: sdam + events: + - serverHeartbeatStartedEvent: + awaited: False + - serverHeartbeatSucceededEvent: + awaited: False + - serverHeartbeatStartedEvent: + awaited: True - description: "connect with serverMonitoringMode=poll" operations: @@ -55,6 +90,11 @@ tests: id: &client2 client2 uriOptions: serverMonitoringMode: "poll" + useMultipleMongoses: false + observeEvents: + - serverHeartbeatStartedEvent + - serverHeartbeatSucceededEvent + - serverHeartbeatFailedEvent - database: id: &dbServerMonitoringModePoll dbServerMonitoringModePoll client: *client2 @@ -65,3 +105,13 @@ tests: commandName: ping command: { ping: 1 } expectResult: { ok: 1 } + expectEvents: + - client: *client2 + eventType: sdam + events: + - serverHeartbeatStartedEvent: + awaited: False + - serverHeartbeatSucceededEvent: + awaited: False + - serverHeartbeatStartedEvent: + awaited: False diff --git a/source/unified-test-format/schema-1.17.json b/source/unified-test-format/schema-1.17.json new file mode 100644 index 0000000000..7d321889a4 --- /dev/null +++ b/source/unified-test-format/schema-1.17.json @@ -0,0 +1,712 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + + "title": "Unified Test Format", + "type": "object", + "additionalProperties": false, + "required": ["description", "schemaVersion", "tests"], + "properties": { + "description": { "type": "string" }, + "schemaVersion": { "$ref": "#/definitions/version" }, + "runOnRequirements": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/runOnRequirement" } + }, + "createEntities": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/entity" } + }, + "initialData": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/collectionData" } + }, + "tests": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/test" } + }, + "_yamlAnchors": { + "type": "object", + "additionalProperties": true + } + }, + + "definitions": { + "version": { + "type": "string", + "pattern": "^[0-9]+(\\.[0-9]+){1,2}$" + }, + + "runOnRequirement": { + "type": "object", + "additionalProperties": false, + "minProperties": 1, + "properties": { + "maxServerVersion": { "$ref": "#/definitions/version" }, + "minServerVersion": { "$ref": "#/definitions/version" }, + "topologies": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": ["single", "replicaset", "sharded", "sharded-replicaset", "load-balanced"] + } + }, + "serverless": { + "type": "string", + "enum": ["require", "forbid", "allow"] + }, + "serverParameters": { + "type": "object", + "minProperties": 1 + }, + "auth": { "type": "boolean" }, + "csfle": { "type": "boolean" } + } + }, + + "entity": { + "type": "object", + "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, + "properties": { + "client": { + "type": "object", + "additionalProperties": false, + "required": ["id"], + "properties": { + "id": { "type": "string" }, + "uriOptions": { "type": "object" }, + "useMultipleMongoses": { "type": "boolean" }, + "observeEvents": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": [ + "commandStartedEvent", + "commandSucceededEvent", + "commandFailedEvent", + "poolCreatedEvent", + "poolReadyEvent", + "poolClearedEvent", + "poolClosedEvent", + "connectionCreatedEvent", + "connectionReadyEvent", + "connectionClosedEvent", + "connectionCheckOutStartedEvent", + "connectionCheckOutFailedEvent", + "connectionCheckedOutEvent", + "connectionCheckedInEvent", + "serverDescriptionChangedEvent", + "topologyDescriptionChangedEvent" + ] + } + }, + "ignoreCommandMonitoringEvents": { + "type": "array", + "minItems": 1, + "items": { "type": "string" } + }, + "storeEventsAsEntities": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/storeEventsAsEntity" } + }, + "observeLogMessages": { + "type": "object", + "minProperties": 1, + "additionalProperties": false, + "properties": { + "command": { "$ref": "#/definitions/logSeverityLevel" }, + "topology": { "$ref": "#/definitions/logSeverityLevel" }, + "serverSelection": { "$ref": "#/definitions/logSeverityLevel" }, + "connection": { "$ref": "#/definitions/logSeverityLevel" } + } + }, + "serverApi": { "$ref": "#/definitions/serverApi" }, + "observeSensitiveCommands": { "type": "boolean" } + } + }, + "clientEncryption": { + "type": "object", + "additionalProperties": false, + "required": ["id", "clientEncryptionOpts"], + "properties": { + "id": { "type": "string" }, + "clientEncryptionOpts": { "$ref": "#/definitions/clientEncryptionOpts" } + } + }, + "database": { + "type": "object", + "additionalProperties": false, + "required": ["id", "client", "databaseName"], + "properties": { + "id": { "type": "string" }, + "client": { "type": "string" }, + "databaseName": { "type": "string" }, + "databaseOptions": { "$ref": "#/definitions/collectionOrDatabaseOptions" } + } + }, + "collection": { + "type": "object", + "additionalProperties": false, + "required": ["id", "database", "collectionName"], + "properties": { + "id": { "type": "string" }, + "database": { "type": "string" }, + "collectionName": { "type": "string" }, + "collectionOptions": { "$ref": "#/definitions/collectionOrDatabaseOptions" } + } + }, + "session": { + "type": "object", + "additionalProperties": false, + "required": ["id", "client"], + "properties": { + "id": { "type": "string" }, + "client": { "type": "string" }, + "sessionOptions": { "type": "object" } + } + }, + "bucket": { + "type": "object", + "additionalProperties": false, + "required": ["id", "database"], + "properties": { + "id": { "type": "string" }, + "database": { "type": "string" }, + "bucketOptions": { "type": "object" } + } + }, + "thread": { + "type": "object", + "additionalProperties": false, + "required": ["id"], + "properties": { + "id": { "type": "string" } + } + } + } + }, + + "logComponent": { + "type": "string", + "enum": ["command", "topology", "serverSelection", "connection"] + }, + + "logSeverityLevel": { + "type": "string", + "enum": ["emergency", "alert", "critical", "error", "warning", "notice", "info", "debug", "trace"] + }, + + "clientEncryptionOpts": { + "type": "object", + "additionalProperties": false, + "required": ["keyVaultClient", "keyVaultNamespace", "kmsProviders"], + "properties": { + "keyVaultClient": { "type": "string" }, + "keyVaultNamespace": { "type": "string" }, + "kmsProviders": { "$ref": "#/definitions/kmsProviders" } + } + }, + + "kmsProviders": { + "$defs": { + "stringOrPlaceholder": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "object", + "additionalProperties": false, + "required": ["$$placeholder"], + "properties": { + "$$placeholder": {} + } + } + ] + } + }, + "type": "object", + "additionalProperties": false, + "properties": { + "aws": { + "type": "object", + "additionalProperties": false, + "properties": { + "accessKeyId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "secretAccessKey": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "sessionToken": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + }, + "azure": { + "type": "object", + "additionalProperties": false, + "properties": { + "tenantId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "clientId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "clientSecret": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "identityPlatformEndpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + }, + "gcp": { + "type": "object", + "additionalProperties": false, + "properties": { + "email": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "privateKey": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "endpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + }, + "kmip": { + "type": "object", + "additionalProperties": false, + "properties": { + "endpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + }, + "local": { + "type": "object", + "additionalProperties": false, + "properties": { + "key": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + } + } + }, + + "storeEventsAsEntity": { + "type": "object", + "additionalProperties": false, + "required": ["id", "events"], + "properties": { + "id": { "type": "string" }, + "events": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": [ + "PoolCreatedEvent", + "PoolReadyEvent", + "PoolClearedEvent", + "PoolClosedEvent", + "ConnectionCreatedEvent", + "ConnectionReadyEvent", + "ConnectionClosedEvent", + "ConnectionCheckOutStartedEvent", + "ConnectionCheckOutFailedEvent", + "ConnectionCheckedOutEvent", + "ConnectionCheckedInEvent", + "CommandStartedEvent", + "CommandSucceededEvent", + "CommandFailedEvent", + "ServerDescriptionChangedEvent", + "TopologyDescriptionChangedEvent" + ] + } + } + } + }, + + "collectionData": { + "type": "object", + "additionalProperties": false, + "required": ["collectionName", "databaseName", "documents"], + "properties": { + "collectionName": { "type": "string" }, + "databaseName": { "type": "string" }, + "createOptions": { + "type": "object", + "properties": { + "writeConcern": false + } + }, + "documents": { + "type": "array", + "items": { "type": "object" } + } + } + }, + + "expectedEventsForClient": { + "type": "object", + "additionalProperties": false, + "required": ["client", "events"], + "properties": { + "client": { "type": "string" }, + "eventType": { + "type": "string", + "enum": ["command", "cmap", "sdam"] + }, + "events": { "type": "array" }, + "ignoreExtraEvents": { "type": "boolean" } + }, + "oneOf": [ + { + "required": ["eventType"], + "properties": { + "eventType": { "const": "command" }, + "events": { + "type": "array", + "items": { "$ref": "#/definitions/expectedCommandEvent" } + } + } + }, + { + "required": ["eventType"], + "properties": { + "eventType": { "const": "cmap" }, + "events": { + "type": "array", + "items": { "$ref": "#/definitions/expectedCmapEvent" } + } + } + }, + { + "required": ["eventType"], + "properties": { + "eventType": { "const": "sdam" }, + "events": { + "type": "array", + "items": { "$ref": "#/definitions/expectedSdamEvent" } + } + } + }, + { + "additionalProperties": false, + "properties": { + "client": { "type": "string" }, + "events": { + "type": "array", + "items": { "$ref": "#/definitions/expectedCommandEvent" } + }, + "ignoreExtraEvents": { "type": "boolean" } + } + } + ] + }, + + "expectedCommandEvent": { + "type": "object", + "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, + "properties": { + "commandStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "command": { "type": "object" }, + "commandName": { "type": "string" }, + "databaseName": { "type": "string" }, + "hasServiceId": { "type": "boolean" }, + "hasServerConnectionId": { "type": "boolean" } + } + }, + "commandSucceededEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reply": { "type": "object" }, + "commandName": { "type": "string" }, + "databaseName": { "type": "string" }, + "hasServiceId": { "type": "boolean" }, + "hasServerConnectionId": { "type": "boolean" } + } + }, + "commandFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "commandName": { "type": "string" }, + "databaseName": { "type": "string" }, + "hasServiceId": { "type": "boolean" }, + "hasServerConnectionId": { "type": "boolean" } + } + } + } + }, + + "expectedCmapEvent": { + "type": "object", + "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, + "properties": { + "poolCreatedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "poolReadyEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "poolClearedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "hasServiceId": { "type": "boolean" }, + "interruptInUseConnections": { "type": "boolean" } + } + }, + "poolClosedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCreatedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionReadyEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionClosedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reason": { "type": "string" } + } + }, + "connectionCheckOutStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCheckOutFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reason": { "type": "string" } + } + }, + "connectionCheckedOutEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCheckedInEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + } + } + }, + + "expectedSdamEvent": { + "type": "object", + "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, + "properties": { + "serverDescriptionChangedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "previousDescription": { "$ref": "#/definitions/serverDescription" }, + "newDescription": { "$ref": "#/definitions/serverDescription" } + } + }, + "topologyDescriptionChangedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "serverHeartbeatStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { "type": "boolean" } + } + }, + "serverHeartbeatSucceededEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { "type": "boolean" } + } + }, + "serverHeartbeatFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { "type": "boolean" } + } + } + } + }, + + "serverDescription": { + "type": "object", + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "enum": [ + "Standalone", + "Mongos", + "PossiblePrimary", + "RSPrimary", + "RSSecondary", + "RSOther", + "RSArbiter", + "RSGhost", + "LoadBalancer", + "Unknown" + ] + } + } + }, + + "expectedLogMessagesForClient": { + "type": "object", + "additionalProperties": false, + "required": ["client", "messages"], + "properties": { + "client": { "type": "string" }, + "messages": { + "type": "array", + "items": { "$ref": "#/definitions/expectedLogMessage" } + }, + "ignoreExtraMessages": { "type": "boolean" }, + "ignoreMessages": { + "type": "array", + "items": { "$ref": "#/definitions/expectedLogMessage" } + } + } + }, + + "expectedLogMessage": { + "type": "object", + "additionalProperties": false, + "required": ["level", "component", "data"], + "properties": { + "level": { "$ref": "#/definitions/logSeverityLevel" }, + "component": { "$ref": "#/definitions/logComponent" }, + "data": { "type": "object" }, + "failureIsRedacted": { "type": "boolean" } + } + }, + + "collectionOrDatabaseOptions": { + "type": "object", + "additionalProperties": false, + "properties": { + "readConcern": { "type": "object" }, + "readPreference": { "type": "object" }, + "writeConcern": { "type": "object" }, + "timeoutMS": { "type": "integer" } + } + }, + + "serverApi": { + "type": "object", + "additionalProperties": false, + "required": ["version"], + "properties": { + "version": { "type": "string" }, + "strict": { "type": "boolean" }, + "deprecationErrors": { "type": "boolean" } + } + }, + + "operation": { + "type": "object", + "additionalProperties": false, + "required": ["name", "object"], + "properties": { + "name": { "type": "string" }, + "object": { "type": "string" }, + "arguments": { "type": "object" }, + "ignoreResultAndError": { "type": "boolean" }, + "expectError": { "$ref": "#/definitions/expectedError" }, + "expectResult": {}, + "saveResultAsEntity": { "type": "string" } + }, + "allOf": [ + { "not": { "required": ["expectError", "expectResult"] } }, + { "not": { "required": ["expectError", "saveResultAsEntity"] } }, + { "not": { "required": ["ignoreResultAndError", "expectResult"] } }, + { "not": { "required": ["ignoreResultAndError", "expectError"] } }, + { "not": { "required": ["ignoreResultAndError", "saveResultAsEntity"] } } + ] + }, + + "expectedError": { + "type": "object", + "additionalProperties": false, + "minProperties": 1, + "properties": { + "isError": { + "type": "boolean", + "const": true + }, + "isClientError": { "type": "boolean" }, + "isTimeoutError": { "type": "boolean" }, + "errorContains": { "type": "string" }, + "errorCode": { "type": "integer" }, + "errorCodeName": { "type": "string" }, + "errorLabelsContain": { + "type": "array", + "minItems": 1, + "items": { "type": "string" } + }, + "errorLabelsOmit": { + "type": "array", + "minItems": 1, + "items": { "type": "string" } + }, + "errorResponse": { + "type": "object" + }, + "expectResult": {} + } + }, + + "test": { + "type": "object", + "additionalProperties": false, + "required": ["description", "operations"], + "properties": { + "description": { "type": "string" }, + "runOnRequirements": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/runOnRequirement" } + }, + "skipReason": { "type": "string" }, + "operations": { + "type": "array", + "items": { "$ref": "#/definitions/operation" } + }, + "expectEvents": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/expectedEventsForClient" } + }, + "expectLogMessages": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/expectedLogMessagesForClient" } + }, + "outcome": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/collectionData" } + } + } + } + } + } diff --git a/source/unified-test-format/unified-test-format.rst b/source/unified-test-format/unified-test-format.rst index 43218840a9..72a9a691e8 100644 --- a/source/unified-test-format/unified-test-format.rst +++ b/source/unified-test-format/unified-test-format.rst @@ -4,7 +4,7 @@ Unified Test Format :Status: Accepted :Minimum Server Version: N/A -:Current Schema Version: 1.13.0 +:Current Schema Version: 1.17.0 .. contents:: @@ -598,6 +598,12 @@ The structure of this object is as follows: - `serverDescriptionChangedEvent `_ + - `serverHeartbeatStartedEvent `_ + + - `serverHeartbeatSucceededEvent `_ + + - `serverHeartbeatFailedEvent `_ + - `topologyDescriptionChangedEvent `_ .. _entity_client_ignoreCommandMonitoringEvents: @@ -1399,6 +1405,33 @@ The structure of this object is as follows: <../server-discovery-and-monitoring/server-discovery-and-monitoring.rst#servertype>`__ for a list of valid values. +.. _expectedEvent_serverHeartbeatStartedEvent: + +- ``serverHeartbeatStartedEvent``: Optional object. Assertions for one or more + `ServerHeartbeatStartedEvent <../server-discovery-and-monitoring/server-discovery-and-monitoring-logging-and-monitoring.rst#events>`__ fields. + + The structure of this object is as follows: + + - ``awaited``: Optional boolean. If specified, test runners MUST assert that the field is set and matches this value. + +.. _expectedEvent_serverHeartbeatSucceededEvent: + +- ``serverHeartbeatSucceededEvent``: Optional object. Assertions for one or more + `ServerHeartbeatSucceededEvent <../server-discovery-and-monitoring/server-discovery-and-monitoring-logging-and-monitoring.rst#events>`__ fields. + + The structure of this object is as follows: + + - ``awaited``: Optional boolean. If specified, test runners MUST assert that the field is set and matches this value. + +.. _expectedEvent_serverHeartbeatFailedEvent: + +- ``serverHeartbeatFailedEvent``: Optional object. Assertions for one or more + `ServerHeartbeatFailedEvent <../server-discovery-and-monitoring/server-discovery-and-monitoring-logging-and-monitoring.rst#events>`__ fields. + + The structure of this object is as follows: + + - ``awaited``: Optional boolean. If specified, test runners MUST assert that the field is set and matches this value. + .. _expectedEvent_topologyDescriptionChangedEvent: - ``topologyDescriptionChangedEvent``: Optional object. If present, this object @@ -4028,6 +4061,9 @@ Changelog .. Please note schema version bumps in changelog entries where applicable. +:2023-10-04: **Schema version 1.17.** + Add ``serverHeartbeatStartedEvent``, ``serverHeartbeatSucceededEvent``, and + ``serverHeartbeatFailedEvent`` for asserting on SDAM server heartbeat events. :2023-09-25: Clarify that the UTR is intended to be run against enterprise servers. :2022-07-18: **Schema version 1.16.** Add ``ignoreMessages`` and ``ignoreExtraMessages`` fields From 4da545648cea9ff8ca2df8ae4bfad1353d5ce7ef Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Thu, 5 Oct 2023 11:20:32 -0700 Subject: [PATCH 09/15] DRIVERS-2578 Fix serverMonitoringMode=poll test --- .../tests/unified/serverMonitoringMode.json | 14 +++++++++++++- .../tests/unified/serverMonitoringMode.yml | 9 +++++++++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json index 651fb4383e..28903fe34f 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json @@ -164,7 +164,8 @@ "client": { "id": "client2", "uriOptions": { - "serverMonitoringMode": "poll" + "serverMonitoringMode": "poll", + "heartbeatFrequencyMS": 500 }, "useMultipleMongoses": false, "observeEvents": [ @@ -196,6 +197,17 @@ "expectResult": { "ok": 1 } + }, + { + "name": "waitForEvent", + "object": "testRunner", + "arguments": { + "client": "client2", + "event": { + "serverHeartbeatStartedEvent": {} + }, + "count": 2 + } } ], "expectEvents": [ diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml index c9cc31b78f..00eb5821eb 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml @@ -90,6 +90,7 @@ tests: id: &client2 client2 uriOptions: serverMonitoringMode: "poll" + heartbeatFrequencyMS: 500 useMultipleMongoses: false observeEvents: - serverHeartbeatStartedEvent @@ -105,6 +106,14 @@ tests: commandName: ping command: { ping: 1 } expectResult: { ok: 1 } + # Wait for the second serverHeartbeatStartedEvent to ensure we do not stream. + - name: waitForEvent + object: testRunner + arguments: + client: *client2 + event: + serverHeartbeatStartedEvent: {} + count: 2 expectEvents: - client: *client2 eventType: sdam From 0cccfab9ee50bf74c09db88451954a550ea62d34 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Thu, 5 Oct 2023 11:23:08 -0700 Subject: [PATCH 10/15] DRIVERS-2578 Use ignoreExtraEvents for extra SDAM events --- .../tests/unified/serverMonitoringMode.json | 3 +++ .../tests/unified/serverMonitoringMode.yml | 3 +++ 2 files changed, 6 insertions(+) diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json index 28903fe34f..1fbe52a3dc 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json @@ -62,6 +62,7 @@ { "client": "client0", "eventType": "sdam", + "ignoreExtraEvents": true, "events": [ { "serverHeartbeatStartedEvent": { @@ -132,6 +133,7 @@ { "client": "client1", "eventType": "sdam", + "ignoreExtraEvents": true, "events": [ { "serverHeartbeatStartedEvent": { @@ -214,6 +216,7 @@ { "client": "client2", "eventType": "sdam", + "ignoreExtraEvents": true, "events": [ { "serverHeartbeatStartedEvent": { diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml index 00eb5821eb..0b681292f3 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml @@ -36,6 +36,7 @@ tests: expectEvents: - client: *client0 eventType: sdam + ignoreExtraEvents: true events: - serverHeartbeatStartedEvent: awaited: False @@ -72,6 +73,7 @@ tests: expectEvents: - client: *client1 eventType: sdam + ignoreExtraEvents: true events: - serverHeartbeatStartedEvent: awaited: False @@ -117,6 +119,7 @@ tests: expectEvents: - client: *client2 eventType: sdam + ignoreExtraEvents: true events: - serverHeartbeatStartedEvent: awaited: False From 73fcd6f4bcc9287efc620b0faccbf59a072932a7 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Thu, 5 Oct 2023 13:17:29 -0700 Subject: [PATCH 11/15] DRIVERS-2578 Make tests more reliable with waitForEvent --- .../tests/unified/serverMonitoringMode.json | 68 ++++++++++++++----- .../tests/unified/serverMonitoringMode.yml | 56 ++++++++++----- 2 files changed, 88 insertions(+), 36 deletions(-) diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json index 1fbe52a3dc..20bf19155b 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json @@ -13,7 +13,12 @@ ], "tests": [ { - "description": "connect with serverMonitoringMode=auto", + "description": "connect with serverMonitoringMode=auto >=4.4", + "runOnRequirements": [ + { + "minServerVersion": "4.4" + } + ], "operations": [ { "name": "createEntities", @@ -22,7 +27,7 @@ "entities": [ { "client": { - "id": "client0", + "id": "client", "uriOptions": { "serverMonitoringMode": "auto" }, @@ -36,8 +41,8 @@ }, { "database": { - "id": "dbServerMonitoringModeAuto", - "client": "client0", + "id": "db", + "client": "client", "databaseName": "sdam-tests" } } @@ -46,7 +51,7 @@ }, { "name": "runCommand", - "object": "dbServerMonitoringModeAuto", + "object": "db", "arguments": { "commandName": "ping", "command": { @@ -56,11 +61,22 @@ "expectResult": { "ok": 1 } + }, + { + "name": "waitForEvent", + "object": "testRunner", + "arguments": { + "client": "client", + "event": { + "serverHeartbeatStartedEvent": {} + }, + "count": 2 + } } ], "expectEvents": [ { - "client": "client0", + "client": "client", "eventType": "sdam", "ignoreExtraEvents": true, "events": [ @@ -84,7 +100,12 @@ ] }, { - "description": "connect with serverMonitoringMode=stream", + "description": "connect with serverMonitoringMode=stream >=4.4", + "runOnRequirements": [ + { + "minServerVersion": "4.4" + } + ], "operations": [ { "name": "createEntities", @@ -93,7 +114,7 @@ "entities": [ { "client": { - "id": "client1", + "id": "client", "uriOptions": { "serverMonitoringMode": "stream" }, @@ -107,8 +128,8 @@ }, { "database": { - "id": "dbServerMonitoringModeStream", - "client": "client1", + "id": "db", + "client": "client", "databaseName": "sdam-tests" } } @@ -117,7 +138,7 @@ }, { "name": "runCommand", - "object": "dbServerMonitoringModeStream", + "object": "db", "arguments": { "commandName": "ping", "command": { @@ -127,11 +148,22 @@ "expectResult": { "ok": 1 } + }, + { + "name": "waitForEvent", + "object": "testRunner", + "arguments": { + "client": "client", + "event": { + "serverHeartbeatStartedEvent": {} + }, + "count": 2 + } } ], "expectEvents": [ { - "client": "client1", + "client": "client", "eventType": "sdam", "ignoreExtraEvents": true, "events": [ @@ -164,7 +196,7 @@ "entities": [ { "client": { - "id": "client2", + "id": "client", "uriOptions": { "serverMonitoringMode": "poll", "heartbeatFrequencyMS": 500 @@ -179,8 +211,8 @@ }, { "database": { - "id": "dbServerMonitoringModePoll", - "client": "client2", + "id": "db", + "client": "client", "databaseName": "sdam-tests" } } @@ -189,7 +221,7 @@ }, { "name": "runCommand", - "object": "dbServerMonitoringModePoll", + "object": "db", "arguments": { "commandName": "ping", "command": { @@ -204,7 +236,7 @@ "name": "waitForEvent", "object": "testRunner", "arguments": { - "client": "client2", + "client": "client", "event": { "serverHeartbeatStartedEvent": {} }, @@ -214,7 +246,7 @@ ], "expectEvents": [ { - "client": "client2", + "client": "client", "eventType": "sdam", "ignoreExtraEvents": true, "events": [ diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml index 0b681292f3..d24bceae77 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml @@ -8,14 +8,16 @@ runOnRequirements: - topologies: [single, sharded, sharded-replicaset] serverless: forbid tests: - - description: "connect with serverMonitoringMode=auto" + - description: "connect with serverMonitoringMode=auto >=4.4" + runOnRequirements: + - minServerVersion: "4.4" operations: - name: createEntities object: testRunner arguments: entities: - client: - id: &client0 client0 + id: client uriOptions: serverMonitoringMode: "auto" useMultipleMongoses: false @@ -24,17 +26,25 @@ tests: - serverHeartbeatSucceededEvent - serverHeartbeatFailedEvent - database: - id: &dbServerMonitoringModeAuto dbServerMonitoringModeAuto - client: *client0 + id: db + client: client databaseName: sdam-tests - name: runCommand - object: *dbServerMonitoringModeAuto + object: db arguments: commandName: ping command: { ping: 1 } expectResult: { ok: 1 } + # Wait for the second serverHeartbeatStartedEvent to ensure we start streaming. + - name: waitForEvent + object: testRunner + arguments: + client: client + event: + serverHeartbeatStartedEvent: {} + count: 2 expectEvents: - - client: *client0 + - client: client eventType: sdam ignoreExtraEvents: true events: @@ -45,14 +55,16 @@ tests: - serverHeartbeatStartedEvent: awaited: True - - description: "connect with serverMonitoringMode=stream" + - description: "connect with serverMonitoringMode=stream >=4.4" + runOnRequirements: + - minServerVersion: "4.4" operations: - name: createEntities object: testRunner arguments: entities: - client: - id: &client1 client1 + id: client uriOptions: serverMonitoringMode: "stream" useMultipleMongoses: false @@ -61,17 +73,25 @@ tests: - serverHeartbeatSucceededEvent - serverHeartbeatFailedEvent - database: - id: &dbServerMonitoringModeStream dbServerMonitoringModeStream - client: *client1 + id: db + client: client databaseName: sdam-tests - name: runCommand - object: *dbServerMonitoringModeStream + object: db arguments: commandName: ping command: { ping: 1 } expectResult: { ok: 1 } + # Wait for the second serverHeartbeatStartedEvent to ensure we start streaming. + - name: waitForEvent + object: testRunner + arguments: + client: client + event: + serverHeartbeatStartedEvent: {} + count: 2 expectEvents: - - client: *client1 + - client: client eventType: sdam ignoreExtraEvents: true events: @@ -89,7 +109,7 @@ tests: arguments: entities: - client: - id: &client2 client2 + id: client uriOptions: serverMonitoringMode: "poll" heartbeatFrequencyMS: 500 @@ -99,11 +119,11 @@ tests: - serverHeartbeatSucceededEvent - serverHeartbeatFailedEvent - database: - id: &dbServerMonitoringModePoll dbServerMonitoringModePoll - client: *client2 + id: db + client: client databaseName: sdam-tests - name: runCommand - object: *dbServerMonitoringModePoll + object: db arguments: commandName: ping command: { ping: 1 } @@ -112,12 +132,12 @@ tests: - name: waitForEvent object: testRunner arguments: - client: *client2 + client: client event: serverHeartbeatStartedEvent: {} count: 2 expectEvents: - - client: *client2 + - client: client eventType: sdam ignoreExtraEvents: true events: From acf3ed903f2ac6e509b6a4f63468f012cc20ae64 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Thu, 5 Oct 2023 13:21:25 -0700 Subject: [PATCH 12/15] DRIVERS-2578 Refactor to reduce boilerplate in yaml test --- .../tests/unified/serverMonitoringMode.yml | 52 ++++--------------- 1 file changed, 11 insertions(+), 41 deletions(-) diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml index d24bceae77..eae0a3aeea 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml @@ -29,21 +29,23 @@ tests: id: db client: client databaseName: sdam-tests - - name: runCommand + - &ping + name: runCommand object: db arguments: commandName: ping command: { ping: 1 } expectResult: { ok: 1 } # Wait for the second serverHeartbeatStartedEvent to ensure we start streaming. - - name: waitForEvent + - &waitForSecondHeartbeatStarted + name: waitForEvent object: testRunner arguments: client: client event: serverHeartbeatStartedEvent: {} count: 2 - expectEvents: + expectEvents: &streamingStartedEvents - client: client eventType: sdam ignoreExtraEvents: true @@ -76,31 +78,10 @@ tests: id: db client: client databaseName: sdam-tests - - name: runCommand - object: db - arguments: - commandName: ping - command: { ping: 1 } - expectResult: { ok: 1 } + - *ping # Wait for the second serverHeartbeatStartedEvent to ensure we start streaming. - - name: waitForEvent - object: testRunner - arguments: - client: client - event: - serverHeartbeatStartedEvent: {} - count: 2 - expectEvents: - - client: client - eventType: sdam - ignoreExtraEvents: true - events: - - serverHeartbeatStartedEvent: - awaited: False - - serverHeartbeatSucceededEvent: - awaited: False - - serverHeartbeatStartedEvent: - awaited: True + - *waitForSecondHeartbeatStarted + expectEvents: *streamingStartedEvents - description: "connect with serverMonitoringMode=poll" operations: @@ -122,21 +103,10 @@ tests: id: db client: client databaseName: sdam-tests - - name: runCommand - object: db - arguments: - commandName: ping - command: { ping: 1 } - expectResult: { ok: 1 } + - *ping # Wait for the second serverHeartbeatStartedEvent to ensure we do not stream. - - name: waitForEvent - object: testRunner - arguments: - client: client - event: - serverHeartbeatStartedEvent: {} - count: 2 - expectEvents: + - *waitForSecondHeartbeatStarted + expectEvents: &pollingStartedEvents - client: client eventType: sdam ignoreExtraEvents: true From 9e2acb0f51816adaae8c25733ff831fbd497abd1 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Thu, 5 Oct 2023 13:25:44 -0700 Subject: [PATCH 13/15] DRIVERS-2578 Add tests for pre and post 4.4 behavior --- .../tests/unified/serverMonitoringMode.json | 180 +++++++++++++++++- .../tests/unified/serverMonitoringMode.yml | 80 ++++++-- 2 files changed, 245 insertions(+), 15 deletions(-) diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json index 20bf19155b..7d681b4f9e 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.json @@ -16,7 +16,7 @@ "description": "connect with serverMonitoringMode=auto >=4.4", "runOnRequirements": [ { - "minServerVersion": "4.4" + "minServerVersion": "4.4.0" } ], "operations": [ @@ -99,11 +99,99 @@ } ] }, + { + "description": "connect with serverMonitoringMode=auto <4.4", + "runOnRequirements": [ + { + "maxServerVersion": "4.2.99" + } + ], + "operations": [ + { + "name": "createEntities", + "object": "testRunner", + "arguments": { + "entities": [ + { + "client": { + "id": "client", + "uriOptions": { + "serverMonitoringMode": "auto", + "heartbeatFrequencyMS": 500 + }, + "useMultipleMongoses": false, + "observeEvents": [ + "serverHeartbeatStartedEvent", + "serverHeartbeatSucceededEvent", + "serverHeartbeatFailedEvent" + ] + } + }, + { + "database": { + "id": "db", + "client": "client", + "databaseName": "sdam-tests" + } + } + ] + } + }, + { + "name": "runCommand", + "object": "db", + "arguments": { + "commandName": "ping", + "command": { + "ping": 1 + } + }, + "expectResult": { + "ok": 1 + } + }, + { + "name": "waitForEvent", + "object": "testRunner", + "arguments": { + "client": "client", + "event": { + "serverHeartbeatStartedEvent": {} + }, + "count": 2 + } + } + ], + "expectEvents": [ + { + "client": "client", + "eventType": "sdam", + "ignoreExtraEvents": true, + "events": [ + { + "serverHeartbeatStartedEvent": { + "awaited": false + } + }, + { + "serverHeartbeatSucceededEvent": { + "awaited": false + } + }, + { + "serverHeartbeatStartedEvent": { + "awaited": false + } + } + ] + } + ] + }, { "description": "connect with serverMonitoringMode=stream >=4.4", "runOnRequirements": [ { - "minServerVersion": "4.4" + "minServerVersion": "4.4.0" } ], "operations": [ @@ -186,6 +274,94 @@ } ] }, + { + "description": "connect with serverMonitoringMode=stream <4.4", + "runOnRequirements": [ + { + "maxServerVersion": "4.2.99" + } + ], + "operations": [ + { + "name": "createEntities", + "object": "testRunner", + "arguments": { + "entities": [ + { + "client": { + "id": "client", + "uriOptions": { + "serverMonitoringMode": "stream", + "heartbeatFrequencyMS": 500 + }, + "useMultipleMongoses": false, + "observeEvents": [ + "serverHeartbeatStartedEvent", + "serverHeartbeatSucceededEvent", + "serverHeartbeatFailedEvent" + ] + } + }, + { + "database": { + "id": "db", + "client": "client", + "databaseName": "sdam-tests" + } + } + ] + } + }, + { + "name": "runCommand", + "object": "db", + "arguments": { + "commandName": "ping", + "command": { + "ping": 1 + } + }, + "expectResult": { + "ok": 1 + } + }, + { + "name": "waitForEvent", + "object": "testRunner", + "arguments": { + "client": "client", + "event": { + "serverHeartbeatStartedEvent": {} + }, + "count": 2 + } + } + ], + "expectEvents": [ + { + "client": "client", + "eventType": "sdam", + "ignoreExtraEvents": true, + "events": [ + { + "serverHeartbeatStartedEvent": { + "awaited": false + } + }, + { + "serverHeartbeatSucceededEvent": { + "awaited": false + } + }, + { + "serverHeartbeatStartedEvent": { + "awaited": false + } + } + ] + } + ] + }, { "description": "connect with serverMonitoringMode=poll", "operations": [ diff --git a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml index eae0a3aeea..28c7853d04 100644 --- a/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml +++ b/source/server-discovery-and-monitoring/tests/unified/serverMonitoringMode.yml @@ -10,7 +10,7 @@ runOnRequirements: tests: - description: "connect with serverMonitoringMode=auto >=4.4" runOnRequirements: - - minServerVersion: "4.4" + - minServerVersion: "4.4.0" operations: - name: createEntities object: testRunner @@ -57,9 +57,46 @@ tests: - serverHeartbeatStartedEvent: awaited: True + - description: "connect with serverMonitoringMode=auto <4.4" + runOnRequirements: + - maxServerVersion: "4.2.99" + operations: + - name: createEntities + object: testRunner + arguments: + entities: + - client: + id: client + uriOptions: + serverMonitoringMode: "auto" + heartbeatFrequencyMS: 500 + useMultipleMongoses: false + observeEvents: + - serverHeartbeatStartedEvent + - serverHeartbeatSucceededEvent + - serverHeartbeatFailedEvent + - database: + id: db + client: client + databaseName: sdam-tests + - *ping + # Wait for the second serverHeartbeatStartedEvent to ensure we do not stream. + - *waitForSecondHeartbeatStarted + expectEvents: &pollingStartedEvents + - client: client + eventType: sdam + ignoreExtraEvents: true + events: + - serverHeartbeatStartedEvent: + awaited: False + - serverHeartbeatSucceededEvent: + awaited: False + - serverHeartbeatStartedEvent: + awaited: False + - description: "connect with serverMonitoringMode=stream >=4.4" runOnRequirements: - - minServerVersion: "4.4" + - minServerVersion: "4.4.0" operations: - name: createEntities object: testRunner @@ -83,6 +120,33 @@ tests: - *waitForSecondHeartbeatStarted expectEvents: *streamingStartedEvents + - description: "connect with serverMonitoringMode=stream <4.4" + runOnRequirements: + - maxServerVersion: "4.2.99" + operations: + - name: createEntities + object: testRunner + arguments: + entities: + - client: + id: client + uriOptions: + serverMonitoringMode: "stream" + heartbeatFrequencyMS: 500 + useMultipleMongoses: false + observeEvents: + - serverHeartbeatStartedEvent + - serverHeartbeatSucceededEvent + - serverHeartbeatFailedEvent + - database: + id: db + client: client + databaseName: sdam-tests + - *ping + # Wait for the second serverHeartbeatStartedEvent to ensure we do not stream. + - *waitForSecondHeartbeatStarted + expectEvents: *pollingStartedEvents + - description: "connect with serverMonitoringMode=poll" operations: - name: createEntities @@ -106,14 +170,4 @@ tests: - *ping # Wait for the second serverHeartbeatStartedEvent to ensure we do not stream. - *waitForSecondHeartbeatStarted - expectEvents: &pollingStartedEvents - - client: client - eventType: sdam - ignoreExtraEvents: true - events: - - serverHeartbeatStartedEvent: - awaited: False - - serverHeartbeatSucceededEvent: - awaited: False - - serverHeartbeatStartedEvent: - awaited: False + expectEvents: *pollingStartedEvents From fa108ae4b731791b0bb971b0dc57a7db3fd55ad8 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Thu, 5 Oct 2023 14:32:45 -0700 Subject: [PATCH 14/15] DRIVERS-2578 Clients MUST NOT use dedicated connections to measure RTT when using the polling protocol --- .../server-monitoring.rst | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/source/server-discovery-and-monitoring/server-monitoring.rst b/source/server-discovery-and-monitoring/server-monitoring.rst index c794fbc9c4..03cb8906a2 100644 --- a/source/server-discovery-and-monitoring/server-monitoring.rst +++ b/source/server-discovery-and-monitoring/server-monitoring.rst @@ -566,9 +566,8 @@ Measuring RTT When using the streaming protocol, clients MUST issue a hello or legacy hello command to each server to measure RTT every heartbeatFrequencyMS. The RTT command -MUST be run on a dedicated connection to each server. For consistency, -clients MAY use dedicated connections to measure RTT for all servers, even -those that do not support awaitable hello or legacy hello. (See +MUST be run on a dedicated connection to each server. Clients MUST NOT use +dedicated connections to measure RTT when the streaming protocol is not used. (See `Monitors MUST use a dedicated connection for RTT commands`_.) Clients MUST update the RTT from the hello or legacy hello duration of the initial @@ -1188,12 +1187,7 @@ awaitable hello or legacy hello heartbeat in the new protocol. Why disable the streaming protocol on FaaS platforms like AWS Lambda? ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' -The streaming protocol requires an extra connection and thread per monitored -server which is expensive on platforms like AWS Lambda. The extra connection -is particularly inefficient when thousands of AWS instances and thus -thousands of clients are used. - -Additionally, the streaming protocol relies on the assumption that the client +The streaming protocol relies on the assumption that the client can read the server's heartbeat responses in a timely manner, otherwise the client will be acting on stale information. In many FaaS platforms, like AWS Lambda, host applications will be suspended and resumed many minutes later. @@ -1201,6 +1195,11 @@ This behavior causes a build up of heartbeat responses and the client can end up spending a long time in a catch up phase processing outdated responses. This problem was discovered in `DRIVERS-2246`_. +Additionally, the streaming protocol requires an extra connection and thread +per monitored server which is expensive on platforms like AWS Lambda. The +extra connection is particularly inefficient when thousands of AWS instances +and thus thousands of clients are used. + We decided to make polling the default behavior when running on FaaS platforms like AWS Lambda to improve scalability, performance, and reliability. @@ -1230,7 +1229,8 @@ Changelog :2022-04-05: Preemptively cancel in progress operations when SDAM heartbeats timeout. :2022-10-05: Remove spec front matter reformat changelog. :2022-11-17: Add minimum RTT tracking and remove 90th percentile RTT. -:2023-08-21: Add serverMonitoringMode and default to the Polling Protocol on FaaS. +:2023-10-05: Add serverMonitoringMode and default to the polling protocol on FaaS. + Clients MUST NOT use dedicated connections to measure RTT when using the polling protocol. ---- From 0eb23a0fe72eaed864affcc98e9e67339f0801e9 Mon Sep 17 00:00:00 2001 From: Shane Harvey Date: Thu, 5 Oct 2023 15:02:48 -0700 Subject: [PATCH 15/15] DRIVERS-2578 Fix "an workaround" typo --- source/server-discovery-and-monitoring/server-monitoring.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/server-discovery-and-monitoring/server-monitoring.rst b/source/server-discovery-and-monitoring/server-monitoring.rst index 03cb8906a2..a4622ce619 100644 --- a/source/server-discovery-and-monitoring/server-monitoring.rst +++ b/source/server-discovery-and-monitoring/server-monitoring.rst @@ -1206,7 +1206,7 @@ like AWS Lambda to improve scalability, performance, and reliability. Why introduce a knob for serverMonitoringMode? '''''''''''''''''''''''''''''''''''''''''''''' -The serverMonitoringMode knob provides an workaround in cases where the polling +The serverMonitoringMode knob provides a workaround in cases where the polling protocol would be a better choice but the driver is not running on a FaaS platform. It also provides a workaround in case the FaaS detection logic becomes outdated or inaccurate.