REP-6088 Tolerate high numbers of mismatches #117
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tdq45gj
requested changes
Jun 3, 2025
| return errors.Wrapf(err, "starting session") | ||
| } | ||
|
|
||
| sctx := mongo.NewSessionContext(ctx, sess) |
There was a problem hiding this comment.
Are we reading in a session just to get the cluster time?
There was a problem hiding this comment.
Yes. That, per the driver team, is the approved way to do this.
There was a problem hiding this comment.
Could we add a comment to explain the purpose of a session here?
FGasper
added a commit
that referenced
this pull request
Jun 23, 2025
- Mismatches were previously shown in an indeterminate order. Now they’re consistently sorted by the mismatched documents’ `_id`. - Documents with missing fields were being logged as entirely missing. That logic is corrected here. - The logic to create the table of missing/changed documents previously iterated through the _ids persisted in the task rather than the actual missing/changed documents. This was appropriate when that list stored mismatches but is no longer correct since the list now always stores the list of documents to check in the task. Thus, if there were only a handful of missing documents in a recheck task that contained thousands of document IDs, all of that task’s document IDs would be logged as missing. This was an oversight from PR #117, which should have updated the logic to build that table as it migrated that for the mismatched-documents table. This changeset does the necessary update.
FGasper
added a commit
that referenced
this pull request
Jun 23, 2025
- Mismatches were previously shown in an indeterminate order. Now they’re consistently sorted by the mismatched documents’ `_id`. - Documents with missing fields were being logged as entirely missing. That logic is corrected here. - The logic to create the table of missing/changed documents previously iterated through the _ids persisted in the task rather than the actual missing/changed documents. This was appropriate when that list stored mismatches but is no longer correct since the list now always stores the list of documents to check in the task. Thus, if there were only a handful of missing documents in a recheck task that contained thousands of document IDs, all of that task’s document IDs would be logged as missing. This was an oversight from PR #117, which should have updated the logic to build that table as it migrated that for the mismatched-documents table. This changeset does the necessary update.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously all document mismatches were recorded directly in the verification task. This meant, though, that if a task encompassed a large number of mismatched or missing documents, the verifier could fail to persist all of the mismatches, which caused a crash.
(The usual cause of excess mismatched/missing documents is starting migration-verifier before initial sync finishes, but it can also reasonably happen without REP-6129’s fix for queries against pre-v5 servers. See HELP-75910.)
This changeset makes the verifier save mismatches to a dedicated collection instead, one document per mismatch.
This change upends some familiar workflows for investigating mismatches: it’s no longer sufficient just to query the
verification_taskscollection for mismatch information since the actual mismatches are recorded in a separate collection. To address this, the documentation now gives an aggregation pipeline that yields a similarly-useful result.This entails a metadata version change. Because that’s happening, this also changes the task type
verifytoverifyDocuments. (That required some sorting workarounds in tests, which were tight-coupled to the task type strings.)