-
Notifications
You must be signed in to change notification settings - Fork 1.1k
RFC: incremental delivery with deduplication + concurrent execution #1026
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update Section 3 -- Type System.md Update Section 3 -- Type System.md Update Section 3 -- Type System.md Update Section 6 -- Execution.md Update Section 6 -- Execution.md Update Section 6 -- Execution.md Update Section 6 -- Execution.md Update Section 6 -- Execution.md Update Section 6 -- Execution.md Update Section 6 -- Execution.md Update Section 6 -- Execution.md Update Section 6 -- Execution.md Amend changes change initial_count to initialCount add payload fields to Response section add stream validation for overlapping fields spelling updates add note about re-execution add note about final payloads label is optional fix build Update ExecuteQuery with hasNext logic fix spelling fix spaces Update execution to add defer/stream to mutations and subscriptions clarify stream records Apply suggestions from code review Co-authored-by: Benjie Gillam <[email protected]> missing bracket Update spec/Section 7 -- Response.md Co-authored-by: Benjie Gillam <[email protected]> clarify line about stream record iterator update visitedFragments with defer Updates to consolidate subsequent payload logic for queries, mutations, and subscriptions Apply suggestions from code review Co-authored-by: Benjie Gillam <[email protected]> address review feedback Add handling of termination signal more formatting fix spelling Add assertion for record type add "Stream Directives Are Used On List Fields" validation rule Add defaultValue to @stream initialCount Update spec/Section 5 -- Validation.md Co-authored-by: Benjie Gillam <[email protected]> # Conflicts: # spec/Section 3 -- Type System.md # spec/Section 5 -- Validation.md # spec/Section 6 -- Execution.md # spec/Section 7 -- Response.md
# Conflicts: # spec/Section 3 -- Type System.md
# Conflicts: # spec/Section 3 -- Type System.md
# Conflicts: # spec/Section 3 -- Type System.md
# Conflicts: # spec/Section 3 -- Type System.md
# Conflicts: # spec/Section 7 -- Response.md
# Conflicts: # spec/Section 3 -- Type System.md # spec/Section 6 -- Execution.md
# Conflicts: # spec/Section 7 -- Response.md
# Conflicts: # spec/Section 6 -- Execution.md
# Conflicts: # spec/Section 6 -- Execution.md
# Conflicts: # spec/Section 6 -- Execution.md
…ators # Conflicts: # spec/Section 6 -- Execution.md
60347a3
to
b53af33
Compare
This was referenced May 23, 2023
7d67c8c
to
5c6181a
Compare
Pushed some fixes — see graphql/graphql-js#3886 (comment) |
364a6a7
to
b1a1d46
Compare
8451f2c
to
5399d23
Compare
closing in favor of #1034 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These spec edits should correspond to the working implementation of incremental delivery with deduplication currently posted as PR stack:
graphql/graphql-js#3894 => introduces Publisher
graphql/graphql-js#3886 => bulk of effort, introduces deduplication
graphql/graphql-js#3897 => adds pending
These spec edits do not currently include optional follow-on:
graphql/graphql-js#3895 => payload consolidation
[The diff to main might be helpful, but this is built on top of the amazing #742 and so the diff from that branch could be more useful.]
IMPORTANT NOTE:
These are hopefully trending toward complete in terms of the algorithm -- more explanatory prose should definitely be added.
TLDR?
The implementation and spec changes show how one can start executing deferred fragments semi-immediately (i.e. after deferring in an implementation-specific way), rather than waiting for the entire initial result to be emitted. This is not required -- one could still be compliant with the spec by deferring all the way until the initial result completes! In fact, how one defers is not per se observable and so the spec cannot mandate much about it with great normative force. But -- and I think this is important -- this PR and the implementation PR provide an algorithm/implementation/spec changes that give servers the flexibility to do what they think is right in that regard, and that might be desirable.
As of this moment, I am fairly confident in the implementation PR over at
graphql-js
, and the spec PR should generally correspond, demonstrating:= the
Field Group
andDefer Usage
record types that contain the information derived from the operation during field collection= the introduction of a distinction between
Incremental Data
records andSubsequent Result
records.Deferred Fragment
records andStream Items
records exemplifySubsequent Result
s that are sent as a group to the client. But an individualDeferred Fragment
Record may consist of a number of distinctDeferred Grouped Field Set
records, which may overlap with otherDeferred Fragment
Records and should not be sent more than once.Deferred Grouped Field Set
records are therefore a unit ofIncremental Data
and are tracked with a new record type.Stream Items
records always contain a single unit of incremental data that is sent only once with little complication; they therefore represent bothSubsequent Result
andIncremental Data
Records.= the new
Publisher
construct, with a set of algorithms that create and manipulate Subsequent Result and Incremental Data records.Mutation
of these records is not performed directly during execution, but only via interaction with the Publisher.= the deferMap, which maps
Defer Usage
records to individualDeferred Fragment
Subsequent Result
records** huge thanks to @Urigo and @dotansimha of the guild for sponsoring my open-source work on this. **