Skip to content

Conversation

radeksimko
Copy link
Member


⚠️ Not to be merged at this point ⚠️


Only for prototyping and discussion for now.

@radeksimko radeksimko force-pushed the f-pluggable-state-store branch 4 times, most recently from 0c90fa1 to 0f65d66 Compare July 18, 2025 10:08
@radeksimko radeksimko force-pushed the f-pluggable-state-store branch 2 times, most recently from f7660c3 to 9394014 Compare August 18, 2025 14:23
@radeksimko radeksimko force-pushed the f-pluggable-state-store branch 2 times, most recently from 5c186b3 to a813345 Compare August 28, 2025 14:52
SarahFrench added a commit to hashicorp/terraform that referenced this pull request Aug 29, 2025
SarahFrench added a commit to hashicorp/terraform that referenced this pull request Sep 2, 2025
* Update Read/WriteStateBytes RPCs to match hashicorp/terraform-plugin-go#531

* Run `make protobuf`

* Run `make generate`

* Update use of `proto.ReadStateBytes_ResponseChunk` in tests

* Fix how diagnostics are handled alongside EOF error, update ReadStateBytes test

* More fixes - test setup was incorrect

I think? I assume that a response would be returned full of zero-values when EOF is encountered.
@radeksimko radeksimko force-pushed the f-pluggable-state-store branch 4 times, most recently from ad19d54 to bebe220 Compare September 5, 2025 13:39
@radeksimko radeksimko force-pushed the f-pluggable-state-store branch from bebe220 to 54a47f3 Compare September 5, 2025 14:16
Comment on lines +1674 to +1704
if err != nil {
// attempt to send the error back to client
msgErr := srv.SendMsg(&tfplugin6.WriteStateBytes_Response{
Diagnostics: toproto.Diagnostics([]*tfprotov6.Diagnostic{
{
Severity: tfprotov6.DiagnosticSeverityError,
Summary: "Writing state chunk failed",
Detail: fmt.Sprintf("Attempt to write a byte chunk of state %q to %q failed: %s",
chunk.StateId, chunk.TypeName, err),
},
}),
})
if msgErr != nil {
err := status.Error(codes.Unimplemented, "ProviderServer does not implement WriteStateBytes")
logging.ProtocolError(ctx, err.Error())
return
}
return
}

ok := yield(tfprotov6.WriteStateByteChunk{
Bytes: chunk.Bytes,
TotalLength: chunk.TotalLength,
Range: tfprotov6.StateByteRange{
Start: chunk.Range.Start,
End: chunk.Range.End,
},
})
if !ok {
return
}
Copy link
Member

@SarahFrench SarahFrench Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been looking at this part and how to handle the errors encountered here. I think we can use the iter.Seq2 type to just return the errors to the calling code that uses the iterator, and the error can be handled there.

This would mean updating the WriteStateBytesStream struct:

type WriteStateBytesStream struct {
-	Chunks iter.Seq[WriteStateByteChunk]
+	Chunks iter.Seq2[WriteStateByteChunk, error]
}

This allows the iterator to return two values when it's used with range, and the calling code will need to handle when the err isn't nil.

for chunk, err := range stream.Chunks {
    if err != nil {
        // handle error.
        // The io.EOF is never returned from the iterator;
        // when io.EOF happens it stops the for loop.
    }
    
    // Use chunk value...
}

Changes to the iterator would look like this:

			if err != nil {
-				// attempt to send the error back to client
-				msgErr := srv.SendMsg(&tfplugin6.WriteStateBytes_Response{
-					Diagnostics: toproto.Diagnostics([]*tfprotov6.Diagnostic{
-						{
-							Severity: tfprotov6.DiagnosticSeverityError,
-							Summary:  "Writing state chunk failed",
-							Detail: fmt.Sprintf("Attempt to write a byte chunk of state %q to %q failed: %s",
-								chunk.StateId, chunk.TypeName, err),
-						},
-					}),
-				})
-				if msgErr != nil {
-					err := status.Error(codes.Unimplemented, "ProviderServer does not implement WriteStateBytes")
-					logging.ProtocolError(ctx, err.Error())
-					return
-				}
-				return
+				// A non-sentinel error has ocurred
+				logging.ProtocolError(ctx, fmt.Sprintf(
+					"WriteStateBytes experienced an error when receiving state data from Terraform: %s",
+					err),
				)
			}

			ok := yield(tfprotov6.WriteStateByteChunk{
				Bytes:       chunk.Bytes,
				TotalLength: chunk.TotalLength,
				Range: tfprotov6.StateByteRange{
					Start: chunk.Range.Start,
					End:   chunk.Range.End,
				},
+			}, err)
-			})
			if !ok {
				return
			}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After experimenting with this yesterday I found this blog, which I found helpful: https://www.dolthub.com/blog/2024-07-12-golang-range-iters-demystified. The 'handling sentinel errors' section in particular.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants