Skip to content

Add protobuf codecs for FFI physical plans #13309

@timsaucer

Description

@timsaucer

Is your feature request related to a problem or challenge?

#12920 added FFI Physical Plans. These cannot be encoded or decoded with the DefaultPhysicalExtensionCodec. We cannot serialize these plans that include foreign elements. I would like to add a way to provide for serialization/deserialization across the FFI boundary.

Describe the solution you'd like

Add an additional struct along the lines of FFI_PhysicalExtensionCodec that implements PhysicalExtensionCodec.

Describe alternatives you've considered

None that I can think of, other than to find work arounds to not serialize these plans.

Additional context

This is needed to support adding a FFI_QueryPlanner so that we can do things like use a different library to generate physical plans. The use case I currently have in mind is datafusion-ray where we want to make a query planner that can automatically distribute work across multiple nodes in a cluster.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions