Custom serializer is not transfered to the workers #5561
Comments
|
Yeah unfortunately this doesn't work as well as we might like today. Basically what one would need to do is write this registration code and put it in some module that one imports on all Workers (running something like Dask has a bunch of serializers that get registered in this Issue ( #3831 ) discusses how we could better handle this registration of 3rd party serializes (possibly with entrypoints). |
|
Legit. The docs could maybe use a cleanup then since it's not very clear from the docs that you need to add this registration. The current easy workaround is to just extend the class and add custom pickling getstate/setstate but it's a bit of a pain when your wrapping someone elses library. |
|
Yeah that's a fair criticism. If you have thoughts on how this should be worded/where it should go, please feel free to share. Yep I think improvements on the pickling side of the story have made this less pressing. For example we handle out-of-band pickling, which avoids additional copies that older pickle protocols incurred. This combined with the fact that NumPy supports this protocol ( numpy/numpy#12011 ) really gives one a lot of mileage with pickle already. For example Pandas use of NumPy makes things just work ( pandas-dev/pandas#34244 ). Maybe you already know this and it doesn't work for some reason, but one could register a pickler for a 3rd party object. If that doesn't work though, would be curious to know more 🙂 |
|
Oh interesting, although I suspect that it also wouldn't get picked up and sent through to the workers unless the registering manipulates the running class. |
|
Hmm...yeah we don't have a dispatch table shared between Workers. Though this is an interesting idea in its own right |
What happened:
I got an exception:
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/distributed/protocol/core.py", line 111, in loads
return msgpack.loads(
File "msgpack/_unpacker.pyx", line 195, in msgpack._cmsgpack.unpackb
File "/opt/conda/lib/python3.8/site-packages/distributed/protocol/core.py", line 103, in _decode_default
return merge_and_deserialize(
File "/opt/conda/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 475, in merge_and_deserialize
return deserialize(header, merged_frames, deserializers=deserializers)
File "/opt/conda/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 407, in deserialize
return loads(header, frames)
File "/opt/conda/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 170, in serialization_error_loads
raise TypeError(msg)
TypeError: Could not serialize object of type ConnectionClass.
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/distributed/protocol/pickle.py", line 49, in dumps
result = pickle.dumps(x, **dump_kwargs)
_pickle.PicklingError: Can't pickle <class 'main.ConnectionClass'>: attribute lookup ConnectionClass on main failed
What you expected to happen:
Get the result
Minimal Complete Verifiable Example:
Anything else we need to know?:
Environment:
Packages are:
Ubuntu
The text was updated successfully, but these errors were encountered: