Can't cache a ReturnDict.

[StevenVeshkini]

Has anyone been able to cache serializer.data from a custom Serializer on one object? Seems like there's no way to store the ReturnDict that results from serializer.data.

[lovelydinosaur]

Could you also include a concrete use case to work against. In get the question generally, but it'd be helpful to see the specific case you're trying to achieve.

[lovelydinosaur]

Also what error is raised?

[xordoquy]

It seems that the ReturnDict having a link to the serializer breaks the pickling:

>>> from rest_framework import serializers
>>> class NestedSerializer(serializers.Serializer):
...             one = serializers.IntegerField(max_value=10)
...             two = serializers.IntegerField(max_value=10)
... 
>>> class TestSerializer(serializers.Serializer):
...             nested = NestedSerializer()
... 
>>> input_data = {
...             'nested': {
...                 'one': 1,
...                 'two': 2,
...             }
...         }
>>> s = TestSerializer(input_data)
>>> s.data
ReturnDict([('nested', OrderedDict([('one', 1), ('two', 2)]))])
>>> d = s.data
>>> import pickle
>>> pickle.dumps(d)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
_pickle.PicklingError: Can't pickle <class 'TestSerializer'>: attribute lookup TestSerializer on builtins failed
>>> pickle.dumps({'nested': [{'one': 1, 'two': 2}]})
b'\x80\x03}q\x00X\x06\x00\x00\x00nestedq\x01]q\x02}q\x03(X\x03\x00\x00\x00twoq\x04K\x02X\x03\x00\x00\x00oneq\x05K\x01uas.'

[xordoquy]

Note that Django uses pickle to cache things.

[StevenVeshkini]

@xordoquy Yes, that is the exact problem I am having (nested serializers).

[xordoquy]

It was a bit late yesterday. I'll add a proper test case this evening.

[StevenVeshkini]

@xordoquy Is this a bug or was this the way DRF was designed?

[xordoquy]

It's both ;)

[StevenVeshkini]

hahahaha nice :P

[StevenVeshkini]

Any luck?

[lovelydinosaur]

So the right way to resolve this would be either...

• Make serializer instances picklable, which oughta resolve this.

Or...

• Make ReturnDict's pickle representation not include the serializer backlink. Given that we render the .content that oughta be okay in most cases anyways, not clear if accessing response.data from cached responses is something that should happen or not.

My personal take is that using pickle to deal with HTTP caching is a wonky approach. Rather store the raw headers and content, and return a newly created response instance when returning cached content. (Then you don't bump into all this horrible sorts of cases.)