DateTimeField should be able to keep original timezone info

[favll]

Checklist

• I have verified that that issue exists against the master branch of Django REST framework.
• I have searched for similar issues in both open and closed tickets and cannot find a duplicate.
• This is not a usage question. (Those should be directed to the discussion group instead.)
• This cannot be dealt with as a third party library. (We prefer new functionality to be in the form of third party libraries where possible.)
• I have reduced the issue to the simplest possible case.
• I have included a failing test as a pull request. (If you are unable to do so we can still accept the issue.)

Steps to reproduce / Actual behavior

DRF's DateTimeField has no option to keep a datetime's original timezone information during serialization/deserialization. Here a serialization example:

0. Set USE_TZ = True and TIME_ZONE = "UTC".
1. Consider the following datetime object: dt = datetime.datetime(2018, 9, 4, 7, 30, tzinfo=<DstTzInfo 'Europe/Berlin' CEST+2:00:00 DST>).
2. Serialize it with serializers.DateTimeField().to_representation(dt)

This yields '2018-09-04T05:30:00Z' which represents the datetime in the UTC timezone.

Expected behavior

The serialized datetime is correct, however, does not contain any information about the original timezone. DRF's DateTimeField should provide an option to preserve this timezone info by instead serializing to: '2018-09-04T07:30:00+02:00'.

Why?

While the returned string is correct, there are use cases in which a frontend (API consumer) might want to display the datetime in the timezone it was originally in. For example, a flight ticket search engine generally displays any datetimes in the local timezone of the airport. So instead of displaying the departure time of a flight in UTC it generally displays it in the timezone of the airport. This is however not possible with the current implementation of DateTimeField.

Possible Solution

Would the maintainers be open to a PR that adds an option to the DateTimeField that would allow it to deserialize / serialize datetimes while keeping their original datetime? I can go into depth about other use cases and possible implementations, however, I would like to first know if this idea is welcome in general.

This problem was already mentioned in #3732 and generally well received ("I’m really happy to take a PR on this."), however, the issue originally addressed another problem and was subsequently closed before the issue I am mentioning here was addressed.

[carltongibson]

Ref #5435 and related.

[agalazis]

to me not being able to store timezone is like losing data as an event happens in a specific timezone. This could be stored in an all in one fashion but limitations imply saving event timezone elsewhere

[georgek]

Agreed, this causes data loss by converting the timestamp into the timezone of the server.

To expand on the issue, consider the deserialisation case:

DateTimeField().to_internal_value("2020-01-21T17:57:54+05:00")

This gives me:

datetime.datetime(2020, 1, 21, 12, 57, 54, tzinfo=<DstTzInfo 'Europe/London' GMT0:00:00 STD>)

But I would expect something more like:

datetime.datetime(2020, 1, 21, 17, 57, 54, tzinfo=datetime.timezone(datetime.timedelta(seconds=18000)))

These are equal, but the latter has preserved the submitted timezone information, which might be important.

[xordoquy]

I believe this would be a nice thing to have... but I suspect the implementation will not be possible out of the box.
To my understanding, Django converts timezone aware DateTime into UTC (if USE_TZ is set) to store it as such in PostgresQL. That means that the TZ info must be stored in a second field or Django implementation should be updated which places the change out of the core boundaries.
Am I missing something ?

[georgek]

It looks like that is case, yes. But all supported versions of Postgres support a timestamp with timezone type at no extra cost compared to a regular timestamp. I have a feeling Django has this behaviour to support other DBs that don't support storing the timezone.

In any case, it does look like an upstream issue rather than a DRF issue.

Edit: Turns out it's postgres itself turning the timestamp into UTC. The TIMESTAMP WITH TIME ZONE type merely accepts a timestamp with timezone, but stores it internally as UTC and forgets the offset that was provided. For other DBs Django does this itself.

After some toying around seeing if this is fixable I've realised that the best way to solve the problem here is to store the Olson time zone (not just the offset) in its own field with the UTC timestamp. I doubt this could realistically be abstracted in either Django or DRF given the flexibility of the datetime and timezone types in Python, though.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.