Wrong inspect.getsource for datetime

[asmeurer]

BPO	32313
Nosy	@abalkin, @asmeurer, @1st1, @srinivasreddy, @pganssle, @corona10, @tirkarthi, @iritkatriel

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-12-13.21:46:39.074>
labels = ['3.7', '3.8', 'type-bug', 'library']
title = 'Wrong inspect.getsource for datetime'
updated_at = <Date 2020-08-27.21:16:51.918>
user = 'https://github.com/asmeurer'

bugs.python.org fields:

activity = <Date 2020-08-27.21:16:51.918>
actor = 'p-ganssle'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-12-13.21:46:39.074>
creator = 'Aaron.Meurer'
dependencies = []
files = []
hgrepos = []
issue_num = 32313
keywords = []
message_count = 6.0
messages = ['308255', '325495', '375821', '375912', '375932', '376014']
nosy_count = 8.0
nosy_names = ['belopolsky', 'Aaron.Meurer', 'yselivanov', 'thatiparthy', 'p-ganssle', 'corona10', 'xtreak', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32313'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[asmeurer]

inspect.getsource(datetime) shows the Python source code for datetime, even when it is the C extension. This is very confusing.

I believe it's because _datetime is used to override everything in datetime at the end of the file (here

cpython/Lib/datetime.py

Line 2285 in 11a247d

from _datetime import __doc__

), but __file__ is not imported.

[pganssle]

I think this is mostly the correct behavior (though it may indeed be confusing). Note that datetime.py is the source of the module datetime, it's just that most of the code in there is shadowed by the line you've linked.

If you try and get the source of individual methods that were imported from _datetime, you'll get the expected failure:

>>> inspect.getsource(datetime.datetime.fromisoformat)
-- Long traceback --
TypeError: module, class, method, function, traceback, frame, or code object was expected, got builtin_function_or_method

That said, inspect.getsource does seem to be erroneously using the Python source for classes, e.g. print(inspect.getsource(datetime.date))

This is consistent with the behavior of functools, where the Python code for functools.partial, even when the C implementation is used.

Not sure if this is something that should be a warning, an exception or if the behavior should simply be documented in the inspect documentation.

I'll note that both inspect.getsource(datetime) and inspect.getsource(datetime.date) were IOError exceptions in Python 2.7, but have been returning the Python source code since at least Python 3.4.

[iritkatriel]

The documentation for getfile says "This will fail with a TypeError if the object is a built-in module, class, or function."

https://docs.python.org/3/library/inspect.html#inspect.getfile

But it doesn't:
>>> inspect.getfile(datetime.datetime)
'C:\\Users\\User\\src\\cpython\\lib\\datetime.py'

[srinivasreddy]

Looks like for getfile docs should be updated. On python2.7.17_1,

  import inspect, datetime
  print(inspect.getfile(datetime.datetime)) outputs, 

/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/datetime.so

It seems a long time overdue?

[corona10]

_datetime.datetime's tp_name is datetime.datetime so
inspect module guess that _datetime.datetime's module name is datetime, not _datetime.
I don't think we have to fix this issue from inspect module side.

If the _datetime.datetime's tp_name is updated to _datetime.datetime it could be fixed but I don't know this is the right approach and there might be some reason for choosing tp_name as datetime.datetime instead of _datetime.datetime.

@p-ganssle is the best expert for datetime module :)
I think that he could give the answer to this.

[pganssle]

I think that we should re-examine this issue after #64671 is merged. I'm not really sure how that will affect this and indeed *how* it should affect this. I am not sure whether people are relying on the current behavior, or what use cases would be improved if we had a different behavior.

With regards to this:

The documentation for getfile says "This will fail with a TypeError if the object is a built-in module, class, or function."

https://docs.python.org/3/library/inspect.html#inspect.getfile

This is a bit unclear to me, and I'm not entirely sure if datetime qualifies. I think of built-in classes as things like int and float, and built-in functions as things like abs and sum, and datetime is an extension module — albeit one with a C implementation, and one that is in the standard library.

We should probably clarify the wording of inspect.getsource and determine what the intended semantics are for PEP-399-style modules, with both a C and pure Python implementation and the C implementation is what's being used. Error? Point to the Python implementation?

[merwok]

I think of built-in classes as things like int and float, and built-in functions as things like abs and sum, and datetime is an extension module

In many places (source code or docs), built-in is used not to mean «global function in the builtins module» but «defined in an extension module». You can see Guido referring to datetime classes as built-in here for example: #88784 (comment)