bpo-46425: fix direct invocation of test_importlib

[sobolevn]

Multiple tests in test_importlib were not suited to direct invocation. Why?

• Some tests did miss unittest.main() call
• Some tests used relative import style, here's how it were:

» ./python.exe ./Lib/test/test_importlib/test_open.py
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/./Lib/test/test_importlib/test_open.py", line 4, in <module>
    from . import data01
    ^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

In several places relative imports were used in setUp(). It was also problematic:

» ./python.exe ./Lib/test/test_importlib/test_open.py
..................../Users/sobolev/Desktop/cpython/./Lib/test/test_importlib/test_open.py:71: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  from . import namespacedata01
EEEEEE............
======================================================================
ERROR: test_open_binary (__main__.OpenDiskNamespaceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/./Lib/test/test_importlib/test_open.py", line 71, in setUp
    from . import namespacedata01
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

======================================================================
ERROR: test_open_binary_FileNotFoundError (__main__.OpenDiskNamespaceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/./Lib/test/test_importlib/test_open.py", line 71, in setUp
    from . import namespacedata01
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

======================================================================
ERROR: test_open_text_FileNotFoundError (__main__.OpenDiskNamespaceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/./Lib/test/test_importlib/test_open.py", line 71, in setUp
    from . import namespacedata01
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

======================================================================
ERROR: test_open_text_default_encoding (__main__.OpenDiskNamespaceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/./Lib/test/test_importlib/test_open.py", line 71, in setUp
    from . import namespacedata01
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

======================================================================
ERROR: test_open_text_given_encoding (__main__.OpenDiskNamespaceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/./Lib/test/test_importlib/test_open.py", line 71, in setUp
    from . import namespacedata01
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

======================================================================
ERROR: test_open_text_with_errors (__main__.OpenDiskNamespaceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/./Lib/test/test_importlib/test_open.py", line 71, in setUp
    from . import namespacedata01
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

----------------------------------------------------------------------
Ran 38 tests in 0.017s

FAILED (errors=6)

Now, all test_importlib modules pass the local check 🎉

Size of the PR

This PR is quite big. If desireable, I can split it into multiple parts.

CC @corona10 as my mentor.

https://bugs.python.org/issue46425

[brettcannon]

So I'm of two minds on this. One, this was actually done on purpose as the expectation is to run the tests via -m test which does not have any of these issues, e.g. python -m test test_importlib.test_open. And so the entire test suite is kind of viewed as a single unit. Plus I prefer relative imports. 😁

But none of this is also important, so I don't object to this PR either. 🙂 I'll let @corona10 or someone else handle reviewing and merging.

[ericsnowcurrently]

I'm not quite sure what the objective here is. Is it only so you can run any of the files directly as a script? For something as extensive as the importlib test suite, what's the advantage of running individual modules as scripts? Is this something someone needs or is your goal only consistency? (We generally discourage blanket changes motivated by a desire for consistency, due to the costs of churn in a large project like this.)

Almost always, you would want to run all of the importlib tests:

• ./python -m test test_importlib
• ./python -m unittest test.test_importlib
• (rarely) ./python -m test.test_importlib

If you are actually working on importlib and want to run a specific subset of the tests then you'd usually use "-m", for example:

• ./python -m unittest test.test_importlib.builtin.test_loader
• (specific test) ./python -m unittest test.test_importlib.builtin.test_loader.LoaderTests.test_module (which you can't do as cleanly when invoking with a script)

So I'd be interested in hearing the use case for running one of the individual importlib test files as a script. Also, you should definitely get @brettcannon's feedback since he wrote nearly all these tests and likely has a clear opinion on the matter.

In case there is a good enough use case for run-as-script then the change mostly LGTM. I've left some comments noting files that should be left alone.

[sobolevn]

Thanks, Brett.

I think that the main thing here is that most modules are working fine and only some are broken when invoked directly as python Lib/test/*. It might be confusing.

[brettcannon]

I think that the main thing here is that most modules are working fine and only some are broken when invoked directly as python Lib/test/*. It might be confusing.

That's fair, but I want to be clear that nothing is "broken". There is no general rule that sub-tests need to function as separate execution targets. As I have said, I don't object to the change specifically, but I'm also not going to be the one who merges it either. 😁

[terryjreedy]

For me, on Windows, with fresh 3.11, python -m test.test_importlib.test_open runs the open tests. python Lib/test/test_import/test_open.py (or py ...) in a directory containing Lib fails with the import error above. If I load test_open in an IDLE editor and run it (F5), it also fails, similarly. To me, this is broken. I would feel the same if using any other editor that can run a saved file with an appropriate program.

[sobolevn]

That's fair, but I want to be clear that nothing is "broken".

Yeah, this word might a bit harsh. Let's not use it 🙂
Instead, I am going to illustrate my motivation behind this change.

Let's say I am newcomer to CPython's source code (which I +- am). And I want to work on some feature/bugfix in importlib. I obviously need to run my tests. Let's say I need Lib/test/test_importlib/builtin/test_finder.py. I read this file and I see unittest.main() in the end. Oh, I know what it does!

cpython/Lib/test/test_importlib/builtin/test_finder.py

Lines 97 to 98 in cfadcc3

	if __name__ == '__main__':
	unittest.main()

But, if I try to run it directly (which this module declare to support) - tests will fail.

I think, it can be improved! And an it is easy to do: change imports, add unittest.main() in some files where it was missing. Done ✨

[jaraco]

This change causes problems within the backports (importlib_metadata, importlib_resources). This code is kept in sync with those projects and thus (a) relies on the relative imports to find the fixtures in a relative location and (b) would never expect the modules to be invoked as a script. As primary maintainer of importlib.metadata and importlib.resources, I'd prefer to keep the code portable and simple (encapsulated). It's conceivable that the backport could maintain a forked version of these files that doesn't include these behaviors, but that adds a cognitive and maintenance burden to the effort.

In my effort, the lack of __name__ == '__main__' was intentional and represented a signal that these files were not meant to be executed, but were meant to be invoked through a test runner (i.e. python -m test, python regrtest, python -m unittest discover, or pytest (in the case of the backports)). I'd like to restore that expectation and back out these changes for tests relating to importlib.metadata and importlib.resources. I have less of an opinion on the changes to other importlib modules.

[sobolevn]

@jaraco thanks! I will send a new PR with restored changes in importlib_metadata and importlib_resources in 10 mins.

[jaraco]

@jaraco thanks! I will send a new PR with restored changes in importlib_metadata and importlib_resources in 10 mins.

Thanks sobolevn. I missed your message prior to preparing and submitting #30799. Since it's not clear the demarcation of the tests, I figured it would be better if I helped out with that part. I went through and flagged what I think are the relevant files. I'm going to use that PR to validate that I got the correct files as well (by performing a dry-run sync into the backports).