Skip to content

importlib_resources/tests/test_functional.py test failures on big endian architectures #312

Closed
@mgorny

Description

@mgorny

When running the tests on big-endian architectures (sparc64, ppc64), we're getting the followin test failures:

$ tox -e py312 
.pkg-cpython312: recreate env because dependencies removed: setuptools>=56
.pkg-cpython312: remove tox env folder /home/mgorny/importlib_resources/.tox/.pkg-cpython312
.pkg-cpython312: install_requires> python -I -m pip install 'setuptools>=61.2' 'setuptools_scm[toml]>=3.4.1'
.pkg-cpython312: _optional_hooks> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-cpython312: get_requires_for_build_editable> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-cpython312: build_editable> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py312: recreate env because dependencies removed: pytest-mypy; platform_python_implementation != "PyPy", pytest>=6
py312: remove tox env folder /home/mgorny/importlib_resources/.tox/py312
py312: install_package_deps> python -I -m pip install 'jaraco.test>=5.4' 'pytest!=8.1.*,>=6' 'pytest-checkdocs>=2.4' pytest-cov 'pytest-enabler>=2.2' pytest-mypy 'pytest-ruff>=0.2.1' 'zipp>=3.1.0; python_version < "3.10"' 'zipp>=3.17'
py312: install_package> python -I -m pip install --force-reinstall --no-deps /home/mgorny/importlib_resources/.tox/.tmp/package/10/importlib_resources-6.4.1.dev23+gbe5a270-0.editable-py3-none-any.whl
py312: commands[0]> pytest
========================================================= test session starts =========================================================
platform linux -- Python 3.12.3, pytest-8.2.2, pluggy-1.5.0
cachedir: .tox/py312/.pytest_cache
rootdir: /home/mgorny/importlib_resources
configfile: pytest.ini
plugins: enabler-3.1.1, mypy-0.10.3, checkdocs-2.13.0, cov-5.0.0, jaraco.test-5.4.0, ruff-0.4.0, typeguard-4.3.0
collected 313 items                                                                                                                   

docs/conf.py ....                                                                                                               [  1%]
importlib_resources/__init__.py ...                                                                                             [  2%]
importlib_resources/_adapters.py ...                                                                                            [  3%]
importlib_resources/_common.py ....                                                                                             [  4%]
importlib_resources/_functional.py ...                                                                                          [  5%]
importlib_resources/_itertools.py ....                                                                                          [  6%]
importlib_resources/abc.py ...                                                                                                  [  7%]
importlib_resources/compat/__init__.py ...                                                                                      [  8%]
importlib_resources/compat/py38.py ...                                                                                          [  9%]
importlib_resources/compat/py39.py ...                                                                                          [ 10%]
importlib_resources/future/__init__.py ...                                                                                      [ 11%]
importlib_resources/future/adapters.py ...                                                                                      [ 12%]
importlib_resources/readers.py ...                                                                                              [ 13%]
importlib_resources/simple.py ...                                                                                               [ 14%]
importlib_resources/tests/__init__.py ...                                                                                       [ 15%]
importlib_resources/tests/_path.py ....                                                                                         [ 16%]
importlib_resources/tests/compat/__init__.py ...                                                                                [ 17%]
importlib_resources/tests/compat/py312.py ...                                                                                   [ 18%]
importlib_resources/tests/compat/py39.py ...                                                                                    [ 19%]
importlib_resources/tests/data01/__init__.py ...                                                                                [ 20%]
importlib_resources/tests/data01/subdirectory/__init__.py ...                                                                   [ 21%]
importlib_resources/tests/data02/__init__.py ...                                                                                [ 22%]
importlib_resources/tests/data02/one/__init__.py ...                                                                            [ 23%]
importlib_resources/tests/data02/two/__init__.py ...                                                                            [ 24%]
importlib_resources/tests/test_compatibilty_files.py ...................                                                        [ 30%]
importlib_resources/tests/test_contents.py ......                                                                               [ 32%]
importlib_resources/tests/test_custom.py ....                                                                                   [ 33%]
importlib_resources/tests/test_files.py .........................                                                               [ 41%]
importlib_resources/tests/test_functional.py .......F..F.....F..F.                                                              [ 48%]
importlib_resources/tests/test_open.py ...........................................                                              [ 61%]
importlib_resources/tests/test_path.py ................                                                                         [ 67%]
importlib_resources/tests/test_read.py .......................................                                                  [ 79%]
importlib_resources/tests/test_reader.py ..................                                                                     [ 85%]
importlib_resources/tests/test_resource.py ....................s..................                                              [ 97%]
importlib_resources/tests/util.py ...                                                                                           [ 98%]
importlib_resources/tests/zip.py ...                                                                                            [ 99%]
. .                                                                                                                             [100%]

============================================================== FAILURES ===============================================================
____________________________________________ FunctionalAPITest_StringAnchor.test_open_text ____________________________________________

self = <importlib_resources.tests.test_functional.FunctionalAPITest_StringAnchor testMethod=test_open_text>

    def test_open_text(self):
        with resources.open_text(self.anchor01, 'utf-8.file') as f:
            self.assertEqual(f.read(), 'Hello, UTF-8 world!\n')
        for path_parts in self._gen_resourcetxt_path_parts():
            with resources.open_text(
                self.anchor02,
                *path_parts,
                encoding='utf-8',
            ) as f:
                self.assertEqual(f.read(), 'a resource')
        # Use generic OSError, since e.g. attempting to read a directory can
        # fail with PermissionError rather than IsADirectoryError
        with self.assertRaises(OSError):
            resources.open_text(self.anchor01)
        with self.assertRaises(OSError):
            resources.open_text(self.anchor01, 'no-such-file')
        with resources.open_text(self.anchor01, 'utf-16.file') as f:
            with self.assertRaises(UnicodeDecodeError):
                f.read()
        with resources.open_text(
            self.anchor01,
            'binary.file',
            encoding='latin1',
        ) as f:
            self.assertEqual(f.read(), '\x00\x01\x02\x03')
        with resources.open_text(
            self.anchor01,
            'utf-16.file',
            errors='backslashreplace',
        ) as f:
>           self.assertEqual(
                f.read(),
                'Hello, UTF-16 world!\n'.encode('utf-16').decode(
                    errors='backslashreplace',
                ),
            )
E           AssertionError: '\\xff\\xfeH\x00e\x00l\x00l\x00o\x00,\x00 \x00[67 chars]\x00' != '\\xfe\\xff\x00H\x00e\x00l\x00l\x00o\x00,\x00 [67 chars]00\n'
E           - \xff\xfeHello, UTF-16 world!
E           ?     ^^^^
E           + \xfe\xffHello, UTF-16 world!
E           ? ++++    ^
E           - 
E           +

importlib_resources/tests/test_functional.py:131: AssertionError
____________________________________________ FunctionalAPITest_StringAnchor.test_read_text ____________________________________________

self = <importlib_resources.tests.test_functional.FunctionalAPITest_StringAnchor testMethod=test_read_text>

    def test_read_text(self):
        self.assertEqual(
            resources.read_text(self.anchor01, 'utf-8.file'),
            'Hello, UTF-8 world!\n',
        )
        self.assertEqual(
            resources.read_text(
                self.anchor02,
                'subdirectory',
                'subsubdir',
                'resource.txt',
                encoding='utf-8',
            ),
            'a resource',
        )
        for path_parts in self._gen_resourcetxt_path_parts():
            self.assertEqual(
                resources.read_text(
                    self.anchor02,
                    *path_parts,
                    encoding='utf-8',
                ),
                'a resource',
            )
        # Use generic OSError, since e.g. attempting to read a directory can
        # fail with PermissionError rather than IsADirectoryError
        with self.assertRaises(OSError):
            resources.read_text(self.anchor01)
        with self.assertRaises(OSError):
            resources.read_text(self.anchor01, 'no-such-file')
        with self.assertRaises(UnicodeDecodeError):
            resources.read_text(self.anchor01, 'utf-16.file')
        self.assertEqual(
            resources.read_text(
                self.anchor01,
                'binary.file',
                encoding='latin1',
            ),
            '\x00\x01\x02\x03',
        )
>       self.assertEqual(
            resources.read_text(
                self.anchor01,
                'utf-16.file',
                errors='backslashreplace',
            ),
            'Hello, UTF-16 world!\n'.encode('utf-16').decode(
                errors='backslashreplace',
            ),
        )
E       AssertionError: '\\xff\\xfeH\x00e\x00l\x00l\x00o\x00,\x00 \x00[67 chars]\x00' != '\\xfe\\xff\x00H\x00e\x00l\x00l\x00o\x00,\x00 [67 chars]00\n'
E       - \xff\xfeHello, UTF-16 world!
E       ?     ^^^^
E       + \xfe\xffHello, UTF-16 world!
E       ? ++++    ^
E       - 
E       +

importlib_resources/tests/test_functional.py:79: AssertionError
____________________________________________ FunctionalAPITest_ModuleAnchor.test_open_text ____________________________________________

self = <importlib_resources.tests.test_functional.FunctionalAPITest_ModuleAnchor testMethod=test_open_text>

    def test_open_text(self):
        with resources.open_text(self.anchor01, 'utf-8.file') as f:
            self.assertEqual(f.read(), 'Hello, UTF-8 world!\n')
        for path_parts in self._gen_resourcetxt_path_parts():
            with resources.open_text(
                self.anchor02,
                *path_parts,
                encoding='utf-8',
            ) as f:
                self.assertEqual(f.read(), 'a resource')
        # Use generic OSError, since e.g. attempting to read a directory can
        # fail with PermissionError rather than IsADirectoryError
        with self.assertRaises(OSError):
            resources.open_text(self.anchor01)
        with self.assertRaises(OSError):
            resources.open_text(self.anchor01, 'no-such-file')
        with resources.open_text(self.anchor01, 'utf-16.file') as f:
            with self.assertRaises(UnicodeDecodeError):
                f.read()
        with resources.open_text(
            self.anchor01,
            'binary.file',
            encoding='latin1',
        ) as f:
            self.assertEqual(f.read(), '\x00\x01\x02\x03')
        with resources.open_text(
            self.anchor01,
            'utf-16.file',
            errors='backslashreplace',
        ) as f:
>           self.assertEqual(
                f.read(),
                'Hello, UTF-16 world!\n'.encode('utf-16').decode(
                    errors='backslashreplace',
                ),
            )
E           AssertionError: '\\xff\\xfeH\x00e\x00l\x00l\x00o\x00,\x00 \x00[67 chars]\x00' != '\\xfe\\xff\x00H\x00e\x00l\x00l\x00o\x00,\x00 [67 chars]00\n'
E           - \xff\xfeHello, UTF-16 world!
E           ?     ^^^^
E           + \xfe\xffHello, UTF-16 world!
E           ? ++++    ^
E           - 
E           +

importlib_resources/tests/test_functional.py:131: AssertionError
____________________________________________ FunctionalAPITest_ModuleAnchor.test_read_text ____________________________________________

self = <importlib_resources.tests.test_functional.FunctionalAPITest_ModuleAnchor testMethod=test_read_text>

    def test_read_text(self):
        self.assertEqual(
            resources.read_text(self.anchor01, 'utf-8.file'),
            'Hello, UTF-8 world!\n',
        )
        self.assertEqual(
            resources.read_text(
                self.anchor02,
                'subdirectory',
                'subsubdir',
                'resource.txt',
                encoding='utf-8',
            ),
            'a resource',
        )
        for path_parts in self._gen_resourcetxt_path_parts():
            self.assertEqual(
                resources.read_text(
                    self.anchor02,
                    *path_parts,
                    encoding='utf-8',
                ),
                'a resource',
            )
        # Use generic OSError, since e.g. attempting to read a directory can
        # fail with PermissionError rather than IsADirectoryError
        with self.assertRaises(OSError):
            resources.read_text(self.anchor01)
        with self.assertRaises(OSError):
            resources.read_text(self.anchor01, 'no-such-file')
        with self.assertRaises(UnicodeDecodeError):
            resources.read_text(self.anchor01, 'utf-16.file')
        self.assertEqual(
            resources.read_text(
                self.anchor01,
                'binary.file',
                encoding='latin1',
            ),
            '\x00\x01\x02\x03',
        )
>       self.assertEqual(
            resources.read_text(
                self.anchor01,
                'utf-16.file',
                errors='backslashreplace',
            ),
            'Hello, UTF-16 world!\n'.encode('utf-16').decode(
                errors='backslashreplace',
            ),
        )
E       AssertionError: '\\xff\\xfeH\x00e\x00l\x00l\x00o\x00,\x00 \x00[67 chars]\x00' != '\\xfe\\xff\x00H\x00e\x00l\x00l\x00o\x00,\x00 [67 chars]00\n'
E       - \xff\xfeHello, UTF-16 world!
E       ?     ^^^^
E       + \xfe\xffHello, UTF-16 world!
E       ? ++++    ^
E       - 
E       +

importlib_resources/tests/test_functional.py:79: AssertionError
========================================================== warnings summary ===========================================================
docs/conf.py::mypy
  /home/mgorny/importlib_resources/.tox/py312/lib/python3.12/site-packages/mypy/config_parser.py:271: EncodingWarning: 'encoding' argument not specified
    config_parser.read(config_file)

docs/conf.py::mypy
  /home/mgorny/importlib_resources/.tox/py312/lib/python3.12/site-packages/mypy/modulefinder.py:851: EncodingWarning: 'encoding' argument not specified
    with open(versions_path) as f:

docs/conf.py::mypy
docs/conf.py::mypy
docs/conf.py::mypy
docs/conf.py::mypy
  /home/mgorny/importlib_resources/.tox/py312/lib/python3.12/site-packages/mypy/metastore.py:95: EncodingWarning: 'encoding' argument not specified
    with open(os.path.join(self.cache_dir_prefix, name)) as f:

docs/conf.py: 13 warnings
  /home/mgorny/importlib_resources/.tox/py312/lib/python3.12/site-packages/mypy/metastore.py:108: EncodingWarning: 'encoding' argument not specified
    with open(tmp_filename, "w") as f:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.12.3-final-0 -----------
Name                                                        Stmts   Miss  Cover   Missing
-----------------------------------------------------------------------------------------
docs/conf.py                                                   12      0   100%
importlib_resources/__init__.py                                 4      0   100%
importlib_resources/_adapters.py                               90      2    98%   58, 157
importlib_resources/_common.py                                101      5    95%   68-72
importlib_resources/_functional.py                             32      0   100%
importlib_resources/abc.py                                     66      8    88%   27, 40, 48, 53, 162, 165, 168, 171
importlib_resources/compat/__init__.py                          0      0   100%
importlib_resources/compat/py38.py                              6      1    83%   11
importlib_resources/compat/py39.py                              5      1    80%   10
importlib_resources/future/__init__.py                          0      0   100%
importlib_resources/future/adapters.py                         42      3    93%   37, 83-84
importlib_resources/readers.py                                106      9    92%   29, 42-45, 52-53, 188-194
importlib_resources/tests/__init__.py                           0      0   100%
importlib_resources/tests/compat/__init__.py                    0      0   100%
importlib_resources/tests/compat/py39.py                        3      0   100%
importlib_resources/tests/compat/py312.py                       9      4    56%   11-15
importlib_resources/tests/data01/__init__.py                    0      0   100%
importlib_resources/tests/data01/subdirectory/__init__.py       0      0   100%
importlib_resources/tests/data02/__init__.py                    0      0   100%
importlib_resources/tests/data02/one/__init__.py                0      0   100%
importlib_resources/tests/data02/two/__init__.py                0      0   100%
importlib_resources/tests/test_compatibilty_files.py           67      0   100%
importlib_resources/tests/test_contents.py                     19      0   100%
importlib_resources/tests/test_custom.py                       29      0   100%
importlib_resources/tests/test_files.py                        67      1    99%   117
importlib_resources/tests/test_functional.py                  109      2    98%   7-9
importlib_resources/tests/test_open.py                         58      1    98%   89
importlib_resources/tests/test_path.py                         37      1    97%   65
importlib_resources/tests/test_read.py                         51      1    98%   97
importlib_resources/tests/test_reader.py                       94      1    99%   145
importlib_resources/tests/test_resource.py                    128      1    99%   241
importlib_resources/tests/util.py                              91      5    95%   32-35, 51
importlib_resources/tests/zip.py                               20      0   100%
-----------------------------------------------------------------------------------------
TOTAL                                                        1246     46    96%

================================================================ mypy =================================================================
Success: no issues found in 36 source files
======================================================= short test summary info =======================================================
FAILED importlib_resources/tests/test_functional.py::FunctionalAPITest_StringAnchor::test_open_text - AssertionError: '\\xff\\xfeH\x00e\x00l\x00l\x00o\x00,\x00 \x00[67 chars]\x00' != '\\xfe\\xff\x00H\x00e\x00l\x00l\x00o\x00,\x00 [67...
FAILED importlib_resources/tests/test_functional.py::FunctionalAPITest_StringAnchor::test_read_text - AssertionError: '\\xff\\xfeH\x00e\x00l\x00l\x00o\x00,\x00 \x00[67 chars]\x00' != '\\xfe\\xff\x00H\x00e\x00l\x00l\x00o\x00,\x00 [67...
FAILED importlib_resources/tests/test_functional.py::FunctionalAPITest_ModuleAnchor::test_open_text - AssertionError: '\\xff\\xfeH\x00e\x00l\x00l\x00o\x00,\x00 \x00[67 chars]\x00' != '\\xfe\\xff\x00H\x00e\x00l\x00l\x00o\x00,\x00 [67...
FAILED importlib_resources/tests/test_functional.py::FunctionalAPITest_ModuleAnchor::test_read_text - AssertionError: '\\xff\\xfeH\x00e\x00l\x00l\x00o\x00,\x00 \x00[67 chars]\x00' != '\\xfe\\xff\x00H\x00e\x00l\x00l\x00o\x00,\x00 [67...
================================== 4 failed, 308 passed, 1 skipped, 19 warnings in 64.12s (0:01:04) ===================================
py312: exit 1 (67.31 seconds) /home/mgorny/importlib_resources> pytest pid=1199072
  py312: FAIL code 1 (135.61=setup[68.30]+cmd[67.31] seconds)
  evaluation failed :( (136.06 seconds)

Reproduced with be5a270. I'll submit a PR shortly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions