Conflict with pyfakefs ?

[gsemet]

I use pyfakefs and since pytest-html 4+, I have this issue

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/main.py", line 271, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/main.py", line 325, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/main.py", line 350, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 114, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 133, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 226, in call_and_report
INTERNALERROR>     hook.pytest_runtest_logreport(report=report)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 113, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pytest_html/basereport.py", line 253, in pytest_runtest_logreport
INTERNALERROR>     self._generate_report()
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pytest_html/selfcontained_report.py", line 39, in _generate_report
INTERNALERROR>     super()._generate_report(self_contained=True)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pytest_html/basereport.py", line 52, in _generate_report
INTERNALERROR>     rendered_report = self._template.render(
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
INTERNALERROR>     self.environment.handle_exception()
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
INTERNALERROR>     raise rewrite_traceback_stack(source=source)
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pytest_html/resources/index.jinja2", line 117, in top-level template code
INTERNALERROR>     {% include "app.js" %}
INTERNALERROR>   File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/jinja2/loaders.py", line 218, in get_source
INTERNALERROR>     raise TemplateNotFound(template)
INTERNALERROR> jinja2.exceptions.TemplateNotFound: app.js
Traceback (most recent call last):
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 1632, in get_object_from_normpath
    target = target.get_entry(component)  # type: ignore
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_file.py", line 542, in get_entry
    return self.entries[to_string(pathname_name)]
KeyError: 'Users'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 169, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/main.py", line 318, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/main.py", line 303, in wrap_session
    os.chdir(session.startpath)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_os.py", line 1356, in wrapped
    return f(*args, **kwargs)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_os.py", line 421, in chdir
    self.filesystem.confirmdir(path)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 2824, in confirmdir
    self.resolve(target_directory, check_owner=check_owner),
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 1706, in resolve
    return self.get_object_from_normpath(
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 1641, in get_object_from_normpath
    self.raise_os_error(errno.ENOENT, path)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 428, in raise_os_error
    raise OSError(err_no, message, filename)
FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: '/Users/username/Projects/some/project'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/username/Projects/some/project/.venv/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 192, in console_main
    code = main()
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 177, in main
    config._ensure_unconfigure()
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1058, in _ensure_unconfigure
    self.hook.pytest_unconfigure(config=self)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/_pytest/faulthandler.py", line 39, in pytest_unconfigure
    os.close(config.stash[fault_handler_stderr_fd_key])
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_os.py", line 1356, in wrapped
    return f(*args, **kwargs)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_os.py", line 308, in close
    file_handle = self.filesystem.get_open_file(fd)
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 867, in get_open_file
    self.raise_os_error(errno.EBADF, str(file_des))
  File "/Users/username/Projects/some/project/.venv/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py", line 428, in raise_os_error
    raise OSError(err_no, message, filename)
OSError: [Errno 9] Bad file descriptor in the fake filesystem: '9'
make: *** [test] Error 1

I fix by restricting to pytest-html<4.

[RonnyPfannschmidt]

this seems to be a bug in pyfakefs not shutting down correctly

[BeyondEvil]

How are you installing pytest-html?

I can see this in the output: "INTERNALERROR> jinja2.exceptions.TemplateNotFound: app.js"

That shouldn't happen if installing using pip.

[gsemet]

I am using poetry that uses pip underneath

[BeyondEvil]

I am using poetry that uses pip underneath

Hmm... that shouldn't matter. 🤔

app.js is generated when creating the dist that is then uploaded to pypi so it should be present.

I'm not super versed in all the nuances w.r.t. to build-systems, so it's entirely possible I've made a mistake some where.

Relevant bits:

https://github.com/pytest-dev/pytest-html/blob/master/pyproject.toml#L98-L99

https://github.com/pytest-dev/pytest-html/blob/master/scripts/npm.py

Any ideas welcome @RonnyPfannschmidt 🙇

[RonnyPfannschmidt]

@BeyondEvil as per the traceback a jinja template rendering wants to do a include

so build is unrelated, however it seems like pyfakefs is active in that bit

basically at the generate report time, the writeout fails

it seems impossible to prevent unless you defer the writeout until after the session (like junitxml)

[RonnyPfannschmidt]

as far as i'm concerned this is a integration bug with pyfakefs, i reported it upstream
pytest-html can probably sidestep it in part by not running on all reports
however ultimately the patching of pyfakefs is not robust and its jsut the first time someone stumbled on it

a good way to sidestep this would be not to rely on the fs for formatting

[mrbean-bremen]

FWIW, the problem is fixed in the current release of pyfakefs (tested against pytest-html 4.0.2).