Add pytest_before_assert hook.

[robertschweizer]

This hook will run before every assert, in contrast to the
pytest_assertrepr_compare hook which only runs for failing asserts.

As of now, no parameters are passed to the hook.

We use this hook to print the assertion code of passed assertions to
collect test evidence. This is done using inspect::

inspect.stack()[7].code_context[0].strip()

Signed-off-by: Schweizer, Robert [email protected]

Thanks for submitting a PR, your contribution is really appreciated!

Here's a quick checklist that should be present in PRs (you can delete this text from the final description, this is
just a guideline):

• Create a new changelog file in the changelog folder, with a name like <ISSUE NUMBER>.<TYPE>.rst. See changelog/README.rst for details.
• Target the master branch for bug fixes, documentation updates and trivial changes.
• Target the features branch for new features and removals/deprecations.
• Include documentation when adding new features.
• Include new tests or update existing tests when applicable.

Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:

• Add yourself to AUTHORS in alphabetical order;

[coveralls]

Coverage increased (+0.02%) to 93.835% when pulling 98dd9b9 on robertschweizer:assert_hook_feature into 5d8467b on pytest-dev:features.

[nicoddemus]

At first glance it seems there's overlap with #3479, what do you think?

cc @Sup3rGeo

[robertschweizer]

You're right, I should have checked #3479 first.

If the other feature will take much longer, we could consider merging this first as a basic version without parameters being passed to the hook. We'd have to change my before_hook to an assertion pass hook for that.

Otherwise I'm ok with closing this, it's my fault for not checking the other PR.

[Sup3rGeo]

Thanks for the ping @nicoddemus.

@robertschweizer, sounds interesting but I think I still don't understand 100% the hook. Can you give a more concrete example of how would you use the hook? Also how to use the inspect command you mentioned to get information? A small example would be ideal.

[robertschweizer]

Thanks for still considering this. I'll try and give a full example.

For test documentation, we'd like to collect all the assertions in our test code. In the tests where we use this, most assertions compare against explicit values like string literals. When the assertion passes, it is then completely sufficient to print the original code, without information about the actual operand values.

We started with writing our on assertion functions, probably similar to your first solution of #3457. They just used inspect.stack()[1] to access the calling code, and printed one line of it. Now we're looking for a cleaner solution that lets us keep the benfits of nicely formatted Pytest asserts.

Setting this conftest.py:

import inspect

def pytest_before_assert():
    expr = inspect.stack()[7].code_context[0].strip()
    print(expr)

This test code:

def test_some_code():
    encountered = get_software_return_value()
    assert encountered == "expected string"

prints this output:

----------------------------- Captured stdout call -----------------------------
assert encountered == "expected string"

The stack depth of 7 is a bit ugly, but I'm not sure there's a better solution of printing the calling code.

[blueyed]

The stack depth of 7 is a bit ugly, but I'm not sure there's a better solution of printing the calling code.

Reminded me of pytest-pdb which has some methods to find the test etc, and you could use this method to get to the actual assert I assume (https://github.com/fschulze/pytest-pdb/blob/b9af6a801fc62ad2a26bbc60b22a2e9af48ab757/pytest_pdb.py#L7-L33).

[codecov]

Codecov Report

Merging #4047 into features will increase coverage by 0.02%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##           features    #4047      +/-   ##
============================================
+ Coverage     95.79%   95.82%   +0.02%     
============================================
  Files           113      113              
  Lines         25235    25250      +15     
  Branches       2495     2496       +1     
============================================
+ Hits          24175    24195      +20     
+ Misses          741      739       -2     
+ Partials        319      316       -3

Impacted Files	Coverage Δ
src/_pytest/assertion/rewrite.py	95.5% <100%> (+0.02%)	⬆️
testing/test_assertion.py	97.61% <100%> (+0.02%)	⬆️
src/_pytest/hookspec.py	100% <100%> (ø)	⬆️
src/_pytest/assertion/__init__.py	87.5% <100%> (+0.73%)	⬆️
src/_pytest/assertion/util.py	97.64% <100%> (ø)	⬆️
src/_pytest/terminal.py	91.8% <0%> (+0.81%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 936f725...df0a559. Read the comment docs.

[robertschweizer]

Since the other PR has been stuck for a while, what do you think about merging this one?

[robertschweizer]

Reminded me of pytest-pdb which has some methods to find the test etc, and you could use this method to get to the actual assert I assume (https://github.com/fschulze/pytest-pdb/blob/b9af6a801fc62ad2a26bbc60b22a2e9af48ab757/pytest_pdb.py#L7-L33).

Thanks for the suggestion! That solution is also pretty dependent on Pytest implementation details, and I can't get it to work properly with the current versions. I'm thinking about relying only on the pytest_pyfunc_call function name:

def pytest_before_assert():
    stack = inspect.stack()
    for index, frame_info in enumerate(stack):
        if frame_info.function == "pytest_pyfunc_call":
            print(stack[index - 1].code_context[0])

[nicoddemus]

Hi @robertschweizer,

It has been a long time since it has last seen activity, plus we have made sweeping changes on master to drop Python 2.7 and 3.4 support, so this PR has some conflicts which require attention.

In order to clear up our queue and let us focus on the active PRs, I'm closing this PR for now.

Please don't consider this a rejection of your PR, we just want to get this out of sight until you have the time to tackle this again. If you get around to work on this in the future, please don't hesitate in re-opening this!

Thanks for your work, the team definitely appreciates it!