Add first version of pytest infrastructure

[davidrudlstorfer]

Includes first fixtures and tagging approach

[davidrudlstorfer]

@isteinbrecher this is a work in progess - I've added the first pytest fixtures and also enabled the tagging approach. This turned out to be much more complicated than anticipated.

I wanted that the "standard" tests would always execute - this approach was not doable with https://pypi.org/project/pytest_tagging/

Therefore, I simply implemented my own version which turns out to be very easy and doable

The tests can now be executed with:

pytest => execute tests which are not tagged
pytest --4C_ArborX => execute tests which are tagged with fourc_arborx, i.e., @pytest.mark.fourc_arborx
pytest --CubitPy => execute tests which are tagged with cubitpy, i.e., @pytest.mark.cubitpy
pytest --performance => execute tests which are tagged with performance, i.e., @pytest.mark.performance

It is also possible to add multiple tags to tests and also execute multiple tags at once

I am happy with the current status and I'll add the reference comparison function soon

[davidrudlstorfer]

Also pytest coverage is added and works as expected - afterwards we can simply add

1. the coverage report of meshpy
2. the coverage badge to the readme

(Currently I get differing results in the Github actions and local executions -> locally it looks like this:)

Click me

========================================================================================= test session starts =========================================================================================
platform linux -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0
rootdir: /home/rudlstorfer/01_work/01_workspace/meshpy
configfile: pyproject.toml
plugins: cov-6.0.0
collected 9 items / 2 deselected / 7 selected                                                                                                                                                         

tests/test_cosserat_curve.py ......                                                                                                                                                             [ 85%]
tests/test_dummy.py .                                                                                                                                                                           [100%]

---------- coverage: platform linux, python 3.12.8-final-0 -----------
Name                                                         Stmts   Miss  Cover
--------------------------------------------------------------------------------
meshpy/__init__.py                                              14      0   100%
meshpy/abaqus/__init__.py                                        3      3     0%
meshpy/abaqus/beam.py                                           24     24     0%
meshpy/abaqus/input_file.py                                    139    139     0%
meshpy/base_mesh_item.py                                        20      1    95%
meshpy/boundary_condition.py                                    48     21    56%
meshpy/conf.py                                                  74      0   100%
meshpy/container.py                                             49      6    88%
meshpy/cosserat_curve/__init__.py                                0      0   100%
meshpy/cosserat_curve/cosserat_curve.py                        164     10    94%
meshpy/cosserat_curve/warping_along_cosserat_curve.py           98      3    97%
meshpy/coupling.py                                              52     43    17%
meshpy/element.py                                               48     20    58%
meshpy/element_beam.py                                         187    106    43%
meshpy/element_volume.py                                        50     15    70%
meshpy/four_c/__init__.py                                        5      5     0%
meshpy/four_c/beam_potential.py                                 33     33     0%
meshpy/four_c/dbc_monitor.py                                    32     32     0%
meshpy/four_c/locsys_condition.py                                5      5     0%
meshpy/four_c/run_four_c.py                                     34     34     0%
meshpy/four_c/solid_shell_thickness_direction.py               127    127     0%
meshpy/function.py                                              11      3    73%
meshpy/function_utility.py                                      18      1    94%
meshpy/geometric_search/__init__.py                              0      0   100%
meshpy/geometric_search/find_close_points.py                    59     32    46%
meshpy/geometric_search/geometric_search_arborx.py              19      7    63%
meshpy/geometric_search/geometric_search_cython.py              16      4    75%
meshpy/geometric_search/geometric_search_scipy.py               22     19    14%
meshpy/geometry_set.py                                         126     61    52%
meshpy/header_functions.py                                      71     63    11%
meshpy/inputfile.py                                            411    205    50%
meshpy/material.py                                             111     57    49%
meshpy/mesh.py                                                 333    193    42%
meshpy/mesh_creation_functions/__init__.py                      10      0   100%
meshpy/mesh_creation_functions/beam_basic_geometry.py          114     69    39%
meshpy/mesh_creation_functions/beam_curve.py                    80     75     6%
meshpy/mesh_creation_functions/beam_fibers_in_rectangle.py      55     48    13%
meshpy/mesh_creation_functions/beam_generic.py                  90     40    56%
meshpy/mesh_creation_functions/beam_honeycomb.py                85     76    11%
meshpy/mesh_creation_functions/beam_nurbs.py                    35     30    14%
meshpy/mesh_creation_functions/beam_stent.py                   107     96    10%
meshpy/mesh_creation_functions/beam_wire.py                     30     24    20%
meshpy/mesh_creation_functions/nurbs_generic.py                185    175     5%
meshpy/mesh_creation_functions/nurbs_geometries.py             171    160     6%
meshpy/node.py                                                  59     22    63%
meshpy/nurbs_patch.py                                          117    100    15%
meshpy/rotation.py                                             193     38    80%
meshpy/utility.py                                              119     81    32%
meshpy/vtk_writer.py                                           152    132    13%
tutorial/__init__.py                                             1      1     0%
tutorial/tutorial.py                                            75     75     0%
--------------------------------------------------------------------------------
TOTAL                                                         4081   2514    38%
Coverage HTML written to dir htmlcov

Required test coverage of 30% reached. Total coverage: 38.40%

=================================================================================== 7 passed, 2 deselected in 1.95s ===================================================================================

[isteinbrecher]

@davidrudlstorfer thanks a lot for the effort! This looks really great and I really like it. I left some comments. Do you envision switching only a few modules in this PR or more? See my previous comment, we could simply merge the pytest config and already execute the "old" tests and the port file by file, but that is just a suggestion.

[davidrudlstorfer]

@isteinbrecher thanks for your first round of review of the current status

In this PR I only wanted to update the current pytest (cosserat curve) and out of curiosity I've ported one additional feature (rotations). The remaining parts should be done in another PR

The only thing missing here is the implementation of the comparison function. As we've already briefly discussed I want to implement one function which automatically detects if either a string or file is handed over and then automatically compares the files or the string. I'll add that soon

[isteinbrecher]

The only thing missing here is the implementation of the comparison function. As we've already briefly discussed I want to implement one function which automatically detects if either a string or file is handed over and then automatically compares the files or the string. I'll add that soon

@davidrudlstorfer we already have that, the compare_test_result (which calls compare_strings under the hood) does exactly that. If I remember correctly, we were talking about unifying this for general text based files and vtk files (compare_vtk), although I am not sure if this does not introduce more complexity than simply having two functions for fundamental different file types.

[davidrudlstorfer]

@isteinbrecher this would be ready for a review from your side.

In the last commit I've now added my suggestion for the new comparison feature. In my opinion this has a few advantages to the old version:

• one interface for testing comparisons (strings, vtk based objects, ... => easily extendable to other types, i.e. csv, ...)
• the interface is now clearer to the user => input are strings and nothing magical happens within the function (i.e., the retrieval of test names)
• Elaborate error retrieval and processing of errors: The actual differences are now printed for each failing test. This drastically improves the debugging rather than the manual file write out (which was not consistent across tests)

I've tested every path of the implementation and everything works accordingly!

I would be very happy if we could converge on a final pytest testing framework with the current suggested changes.

[davidrudlstorfer]

This would now also enable to port everything to the new testing framework - as well as the deletion of the coverage configs and the test utils

[isteinbrecher]

@davidrudlstorfer thanks a lot for the work on the modernisation of the testing framework. I left some comments, maybe it makes sense to discuss them in person at some point. I would just like to get the functionality best fit for the type of testing done in MeshPy. Please don't get discourage by my comments, I really like this whole approach and greatly appreciate your effort!

Other question, if I run pytest it takes a couple of seconds to gather the tests. Do you have an idea how this could be speed up. This doesn't matter for CI, only for local testing.

[davidrudlstorfer]

@isteinbrecher big thanks for your second round of review! Let's discuss the few open points in person to faster converge on one solution :)

Other question, if I run pytest it takes a couple of seconds to gather the tests. Do you have an idea how this could be speed up. This doesn't matter for CI, only for local testing.

I think this is a combination of many things here. First of all it takes so long because pytest resolves all imports prior to running the tests. This takes some time in MeshPy because the whole module structure is not optimal. Once the imports are improved and modules are correctly split this should be faster (i.e. see #159)

https://stackoverflow.com/questions/16417546/how-to-speed-up-pytest

Pytest imports all modules in the testpaths directories to look for tests. The import itself can be slow. This is the same startup time you'd experience if you ran those tests directly, however, since it imports all of the files it will be a lot longer. It's kind of a worst-case scenario.

This doesn't add time to the whole test run though, as it would need to import those files anyway to execute the tests.

The total test time should, therefore, stay the same in comparison to the current testing approach. The current testing suite takes around 3s on my machine and the current pytests take around 1s for collecting and running. I think once everything is ported we can look at the total execution time again. Also after restructuring the repo and improving the imports this should be even faster.

[isteinbrecher]

@davidrudlstorfer thanks for the changes, I am not sure if the PR was already ready for review, I sill left some comments 😅.

[davidrudlstorfer]

@isteinbrecher sorry if it wasn't clear from my side: I am still working on the changes we discussed yesterday I hope to get them ready this evening or tomorrow morning

They will adress most of the points raised above, i.e., a very fancy way vastly simplify the result comparison with reference files

Currently its a real WIP - once the major things are resolved I'll push it 🚀

EDIT: I also brought back the functionality regarding the diffs in VSCode which now works in a more general way

[isteinbrecher]

I am sorry, I didn’t have a close enough look before reviewing 😅

[davidrudlstorfer]

Of course, it took much longer than expected - it felt like 50 pipeline runs 🥲

Now everything works and the artifact upload works for all containers (macOS, Windows, ubuntu, 4C container)

I've adressed all of your comments and this would be ready for your next review @isteinbrecher

[davidrudlstorfer]

This would be a failed pipeline run where you can download the artifact from each job

https://github.com/imcs-compsim/meshpy/actions/runs/12692452173

[isteinbrecher]

@davidrudlstorfer thanks for the work, I think we are very close to merging this. I took the liberty and added two commits to the branch, as a suggestion:

b769dca Allow for general types in compare function

This commit allows for comparison of all possible combinations of Path, str, and InputFile with each other.

9766fde Refactor assertions and return values from compare functions

While adding the changes form the previous commit, I realised that I missed the changes in the compare_strings_with_tolerance function which raise an error. This should not be done in this function, as we use this one in some other projects where we rely that the return value is either True or False. I changed that accordingly.

While looking at this, I saw that we had some (at least to me) not so intuitive behaviours in the code: We had functions that return True or raise an error, but never return False. At least to me that seemed weird. I tried to unify this, that the compare functions we actually will call from within the testing framework now either run without complaining or raise an assertion. This also allows to simply call the renamed compare function in the testing with

assert_results_equal(...

Without the leading assert - I personally like that, since this eliminates the possible error that someone calls the compare function and forgets the assert.

I am curios about your opinion on these changes.

[davidrudlstorfer]

@isteinbrecher thanks a lot for extending the functions!

Unfortunately, I do not fully agree with you on all changes. In many cases a lot of complexity and nested functions were added where it is not necessary at all. Also functions should always return one thing and not just empty strings depending on the outcome.

We could probably increase the simplicity of the code by a lot. I also fully agree with you if you decide to merge the current changes.

[davidrudlstorfer]

If you like I am more than happy to also provide a suggestion on my comments from above - I think in the end it is worth it if we provide a really nice testing framework 🙏🏻

[isteinbrecher]

Thanks for your comments @davidrudlstorfer. I left some explanations. I will add another commit trying to revert my changes to address most of your major points. We can use that as a starting point for further discussion or simply revert it again.

[davidrudlstorfer]

@isteinbrecher thanks for adding the changes and explanations!

If I understand you correctly your major concern is that you want to use the function externally as well. What do you think if we just provide a wrapper for the external use?

I added this in the last commit - this function now does exactly the same thing you implemented but is just a wrapper around the standard compare_strings_with_tolerance function.

This way the function signature is identical within meshpy for compare_strings_equality and compare_strings_with_tolerance.

I am open for a better name of the function which also returns stuff 😅

Curious about your opinion

[isteinbrecher]

Looks good to me! I left two minor comments, other than that I think we can proceed and merge this. The main functionality should work as expected and we can fine tune the implementation later on if needed.

Thanks again for the effort to modernise the testing framework @davidrudlstorfer .

I will open an issue to track the test modules that still need to be ported.

Ps: I suggest to combine everything in a single, or some very few commits.

[davidrudlstorfer]

Perfect 🤩 I've addressed the 2 minor remarks. I'll squash everything into a few manageable commits and then force push it

[isteinbrecher]

Thanks again for the effort, lets get this merged!