crash when matching overload not found

[germasch]

Issue description

This issue was observed as a ADIOS2 issue. pybind11 is used to generate the python bindings, but I believe it's not specific to ADIOS2. In same cases, calling a C++ function with the wrong arguments causes the python interpreter to crash.

Reproducible example code

So I know I'm supposed to provide a simple test case, which would be easier if I didn't have to deal with the hassles of building it. Actually, your doc says, to run the tests that come with pybind11, do

mkdir build
cd build
cmake ..
make check -j 4

But make check doesn't work (make does).

Anyway, here's what I cut it down to:

void open_test(int i, bool b)
{
  std::cerr << "open_test " << i << " " << b << std::endl;
}

PYBIND11_MODULE(adios2, m)
{
    m.def("open_test", &open_test);
}

import numpy as np
import adios2

myArray = np.array([0, 1., 2., 3., 4., 5., 6., 7., 8., 9.])

adios2.open_test(2, True) # <-- OK
adios2.open_test(2, myArray) # <-- crashes

Output:

[kai@macbook build (python_cleanup *)]$ python ../examples/hello/bpWriter/x.py
open_test 2 1
libc++abi.dylib: terminating with uncaught exception of type pybind11::error_already_set: SystemError: <class 'type'> returned a result with an error set

At:
  /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/arrayprint.py(1416): array_repr
  ../examples/hello/bpWriter/x.py(8): <module>
[...]

I think I roughly figured out what's happening:

• dispatcher goes through available overload and tries to call them
• if none matches, an error message is supposed to be printed, listing the available overloads, and
  the actual arguments.
• composing that message is where the uncaught exception appears. Specifically, it happens in pytypes.h:repr()

PyObject *str_value = PyObject_Repr(h.ptr());
    if (!str_value) throw error_already_set();

when repr is called to make a string out of the (actual) numpy.array argument in order to compose the "incompatible arguments" message.

repr on the numpy array fails because there is already an active exception, which makes things fail in the first line below in arrayprint.c.

    if type(arr) is not ndarray:
        class_name = type(arr).__name__
    else:
        class_name = "array"

The reason that an exception is already set here is that previously, in trying the (only) overload, that caused a python exception about not being able to convert numpy.array to bool.

I can work around the problem by clearing the exceptions after calling an overload has not worked. I don't know whether that's a correct fix, though, I'm really not familiar with all that exception back an forth.

This patch

-- a/thirdparty/pybind11/pybind11/include/pybind11/pybind11.h
+++ b/thirdparty/pybind11/pybind11/include/pybind11/pybind11.h
@@ -632,6 +632,8 @@ protected:
                 if (result.ptr() != PYBIND11_TRY_NEXT_OVERLOAD)
                     break;

+               PyErr_Clear();
+
                 if (overloaded) {
                     // The (overloaded) call failed; if the call has at least one argument that
                     // permits conversion (i.e. it hasn't been explicitly specified `.noconvert()`)
@@ -660,6 +662,7 @@ protected:

                     if (result.ptr() != PYBIND11_TRY_NEXT_OVERLOAD)
                         break;
+                   PyErr_Clear();
                 }
             }
         } catch (error_already_set &e) {

fixes the crash, to now give the expected error:

[kai@macbook build (python_cleanup *)]$ python ../examples/hello/bpWriter/x.py
open_test 2 1
Traceback (most recent call last):
  File "../examples/hello/bpWriter/x.py", line 8, in <module>
    adios2.open_test(2, myArray) # <-- crashes
TypeError: open_test(): incompatible function arguments. The following argument types are supported:
    1. (arg0: int, arg1: bool) -> None

Invoked with: 2, array([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])

[YannickJadoul]

I believe this was fixed by #1976.
I also cannot reproduce with 2.5.0, but I can reproduce with 2.4.3 (which did not include #1976).

Closing; if this is still a problem, please reopen. Thanks for reporting and digging for the error, @germasch!