Skip to content

[BUG] Crash invoking Python function which fails with RecursionError #2505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
RonAvitzur opened this issue Sep 17, 2020 · 4 comments
Closed

Comments

@RonAvitzur
Copy link

RonAvitzur commented Sep 17, 2020

Issue description

With SymPy 1.6.1 and Python 3.8 embedded in C++, solveset(1/sqrt(x)) crashes.

solveset(1/sqrt(x)) fails reporting RecursionError: maximum recursion depth exceeded in the interactive Python interpreter. With embedded Python, it crashes the embedding app with an address error.

Reproducible example code

SString* pythonDirectory = [[NSBundle mainBundle] pathForResource:@"Python" ofType:nil];
Py_SetPythonHome(Py_DecodeLocale([pythonDirectory UTF8String], NULL));

py::initialize_interpreter();
py::module gSymPy = py::module::import("sympy");
py::object parse_expr = gSymPy.attr("parse_expr");
py::object solveset = gSymPy.attr("solveset");
try {
   py::object result = solveset(parse_expr("1/sqrt(x)"))
   }
catch (...) { }

crashes the embedding application with an address error 4084 stack frames deep.

Is there a way to handle this gracefully in the embedding app without fixing the underlying SymPy bug?
(#sympy/sympy#20097)

#0 0x00000001005ae49e in vgetargs1_impl at /Python/getargs.c:257
#1 0x00000001005ae2a1 in vgetargs1 [inlined] at /Python/getargs.c:434
#2 0x00000001005ae253 in PyArg_ParseTuple at /Python/getargs.c:129
#3 0x000000010050341f in super_init at /Objects/typeobject.c:7823
#4 0x0000000100500a94 in type_call at /Objects/typeobject.c:991
#5 0x00000001004a9096 in _PyObject_MakeTpCall at /Objects/call.c:159
#6 0x000000010059002c in call_function ()
#7 0x000000010058cdcd in _PyEval_EvalFrameDefault at /Python/ceval.c:3500
#8 0x0000000100591070 in PyEval_EvalFrameEx [inlined] at /Python/ceval.c:741
#9 0x0000000100591053 in _PyEval_EvalCodeWithName at /Python/ceval.c:4298
#10 0x00000001004a9be3 in _PyFunction_Vectorcall at /Objects/call.c:435
#11 0x0000000100503dfc in call_unbound_noarg [inlined] ()
#12 0x0000000100503dcd in slot_tp_hash at /Objects/typeobject.c:6479
#13 0x00000001004fbcea in tuplehash at /Objects/tupleobject.c:375
#14 0x0000000100630a68 in bounded_lru_cache_wrapper at /Modules/_functoolsmodule.c:939
#15 0x00000001004a954b in PyObject_Call at /Objects/call.c:245
#16 0x000000010058d00b in do_call_core [inlined] at /Python/ceval.c:5010
#17 0x000000010058cfdf in _PyEval_EvalFrameDefault at /Python/ceval.c:3559
...
#4069 0x000000010058cdcd in _PyEval_EvalFrameDefault at /Python/ceval.c:3500
#4070 0x0000000100591070 in PyEval_EvalFrameEx [inlined] at /Python/ceval.c:741
#4071 0x0000000100591053 in _PyEval_EvalCodeWithName at /Python/ceval.c:4298
#4072 0x00000001004a9be3 in _PyFunction_Vectorcall at /Objects/call.c:435
#4073 0x000000010058ff02 in _PyObject_Vectorcall [inlined] at /Include/cpython/abstract.h:127
#4074 0x000000010058fef6 in call_function at /Python/ceval.c:4963
#4075 0x000000010058ce6b in _PyEval_EvalFrameDefault at /Python/ceval.c:3515
#4076 0x0000000100591070 in PyEval_EvalFrameEx [inlined] at /Python/ceval.c:741
#4077 0x0000000100591053 in _PyEval_EvalCodeWithName at /Python/ceval.c:4298
#4078 0x00000001004a9be3 in _PyFunction_Vectorcall at /Objects/call.c:435
#4079 0x00000001004a93ec in PyVectorcall_Call at /Objects/call.c:199
#4080 0x00000001001a1dca in pybind11::detail::unpacking_collector<(pybind11::return_value_policy)1>::call(_object*) const at /pybind11/cast.h:2074
#4081 0x000000010017772d in pybind11::object pybind11::detail::object_apipybind11::handle::operator()<(pybind11::return_value_policy)1, pybind11::object, pybind11::object, pybind11::arg_v>(pybind11::object&&, pybind11::object&&, pybind11::arg_v&&) const at /pybind11/cast.h:2196

@bstaletic
Copy link
Collaborator

First, you didn't finalize the interpreter. Second, you didn't declare parse_expr. After fixing those two, your snippet looks like this:

#include <pybind11/embed.h>
namespace py = pybind11;
int main() {
	py::scoped_interpreter g;
	py::module gSymPy   = py::module::import("sympy");
	py::object gParse   = gSymPy.attr("parse_expr");
	py::object solveset = gSymPy.attr("solveset");
	py::object parse_expr = gSymPy.attr("parse_expr");
	py::object result = solveset(parse_expr("1/sqrt(x)"));
}

The last line raises RecursionError, that gets rethrown as py::error_already_set. Since py::error_already_set isn't caught anywhere, we should expect a crash.

terminate called after throwing an instance of 'pybind11::error_already_set'
  what():  RecursionError: maximum recursion depth exceeded while calling a Python object

At:
  /usr/lib/python3.8/site-packages/sympy/core/basic.py(2017): _preorder_traversal

zsh: abort      ./a.out

The above is with pybind11 2.5.0, python 3.8.5 and sympy 1.6.2.

However, I can see some objective-c in your snippet, which I know nothing about.

@RonAvitzur
Copy link
Author

RonAvitzur commented Sep 17, 2020

Right, thank you. I left out the try/catch portion of my source:

    try {
...
        }
    catch (...) { }

I seeing the crash on the call without seeing any C++ exceptions thrown here. I wonder what is different in my embedded configuration.

@bstaletic
Copy link
Collaborator

https://pybind11.readthedocs.io/en/master/advanced/exceptions.html#handling-exceptions-from-python-in-c

All python exceptions get translated to py::error_already_set. If there are no further questions, feel free to close this.

@RonAvitzur
Copy link
Author

On further investigation, the one additional aspect in the embedding app which I had left out mistakenly thinking it irrelevant is that the app creates a pthread dedicated to Python which always holds the GIL and processes a command queue. On macOS here, pthread_create defaults to a 512K stack size for the thread, which is insufficient. Increases the thread's stack size to 2MB eliminated the crash so the exception now appears and is handled gracefully. Thank you for your patience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants