You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When converting a sequence-like argument to an std::vector<T>, pybind11 ignores exceptions raised by len(arg) and erroneously calls std::vector<T>::reserve() with a very large value. The result is a confusing exception of the following form:
Traceback (most recent call last):
File "./pybind11_repro.py", line 11, in <module>
my_module.f(a)
ValueError: vector::reserve
I used a zero-dimensional numpy array to demonstrate this problem, but it actually occurs with any object that is sequence-like (has __getitem__) and raises from __len__:
class BadLen():
def __getitem__(self, i):
return i
def __len__(self):
raise Exception()
my_module.f(BadLen()) # raises: ValueError: vector::reserve
Result and expected result
When running the Python code above, a C++ exception is thrown by std::vector<int>::reserve(). pybind11 propagates this to the Python caller, but this is confusing and does not indicate why the operation failed.
Traceback (most recent call last):
File "./pybind11_repro.py", line 11, in <module>
my_module.f(a)
ValueError: vector::reserve
This exception occurred because len(a) in the example above raises a Python exception (see Diagnosis section below):
>>> len(a)
Traceback (most recent call last):
File "pybind11_repro.py", line 9, in <module>
len(a)
TypeError: len() of unsized object
I think that pybind11 should propagate this original Python exception to the caller, instead of triggering an unrelated exception at a later point in time.
Diagnosis
This occurs through the following steps:
pybind11::sequence::size()calls PySequence_Size(m_ptr), which returns a signedPy_ssize_t value. The pybind11 member function unconditionally casts this value to size_t. If PySequence_Size returns -1 (indicating failure), pybind11::sequence::size() will return SIZE_MAX, a very large positive number.
I think this problem could be fixed by checking the return value of PySequence_Size, and raising the active CPython exception if that call has failed, as indicated by its return value.
The text was updated successfully, but these errors were encountered:
nmusolino
added a commit
to nmusolino/pybind11
that referenced
this issue
Jan 26, 2020
Uh oh!
There was an error while loading. Please reload this page.
Issue description
When converting a sequence-like argument to an
std::vector<T>
, pybind11 ignores exceptions raised bylen(arg)
and erroneously callsstd::vector<T>::reserve()
with a very large value. The result is a confusing exception of the following form:Reproducible example code
C++ code:
This C++ code was compiled with:
Python code:
I used a zero-dimensional numpy array to demonstrate this problem, but it actually occurs with any object that is sequence-like (has
__getitem__
) and raises from__len__
:Result and expected result
When running the Python code above, a C++ exception is thrown by
std::vector<int>::reserve()
. pybind11 propagates this to the Python caller, but this is confusing and does not indicate why the operation failed.This exception occurred because
len(a)
in the example above raises a Python exception (see Diagnosis section below):I think that pybind11 should propagate this original Python exception to the caller, instead of triggering an unrelated exception at a later point in time.
Diagnosis
This occurs through the following steps:
pybind11::sequence::size()
callsPySequence_Size(m_ptr)
, which returns a signedPy_ssize_t
value. The pybind11 member function unconditionally casts this value tosize_t
. IfPySequence_Size
returns -1 (indicating failure),pybind11::sequence::size()
will returnSIZE_MAX
, a very large positive number.size()
is passed tovector<T>::reserve()
. The reserve() member function can throwstd::length_error
orstd::bad_alloc
if the requested size is too large.I think this problem could be fixed by checking the return value of
PySequence_Size
, and raising the active CPython exception if that call has failed, as indicated by its return value.The text was updated successfully, but these errors were encountered: