Skip to content

[BUG] Segfault in py::buffer::request on a ctypes array #2502

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
fritzr opened this issue Sep 17, 2020 · 1 comment
Closed

[BUG] Segfault in py::buffer::request on a ctypes array #2502

fritzr opened this issue Sep 17, 2020 · 1 comment

Comments

@fritzr
Copy link
Contributor

fritzr commented Sep 17, 2020

Issue description

py::buffer::request() causes a segmentation fault when attempting to get the py::buffer_info for a ctypes array in Python 2 because it has a NULL stride.

Reproducible example code

test_ctypes_buffer.cpp

#include <pybind11/pybind11.h>
namespace py = pybind11; 

PYBIND11_MODULE(test_ctypes_buffer, m) {
  m.def("get_ctypes_buffer_size", [](py::buffer buffer) {  
    py::buffer_info info = buffer.request();
    return info.shape[0];    
  });    
}   

test_ctypes_buffer.py

from test_ctypes_buffer import get_ctypes_buffer_size
import ctypes
assert get_ctypes_buffer_size((ctypes.c_char * 10)()) == 10

Segmentation fault:

$ g++ -dumpversion
7.4.0
$ g++ -g -O0 -Wall -shared -std=c++11 -fPIC `/usr/bin/python -m pybind11 --includes` test_ctypes_buffer.cpp -o test_ctypes_buffer.so
$ /usr/bin/python -V
Python 2.7.5
$ /usr/bin/python test_ctypes_buffer.py
Segmentation fault (core dumped)

Post-mortem analysis:

$ gdb /usr/bin/python core.*
[...]
(gdb) bt
[...]
#0  0x00007f5647fda7d0 in __memmove_ssse3_back ()  from /lib64/libc.so.6
#1  0x00007f564172673a in std::__copy_move<false, true, std::random_access_iterator_tag>::__copy_m<long> (__first=0x0, __last=0x8, __result=0x1023660)  at /opt/gcc-7.4.0/include/c++/7.4.0/bits/stl_algobase.h:368
[...]
#11 0x00007f5641715a40 in pybind11::detail::any_container<long>::any_container<long*, void> (this=0x7ffdd557a7e0, first=0x0, last=0x8)  at /home/fritzr/.local/lib/python2.7/site-packages/pybind11/include/pybind11/detail/common.h:811
#12 0x00007f564170b71a in pybind11::buffer_info::buffer_info (this=0x7ffdd557a8d0, view=0x10235f0, ownview=true)  at /home/fritzr/.local/lib/python2.7/site-packages/pybind11/include/pybind11/buffer_info.h:56
#13 0x00007f564170d25f in pybind11::buffer::request (this=0x7ffdd557a9a8, writable=false)  at /home/foreese/.local/lib/python2.7/site-packages/pybind11/include/pybind11/pytypes.h:1333
[...]
(gdb) up 12
#12 0x00007f564170b71a in pybind11::buffer_info::buffer_info (this=0x7ffdd557a8d0, view=0x10235f0, ownview=true) at /home/fritzr/.local/lib/python2.7/site-packages/pybind11/include/pybind11/buffer_info.h:56
56	            {view->shape, view->shape + view->ndim}, {view->strides, view->strides + view->ndim}, view->readonly) {
(gdb) list 54,+5
    54	    explicit buffer_info(Py_buffer *view, bool ownview = true)
    55	    : buffer_info(view->buf, view->itemsize, view->format, view->ndim,
==> 56	            {view->shape, view->shape + view->ndim}, {view->strides, view->strides + view->ndim}, view->readonly) {
    57	        this->view = view;
    58	        this->ownview = ownview;
    59	    }
(gdb) p *view
$1 = {
  buf = 0x7f5648fcc910, 
  obj = 0x7f5648fcc8c0, 
  len = 10, 
  itemsize = 1, 
  readonly = 0, 
  ndim = 1, 
  format = 0x1035b50 "(10)<c", 
  shape = 0x1035f60, 
  strides = 0x0, 
  suboffsets = 0x0, 
  smalltable =     {0,
    0}, 
  internal = 0x0
}
(gdb) p *view->shape
$2 = 10

We see that the ctypes array's buffer has NULL stride. Therefore the construction of buffer_info::strides{NULL, NULL + view->ndim} causes the segmentation fault.

fritzr added a commit to fritzr/pybind11 that referenced this issue Sep 17, 2020
YannickJadoul pushed a commit that referenced this issue Oct 3, 2020
* tests: New test for ctypes buffers (#2502)

* fix: fix buffer_info segfault on views with no stride (pybind11#2502)

* Explicit conversions in buffer_info to make clang happy (#2502)

* Another explicit cast in buffer_info constructor for clang (#2502)

* Simpler implementation of buffer_info constructor from Py_buffer.

* Move test_ctypes_buffer into test_buffers

* Comment on why view->strides may be NULL (and fix some whitespace)

* Use c_strides() instead of zero when view->strides is NULL.

c_strides and f_strides are moved from numpy.h (py::array)
to buffer_info.h (py::detail) so they can be used from the
buffer_info Py_buffer constructor.

* Increase ctypes buffer test coverage in test_buffers.

* Split ctypes tests and skip one which is broken in PyPy2.
@YannickJadoul
Copy link
Collaborator

Fixed in #2503

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants