[Installation issue]: python fails to build in a virtualenv due to pybind

[LElgueddari]

I tried to install pysap in a virtualenv and it could not find pybind.h. Here is the error that I get:

  /tmp/pip-req-build-wcrcu_4f/sparse2d/python/pysparse.cpp:10:31: fatal error: pybind11/pybind11.h: No such file or directory
  compilation terminated.
  CMakeFiles/pysparse.dir/build.make:62: recipe for target 'CMakeFiles/pysparse.dir/pysparse.cpp.o' failed
  make[2]: *** [CMakeFiles/pysparse.dir/pysparse.cpp.o] Error 1
  CMakeFiles/Makefile2:142: recipe for target 'CMakeFiles/pysparse.dir/all' failed
  make[1]: *** [CMakeFiles/pysparse.dir/all] Error 2
  Makefile:83: recipe for target 'all' failed
  make: *** [all] Error 2```

[sfarrens]

@LElgueddari Did you install pybind11 before building PySAP?

[LElgueddari]

Yes , pybind 11 was installed but it seems more like pybind.h file path is not recognized by the system

[sfarrens]

OK, it looks like this is a known issue with pybind11 and virtualenv.

tbenthompson/cppimport#2

I will try to reproduce this error and see if I can find a solution.

[LElgueddari]

Except that when I run pybind11.get_include() and pybind11.get_include(True), I got twice the same path which is the path of the pybind11 include directory.

Could we export this path and include it in the CMakeLists.txt of sparse2d, this actually solves my issue

[sfarrens]

I believe we can.

Can you tell me exactly what you did that solved the issue for you?

[LElgueddari]

Sure in:

pysap/sparse2d/python/CMakeLists.txt

Line 43 in 732a014

set(CMAKE_CXX_FLAGS "-std=c++11 -DNO_DISP_IO -ggdb3 -fPIC -O2 -ffast-math -fomit-frame-pointer ${APPLE_FLAGS} ${OpenMP_CXX_FLAGS} -Wno-write-strings -DNDEBUG")

I change it to:
set(CMAKE_CXX_FLAGS "-std=c++11 -DNO_DISP_IO -ggdb3 -ffast-math -fomit-frame-pointer ${APPLE_FLAGS} ${OpenMP_CXX_FLAGS} -Wno-write-strings -DNDEBUG -I/path_to_venv_pysap/include/site/python3.5")

[sfarrens]

It appears that this issue comes from two sources. Firstly that virtualenv creates a separate site directory for some modules and secondly

pysap/sparse2d/python/CMakeLists.txt

Line 13 in 732a014

find_package(PythonLibs REQUIRED)

where FindPythonLibs (https://cmake.org/cmake/help/v3.0/module/FindPythonLibs.html) either returns incorrect or empty values for PYTHON_LIBRARIES and PYTHON_INCLUDE_DIRS.

I have opened PR #94 with a potential fix. For this fix you will need to set the environment variable VENV_DIR to the virtualenv being used, then build as usual.

[LElgueddari]

Still getting the same issue
Oh no sorry my bad I set the environment variable to where pybind11 was, but it should be just where the virtual environment path.

[sfarrens]

Resolved in #94

[zaccharieramzi]

So what exactly should the VENV_DIR env variable look like?
So far I did: export VENV_DIR=$HOME/workspace/pysap-mri/venv/, but I got the same error as Loubna's.

[zaccharieramzi]

I see that in this line:
https://github.com/CEA-COSMIC/pysap/pull/94/files#diff-6cfd9a6a570111ce69022c106e1bab06R19

the venv/include directory should have the python executable, but in my case it doesn't. It just had the site dicrectory.

[sfarrens]

@zaccharieramzi The VENV_DIR you set should be correct. Perhaps try expanding $HOME to see if that makes any difference.

The Python executable should be in venv/bin. The line you quoted is for the Python headers which should normally be in e.g. venv/include/python3.7m, while the Pybind11 headers should be in e.g. venv/include/site/python3.7.

Please let me know if your virtualenv was configured differently.

[LElgueddari]

There's two different things, the first one is that pybind11 needs to be installed before the cmake so you will find the python repo that you don't see.
The second thing could we try to use VIRTUAL_ENV environment variable (instead of VENV_DIR ) to automatically set the include path of pybind11

[zaccharieramzi]

try expanding $HOME to see if that makes any difference.
@sfarrens I don't think it will. This is the output of echo $VENV_DIR: /home/zaccharie/workspace/pysap-mri/venv/.

I am sorry I got confused between headers and executable. The executable is indeed in venv/bin. However, the headers are not in venv/include. Pybind11 headers are indeed in venv/include/site/python3.6 (I use python3.6).

[zaccharieramzi]

@LElgueddari I had indeed installed pybind11 before trying to install pysap.

I didn't understand your second note on the VIRTUAL_ENV env variable.

[sfarrens]

@LElgueddari I did not know about this variable. I agree it would be easier to use this as it is automatically set when the environment is activated. I will update this soon.

@zaccharieramzi Can you tell me what the contents of your venv/include directory are?

[zaccharieramzi]

Output of ls -a venv/include: . .. site.

[sfarrens]

OK, @LElgueddari and @zaccharieramzi can you both let me know the setup you used so that I can investigate why the headers are not in the same place.

[zaccharieramzi]

I used virtualenv -p /usr/bin/python3.6 venv

[sfarrens]

@zaccharieramzi What OS are you using? and what version of virtualenv?

[zaccharieramzi]

OS: ubuntu 16.04
virtualenv --version output: 16.4.3

[sfarrens]

@zaccharieramzi I have not been able to reproduce your issue. Even using virtualenv v16.0.0 on Linux it still puts all the python headers in the include directory.

[zaccharieramzi]

I indeed reinstalled my virtual environment and the python headers are at the right place. Checking right now to see if t works.

[zaccharieramzi]

It works !