docs: add a note about compiling the example

[ssiano]

When pybind11 is included as a submodule, the user needs to update their
Python module search path. Otherwise, the first c++ compilation command
in docs/basics.rst will fail.

[ssiano]

This bit of documentation may be helpful to new users. It would have been helpful to me when I first went through the instructions in the doc.

If you believe so, feel free to rewrite it as you see fit and discard this pull request.

[YannickJadoul]

I believe the idea is that if you use pybind11 as a git submodule, that you don't install it through pip. So you don't need to call python3 -m pybind11 --includes, because you know where the pybind11 includes are (i.e., PATH_TO/extern/pybind11/include or so, as noted in the paragraph above.)

I don't even know if python -m pybind11 is supposed to work when pybind11 is not pip-installed. I don't believe so, at least. @henryiii?

[henryiii]

Yes, it is supposed to work, though I would do it this way: https://github.com/scikit-hep/boost-histogram/blob/ca298e7c9008025191e94a927fbd0d1bd012b97f/setup.py#L11-L14

Or, better yet, refer to the docs here under "Suggested usage if you have pybind11 as a submodule": https://pybind11.readthedocs.io/en/stable/compiling.html#copy-manually :)

You should never be setting PYTHONPATH if you are building a library, and an install step must be self contained and not depend on the users environment.

[henryiii]

Ahh, you are looking at building it by hand, but using it as a submodule? You generally should always be using setuptools or CMake if you are working with a library containing a submodule, not building it by hand. That's just a demo to show you how it works and prove it is simple. If you do want to build it by hand but you don't want to install it, then sure, but I don't think it needs to be in the docs. An end-user can use PYTHONPATH, though I still recommend staying away from it.

[ssiano]

That's a good point @YannickJadoul. Maybe it would be better to update docs/basics.rst to show how to do the c++ compilation when using pybind11 as a Git submodule.

Currently that command is

c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`

python -m pybind11 does work when pybind11 is not pip-installed as long as you use PYTHONPATH. But as you mention this may not be necessary if the user knows that the includes can be specified such as -I/usr/include/python3.8 -IPATH_TO/extern/pybind11/include. However, the user may not know the first part (the -I/usr/include/python3.8).

[henryiii]

the user knows that the includes can be specified

It's best written as python3-config --includes, I think.

[henryiii]

Or python3-config --cflags to get all the flags and such.

[henryiii]

I also prefer $() to backticks, it's much easier to avoid missing, but have never proposed the change.

[henryiii]

c++ -shared -std=c++11 -fPIC $(python3-config --cflags) <path to pybind11/include> example.cpp \
    -o example$(python3-config --extension-suffix)

I think is the full, correct way to do it all in one line. For me, it expands to:

c++ -shared -std=c++11 -fPIC -I/usr/local/Cellar/[email protected]/3.9.1/Frameworks/Python.framework/Versions/3.9/include/python3.9 \
  -I/usr/local/Cellar/[email protected]/3.9.1/Frameworks/Python.framework/Versions/3.9/include/python3.9 \
  -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG \
  -g -fwrapv -O3 -Wall -I/usr/local/include \
  -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include \
  -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers \
  <path to pybind11/include> example.cpp -o example.cpython-39-darwin.so

But why are we doing this by hand instead in CMake or setuptools? And why are you expecting python -m pybind11 to work without installing pybind11?

[ssiano]

That command looks good Henry. You would just need to put the -I before the <path to pybind11/include>.

I'm not sure why python3-config --cflags list the first include path twice, but it does the same for me on three different platforms.

Regarding doing it by hand, this is just what a first-time visitor to the doc might try when following the demo.

[henryiii]

Maybe we could just add a note that pybind11 should be installed for these python -m pybind11 commands to work?

[ssiano]

Or maybe a note to use $(python3-config --includes) -Iextern/pybind11/include in place of $(python3 -m pybind11 --includes) if you are doing this demo with pybind11 included as a submodule.

[henryiii]

Looks much better. Let's get at least one more approval before merging, though.

[bstaletic]

-I should really be -isystem.

[henryiii]

I agree, but the pybind11-config and the Python-config both use -I I believe, so it would be a little odd if this causes pybind11 to be system but not Python.h.

[henryiii]

Suggested change

	then use ``$(python3-config --includes) -Iextern/pybind11/include``
	then use ``$(python3-config --includes) -isystem extern/pybind11/include``

For a simple example, not sure this is best, but it's also better to show the correct thing if it's visible.

[bstaletic]

Hm... You're right that pybind11-config and python-config both use -I. I'd argue that both of those are wrong. I personally use -isystem /usr/include/python3.9 and my cmake adds both with SYSTEM.

Anyway, I'm not super determined to change this -I to -isystem.

[YannickJadoul]

Thanks, @ssiano! This seems both more correct, and in a better place (i.e., under manual compiling, rather than seemingly suggesting to manually compile a submodule; CMake should still be preferred, if possible, I think) :-)

[ssiano]

Now I'm thinking that any new note in "First steps" should reference the comments in https://pybind11.readthedocs.io/en/latest/compiling.html#building-manually.

So please don't merge this for now.

[henryiii]

Those probably should be $() too.

[henryiii]

So please don't merge this for now.

Then convert to draft. :)

[henryiii]

Thanks!