pytypes: Add doc section and tests about interaction with None

Author: EricCousineau-TRI

docs/advanced/pycpp/object.rst

@@ -13,6 +13,11 @@ Available types include :class:`handle`, :class:`object`, :class:`bool_`,
 :class:`iterable`, :class:`iterator`, :class:`function`, :class:`buffer`,
 :class:`array`, and :class:`array_t`.
 
+.. warning::
+
+    You should be aware of how these classes interact with :func:`py::none`.
+    See :ref:`pytypes_interaction_with_none` for more details.
+
 Casting back and forth
 ======================
 
@@ -168,3 +173,62 @@ Generalized unpacking according to PEP448_ is also supported:
     Python functions from C++, including keywords arguments and unpacking.
 
 .. _PEP448: https://www.python.org/dev/peps/pep-0448/
+
+.. _pytypes_interaction_with_none:
+
+Interaction with None
+=====================
+
+You may be tempted to use types like ``py::str`` and ``py::dict`` in C++
+signatures (either pure C++, or in bound signatures). However, there are some
+"gotchas" for ``py::none()`` and how it interacts with these types. In best
+case scenarios, it will fail fast (e.g. with default arguments); in worst
+cases, it will silently work but corrupt the types you want to work with.
+
+At a first glance, you may think after executing the following code, the
+expression ``my_value.is(py::none())`` will be true:
+
+.. code-block:: cpp
+
+    py::str my_value = py::none();
+
+However, this is not the case. Instead, the value of ``my_value`` will be equal
+to the Python value of ``str(None)``, due to how :macro:`PYBIND11_OBJECT_CVT`
+is used in :file:`pybind11/pytypes.h`.
+
+Additionally, calling the following binding with the default argument used will
+raise a ``TypeError`` about invalid arguments:
+
+.. code-block:: cpp
+
+    m.def(
+        "my_function",
+        [](py::str my_value) { ... },
+        py::arg("my_value") = py::none());
+
+In both of these cases where you may want to pass ``None`` through any
+signatures where you want to constrain the type, you should either use
+:class:`py::object` in conjunction with :func:`py::isinstance`, or use the
+corresponding C++ type with `std::optional` (if it is available on your
+system).
+
+For the above conversion:
+
+.. code-block:: cpp
+
+    py::object my_value = /* py::none() or some string */;
+    ...
+    if (!my_value.is(py::none()) && !py::isinstance<py::str>(my_value)) {
+        /* error behavior */
+    }
+
+For the above default argument:
+
+.. code-block:: cpp
+
+    m.def(
+        "my_function",
+        [](std::optional<std::string> my_value) { ... },
+        py::arg("my_value") = std::nullopt);
+
+For more details, see the tests for ``pytypes`` mentioned above.

tests/test_pytypes.cpp

@@ -319,6 +319,24 @@ TEST_SUBMODULE(pytypes, m) {
         return a[py::slice(0, -1, 2)];
     });
 
+    // See #2361
+    // These four tests should reflect the text in `object.rst`, the
+    // "Interaction with None" section.
+    m.def("test_str_with_default_arg_none" ,[](py::str value) {
+        return value;
+    }, py::arg("value") = py::none());
+    m.def("test_str_assign_none", []() {
+        py::str is_this_none = py::none();
+        return is_this_none;
+    });
+    m.def("test_dict_with_default_arg_none", [](py::dict value) {
+        return value;
+    }, py::arg("value") = py::none());
+    m.def("test_dict_assign_none", []() {
+        py::dict is_this_none = py::none();
+        return is_this_none;
+    });
+
     m.def("test_memoryview_object", [](py::buffer b) {
         return py::memoryview(b);
     });

tests/test_pytypes.py

@@ -281,6 +281,21 @@ def test_list_slicing():
     assert li[::2] == m.test_list_slicing(li)
 
 
+def test_pytypes_with_none():
+    # See issue #2361
+    assert m.test_str_assign_none() == "None"
+    with pytest.raises(TypeError) as excinfo:
+        m.test_str_with_default_arg_none()
+    assert "incompatible function arguments" in str(excinfo.value)
+
+    with pytest.raises(TypeError) as excinfo:
+        assert m.test_dict_assign_none()
+    assert "'NoneType' object is not iterable" in str(excinfo.value)
+    with pytest.raises(TypeError) as excinfo:
+        m.test_dict_with_default_arg_none()
+    assert "incompatible function arguments" in str(excinfo.value)
+
+
 @pytest.mark.parametrize('method, args, fmt, expected_view', [
     (m.test_memoryview_object, (b'red',), 'B', b'red'),
     (m.test_memoryview_buffer_info, (b'green',), 'B', b'green'),