Issue #390 add ProcessBuilder and openeo.processes docs

Author: soxofaan

CHANGELOG.md

@@ -26,6 +26,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   ([#386](https://github.com/Open-EO/openeo-python-client/issues/386)).
   The (experimental) `fit_class_random_forest()` and `fit_regr_random_forest()` methods
   moved accordingly to the `VectorCube` class.
+- Improved documentation on `openeo.processes` and `ProcessBuilder`
+  ([#390](https://github.com/Open-EO/openeo-python-client/issues/390)).
 
 ### Removed
 

docs/_static/custom.css

@@ -104,3 +104,7 @@ pre {
     font-size: 75%;
     padding: 0 1ex;
 }
+
+nav.contents.local {
+    border: none;
+}

docs/api-processbuilder.rst

@@ -0,0 +1,87 @@
+..  FYI this file is intended to be inlined (with "include" RST directive)
+    in the ProcessBuilder class doc block,
+    which in turn is covered with autodoc/automodule from api-processes.rst.
+
+
+The :py:class:`ProcessBuilder <openeo.processes.ProcessBuilder>` class
+is a helper class that implements
+(much like the :ref:`openEO process functions <openeo_processes_functions>`)
+each openEO process as a method.
+On top of that it also adds syntactic sugar to support Python operators as well
+(e.g. ``+`` is translated to the ``add`` process).
+
+.. attention::
+    As normal user, you should never create a
+    :py:class:`ProcessBuilder <openeo.processes.ProcessBuilder>` instance
+    directly.
+
+    You should only interact with this class inside a callback
+    function/lambda while building a child callback process graph
+    as discussed at :ref:`child_callback_callable`.
+
+
+For example, let's start from this simple usage snippet
+where we want to reduce the temporal dimension
+by taking the temporal mean of each timeseries:
+
+.. code-block:: python
+
+    def my_reducer(data):
+        return data.mean()
+
+    cube.reduce_dimension(reducer=my_reducer, dimension="t")
+
+Note that this ``my_reducer`` function has a ``data`` argument,
+which conceptually corresponds to an array of pixel values
+(along the temporal dimension).
+However, it's important to understand that the ``my_reducer`` function
+is actually *not evaluated when you execute your process graph*
+on an openEO back-end, e.g. as a batch jobs.
+Instead, ``my_reducer`` is evaluated
+*while building your process graph client-side*
+(at the time you execute that ``cube.reduce_dimension()`` statement to be precise).
+This means that that ``data`` argument is actually not a concrete array of EO data,
+but some kind of *virtual placeholder*,
+a :py:class:`ProcessBuilder <openeo.processes.ProcessBuilder>` instance,
+that keeps track of the operations you intend to do on the EO data.
+
+To make that more concrete, it helps to add type hints
+which will make it easier to discover what you can do with the argument
+(depending on which editor or IDE you are using):
+
+.. code-block:: python
+
+    from openeo.processes import ProcessBuilder
+
+    def my_reducer(data: ProcessBuilder) -> ProcessBuilder:
+        return data.mean()
+
+    cube.reduce_dimension(reducer=my_reducer, dimension="t")
+
+
+Because :py:class:`ProcessBuilder <openeo.processes.ProcessBuilder>` methods
+return new :py:class:`ProcessBuilder <openeo.processes.ProcessBuilder>` instances,
+and because it support syntactic sugar to use Python operators on it,
+and because :ref:`openeo.process functions <openeo_processes_functions>`
+also accept and return :py:class:`ProcessBuilder <openeo.processes.ProcessBuilder>` instances,
+we can mix methods, functions and operators in the callback function like this:
+
+.. code-block:: python
+
+    from openeo.processes import ProcessBuilder, cos
+
+    def my_reducer(data: ProcessBuilder) -> ProcessBuilder:
+        return cos(data.mean()) + 1.23
+
+    cube.reduce_dimension(reducer=my_reducer, dimension="t")
+
+or compactly, using an anonymous lambda expression:
+
+.. code-block:: python
+
+    from openeo.processes import cos
+
+    cube.reduce_dimension(
+        reducer=lambda data: cos(data.mean())) + 1.23,
+        dimension="t"
+    )

docs/api-processes.rst

@@ -0,0 +1,69 @@
+=========================
+API: ``openeo.processes``
+=========================
+
+The ``openeo.processes`` module contains building blocks and helpers
+to construct so called "child callbacks" for openEO processes like
+:py:meth:`openeo.rest.datacube.DataCube.apply` and
+:py:meth:`openeo.rest.datacube.DataCube.reduce_dimension`,
+as discussed at :ref:`child_callback_callable`.
+
+.. note::
+    The contents of the ``openeo.processes`` module is automatically compiled
+    from the official openEO process specifications.
+    Developers that want to fix bugs in, or add implementations to this
+    module should not touch the file directly, but instead address it in the
+    upstream `openeo-processes <https://github.com/Open-EO/openeo-processes>`_ repository
+    or in the internal tooling to generate this file.
+
+
+.. contents:: Sections:
+   :depth: 1
+   :local:
+   :backlinks: top
+
+
+.. _openeo_processes_functions:
+
+Functions in ``openeo.processes``
+---------------------------------
+
+The ``openeo.processes`` module implements (at top-level)
+a regular Python function for each openEO process
+(not only the official stable ones, but also experimental ones in "proposal" state).
+
+These functions can be used directly as child callback,
+for example as follows:
+
+.. code-block:: python
+
+    from openeo.processes import absolute, max
+
+    cube.apply(absolute)
+    cube.reduce_dimension(max, dimension="t")
+
+
+Note how the signatures of the parent :py:class:`DataCube <openeo.rest.datacube.DataCube>` methods
+and the callback functions match up:
+
+-   :py:meth:`DataCube.apply() <openeo.rest.datacube.DataCube.apply>`
+    expects a callback that receives a single numerical value,
+    which corresponds to the parameter signature of :py:func:`openeo.processes.absolute`
+-   :py:meth:`DataCube.reduce_dimension() <openeo.rest.datacube.DataCube.reduce_dimension>`
+    expects a callback that receives an array of numerical values,
+    which corresponds to the parameter signature :py:func:`openeo.processes.max`
+
+
+.. automodule:: openeo.processes
+    :members:
+    :exclude-members: ProcessBuilder, process, _process
+
+
+``ProcessBuilder`` helper class
+--------------------------------
+
+..  FYI the ProcessBuilder docs are provided through its doc block
+    with an RST "include" of "api-processbuilder.rst"
+
+.. automodule:: openeo.processes
+    :members: ProcessBuilder

docs/api.rst

@@ -1,6 +1,6 @@
-===
-API
-===
+=============
+API (General)
+=============
 
 High level Interface
 --------------------

docs/index.rst

@@ -60,6 +60,7 @@ Table of contents
    configuration
    cookbook/index
    api
+   api-processes
    process_mapping
    development
    best_practices

docs/process_mapping.py

@@ -9,6 +9,7 @@
 import datetime
 import sys
 from textwrap import dedent
+import importlib
 
 from openeo.internal.documentation import _process_registry
 
@@ -38,6 +39,14 @@ def main():
             *   - openEO process
                 - openEO Python Client Method
     """))
+    # Import some submodules to make sure `_process_registry` is populated
+    for mod in [
+        "openeo.rest.datacube",
+        "openeo.rest.vectorcube",
+        "openeo.rest.mlmodel",
+        "openeo.processes",
+    ]:
+        importlib.import_module(mod)
 
     for process_id in sorted(_process_registry.keys()):
         functions = [x[0] for x in _process_registry[process_id]]