From a524bd97ad0284c83e818a02af8a52c5b354b868 Mon Sep 17 00:00:00 2001 From: Howard Pritchard Date: Tue, 6 Jun 2023 15:59:45 -0600 Subject: [PATCH] docs: update verbiage for using srun related to #10657 Signed-off-by: Howard Pritchard --- docs/launching-apps/slurm.rst | 37 ++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/docs/launching-apps/slurm.rst b/docs/launching-apps/slurm.rst index 17a0fff85e0..56466b219e7 100644 --- a/docs/launching-apps/slurm.rst +++ b/docs/launching-apps/slurm.rst @@ -61,33 +61,34 @@ that information directly from Slurm at run time. Using Slurm's "direct launch" functionality ------------------------------------------- -Assuming that Slurm installed its Open MPI plugin, you can use +Assuming that Slurm was configured with its PMIx plugin, you can use ``srun`` to "direct launch" Open MPI applications without the use of Open MPI's ``mpirun`` command. -.. note:: Using direct launch can be *slightly* faster when launching - very, very large MPI processes (i.e., thousands or millions of MPI - processes in a single job). But it has significantly fewer - features than Open MPI's ``mpirun``. +First, you must ensure that Slurm was built and installed with PMIx +support. This can determined as shown below: -First, you must ensure that Slurm was built and installed with PMI-2 -support. +.. code-block:: sh + + shell$ srun --mpi=list + MPI plugin types are... + none + pmi2 + pmix + specific pmix plugin versions available: pmix_v4 -.. note:: Please ask your friendly neighborhood Slurm developer to - support PMIx. PMIx is the current generation of run-time - support API; PMI-2 is the legacy / antiquated API. Open MPI - *only* supports PMI-2 for Slurm. +The output from ``srun`` may vary somewhat depending on the version of Slurm installed. +If PMIx is not present in the output, then you will not be able to use srun +to launch Open MPI applications. -Next, ensure that Open MPI was configured ``--with-pmi=DIR``, where -``DIR`` is the path to the directory where Slurm's ``pmi2.h`` is -located. +.. note:: PMI-2 is not supported in Open MPI 5.0.0 and later releases. -Open MPI applications can then be launched directly via the ``srun`` -command. For example: +Provided the Slurm installation includes the PMIx plugin, Open MPI applications +can then be launched directly via the ``srun`` command. For example: .. code-block:: sh - shell$ srun -N 4 mpi-hello-world + shell$ srun -N 4 --mpi=pmix mpi-hello-world Or you can use ``sbatch`` with a script: @@ -95,7 +96,7 @@ Or you can use ``sbatch`` with a script: shell$ cat my_script.sh #!/bin/sh - srun mpi-hello-world + srun --mpi=pmix mpi-hello-world shell$ sbatch -N 4 my_script.sh srun: jobid 1235 submitted shell$