upirun (Open MPI) 5.1.0a1 Usage: mpirun [OPTION]... Initiate an instance of the PMIx Reference RTE (PRRTE) DVM /***** General Options *****/ -h|--help This help message -h|--help Help for the specified option @@ Help text is displayed and seems reasonable for both variations @@ mpirun recognizes this option -v|--verbose Enable typical debug options @@ Help text is displayed and seems reasonable. @@ mpirun seems to accept the -v or -V options, but I'm not sure they do anything. -vv and -vvv are @@ flagged as invalid options --vv and --vvv respectively. -V|--version Print version and exit @@ Help text is displayed but is identical to the above text @@ mpirun recognizes both options /***** Debug Options *****/ --debug-daemons Debug daemons - if not set, the "verbose" setting will be limited to the DVM controller to reduce clutter @@ Help text is displayed and help text is reasonable @@ Option is recognized by mpirun --debug-daemons-file Enable debugging of any PRTE daemons used by this application, storing their verbose output in files @@ Help text is displayed, but it only says output is stored in files. It doesn't give filename or @@ filename pattern. @@ mpirun seems to recognize the option, but I cannot find any output file, even if I also use the @@ --host option to run on a remote host --display Comma-delimited list of options for displaying information about the allocation and job. Allowed values: allocation, bind, map, map-devel, topo --get-stack-traces Get stack traces of all application procs on timeout @@ Help text is displayed but is identical to the above text. @@ mpirun recognizs the option. --leave-session-attached Do not discard stdout/stderr of remote PRTE daemons @@ Help text is displayed but is identical with the one line help in the preceding line, so doesn't @@ add any value. Help text is confusing to me since I don't understand why asking to leave session @@ attached has anything with discarding stdout/stderr. I'd expect it to request leaving some daemon @@ running. @@ mpirun accepts this option, but I'm not sure it's doing anything, even with a remote host. --report-state-on-timeout Report all job and process states upon timeout @@ Help text is displayed but is identical to the above text. @@ mpirun recognizes the option. --spawn-timeout Timeout the job if spawn takes more than the specified number of seconds @@ Help text is displayed but is identical to the aove text. @@ mpirun recognizes this option (--spawn-timeout 0) --stop-on-exec If supported, stop each specified process at start of execution @@ Help text is displayed but is essentially identical to the above option. @@ mpirun recognizes the option and handles it --stop-in-init Direct the specified processes to stop in PMIx_Init @@ Help text is displayed and seems reasonable. @@ mpirun accepts the option, but --stop-in-init 0 doesn't seem to do anything. --stop-in-app Direct the specified processes to stop at an application-controlled location @@ Help text is displayed, but there is no mention of how the application-determined point is specified. @@ mpirun recognizes the option, but does not accept the parameter mentioned in the help text, and if @@ the parameter is omitted, mpirun doesn't seem to do anything. --test-suicide Suicide instead of clean abort after delay --timeout Timeout the job after the specified number of seconds @@ Help text is displayed and seems reasonable. @@ mpirun recognizes the option. --output-proctable Print the complete proctable to stdout [-], stderr [+], or a file [anything else] after launch /***** Output Options *****/ --output Comma-delimited list of options that control how output is generated.Allowed values: tag, timestamp, xml, merge-stderr-to-stdout, dir=DIRNAME, file=filename. The dir option redirects output from application processes into DIRNAME/job/rank/std[out,err,diag]. The file option redirects output from application processes into filename.rank. In both cases, the provided name will be converted to an absolute path. Supported qualifiers include NOCOPY (do not copy the output to the stdout/err streams). --report-child-jobs-separately Return the exit status of the primary job only @@ Help text is displayed but is identical to the above text @@ mpirun accepts this option. --xterm Create a new xterm window and display output from the specified ranks there @@ Help text is displayed and seems reasonable @@ mpirun accepts the option, but I'm not sure if it does anything. My environment may not allow me to @@ create xterms. /***** Input Options *****/ --stdin Specify procs to receive stdin [rank, all, none] (default: 0, indicating rank 0) /***** Mapping Options *****/ --map-by Mapping Policy for job [slot | hwthread | core (default:np<=2) | l1cache | l2cache | l3cache | numa (default:np>2) | package | node | seq | dist | ppr |,rankfile] with supported colon-delimited modifiers: PE=y (for multiple cpus/proc), SPAN, OVERSUBSCRIBE, NOOVERSUBSCRIBE, NOLOCAL, HWTCPUS, CORECPUS, DEVICE(for dist policy), INHERIT, NOINHERIT, PE-LIST=a,b (comma-delimited ranges of cpus to use for this job), FILE= for seq and rankfile options /***** Ranking Options *****/ --rank-by Ranking Policy for job [slot (default:np<=2) | hwthread | core | l1cache | l2cache | l3cache | numa (default:np>2) | package | node], with modifier :SPAN or :FILL /***** Binding Options *****/ --bind-to Binding policy for job. Allowed values: none, hwthread, core, l1cache, l2cache, l3cache, numa, package, ("none" is the default when oversubscribed, "core" is the default when np<=2, and "numa" is the default when np>2). Allowed colon-delimited qualifiers: overload-allowed, if-supported /***** Developer Options *****/ --do-not-launch Perform all necessary operations to prepare to launch the application, but do not actually launch it (usually used to test mapping patterns) /***** Launch Options *****/ --default-hostfile Provide a default hostfile -H|--host List of hosts to invoke processes on --hostfile Provide a hostfile --initial-errhandler Specify the initial error handler that is attached to predefined communicators during the first MPI call. --machinefile Provide a hostfile --mca Pass context-specific MCA parameters; they are considered global if --gmca is not used and only one context is specified (arg0 is the parameter name; arg1 is the parameter value) --path PATH to be used to look for executables to start processes --pmixmca Pass context-specific PMIx MCA parameters; they are considered global if @@ Help text is displayed and help text is reasonable @@ Option is recognized by mpirun only one context is specified (arg0 is the parameter name; arg1 is the parameter value) --gpmixmca Pass global PMIx MCA parameters that are applicable to all contexts (arg0 is the parameter name; arg1 is the parameter value) --preload-files Preload the comma separated list of files to the remote machines current working directory before starting the remote process. --prtemca Pass context-specific PRTE MCA parameters to the DVM @@ Help text is displayed and help text is reasonable @@ Option is recognized by mpirun --pset User-specified name assigned to the processes in their given application --rankfile Name of file to specify explicit task mapping -s|--preload-binary Preload the binary on the remote machine before starting the remote process. --set-cwd-to-session-dir Set the working directory of the started processes to their session directory --show-progress Output a brief periodic report on launch progress --stream-buffering Adjust buffering for stdout/stderr [0 unbuffered] [1 line buffered] [2 fully buffered] --wd Synonym for --wdir --wdir Set the working directory of the started processes -x Export an environment variable, optionally specifying a value (e.g., "-x foo" exports the environment variable foo and takes its value from the current environment; "-x foo=bar" exports the environment variable name foo and sets its value to "bar" in the started processes; "-x foo*" exports all current environmental variables starting with "foo") /***** Specific Options *****/ --allow-run-as-root Allow execution as root (STRONGLY DISCOURAGED) @@ Help text is displayed but is identical to the above line. @@ mpirun accepts this option but I'm not sure it does anything since I'm not prompted for a password @@ and the process I ran still runs as my userid, not root. --daemonize Daemonize the DVM daemons into the background --forward-signals Comma-delimited list of additional signals (names or integers) to forward to application processes ["none" => forward nothing]. Signals provided by default include SIGTSTP, SIGUSR1, SIGUSR2, SIGABRT, SIGALRM, and SIGCONT @@ Help text is displayed but is identical to the above text. @@ mpirun recognizes this option but only accepts certain signals. Some, like SIGSEGV result in an error @@ message that they are not recognized signals. Others like SIGINT result in an error message that @@ the system does ot trap them. --keepalive Pipe to monitor - DVM will terminate upon closure --launch-agent Name of daemon executable used to start processes on remote nodes (default: prted) @@ Help text is displayed and help text is reasonable @@ Option is recognized by mpirun but if I specify an invalid executable, like 'zzz', then no error @@ message is issued and the application runs on the remote node specified by --host. --max-vm-size Number of daemons to start @@ Help text is displayed and help text is reasonable @@ Option is recognized by mpirun --no-ready-msg Do not print a DVM ready message --noprefix Disable automatic --prefix behavior @@ Help text is displayed but is identical to the above line. 'automatic --prefix behavior' is not @@ explained. @@ mpirun accepts the option. --personality Specify the personality to be used @@ Help text is displayed but is identical to the above text. @@ mpirun accepts this option, but seems to accept anything, like --personality xxx, without error --prefix Prefix to be used to look for RTE executables @@ Help text is displayed, but is identical to the above line. Maybe it should say something about @@ prefix being the root directory for the MPI installation. @@ mpirun does accept the option. --report-pid Printout pid on stdout [-], stderr [+], or a file [anything else] --report-uri Printout URI on stdout [-], stderr [+], or a file [anything else] --set-sid Direct the DVM daemons to separate from the current session --singleton ID of the singleton process that started us --system-server Start the DVM as the system server --tmpdir Set the root for the session directory tree @@ Help text is displayed, but is identical to the above line. I'm not sure what the relation between @@ a temp dir and a session root directory is (apparently they are the same) @@ mpirun seems to accept the --tmpdir option. --tune File(s) containing MCA params for tuning DVM operations @@ Help text is displayed, but it is not clear from help text that the string must be 'parm=value'. @@ 'parm = value' results in a syntax error message and then a second error message @@ No executable was specified on the prterun command line. @@ where prterun should be mpirun. /***** Fault Tolerance Options (if enabled) *****/ --enable-recovery Enable recovery from process failure [Default = disabled] --max-restarts Max number of times to restart a failed process --disable-recovery Disable recovery (resets all recovery options to off) --continuous Job is to run until explicitly terminated --with-ft Specify the type(s) of error handling that the application will use. /***** MPI Options *****/ --initial-errhandler Specify the initial error handler that is attached to predefined communicators during the first MPI call. --display-comm Display table of communication methods between ranks during MPI_Init --display-comm-finalize "Display table of communication methods between ranks during MPI_Finalize --soft This option is unsupported, but mandated by the MPI standard --arch This option is unsupported, but mandated by the MPI standard --file This option is unsupported, but mandated by the MPI standard Report bugs to https://www.open-mpi.org/community/help/