FAQs for github issue #6517

[ca-taylor]

Please review and merge this commit if you find it acceptable or let me know what how it should be changed.

[jsquyres]

@ca-taylor Can you put a signed-off-by line in your commit message? We use that to indicate you agree to our open source license, yadda yadda yadda (see https://github.com/open-mpi/ompi/blob/master/.github/CONTRIBUTING.md#contributors-declaration).

I pushed a 2nd commit on this PR that contains a bunch of minor tweaks.

If everyone is kosher with them, I can squash the two commits together and we can merge.

[ca-taylor]

I'm fine with the changes with the following caveats.

1. I used the <pre> tags rather than the <geshi> tags to avoid the numbering of lines in the error message. I think there is a "PHP" way to do it but I I tried didn't work.

2. My recollections is that forcing the UCX PML via MCA parameters made no difference and we still got the error message but I won't swear to it. I'm just pretty sure I tried that without success.

[jsquyres]

@ca-taylor Good points.

1. I think using geshi for the error messages is ok. Most everything in the FAQ these days uses geshi instead of <pre>. E.g., https://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages. Although looking through the FAQ, the other examples don't include the lines of dashes in the error messages, so I'll take those out.
2. Oh! I'd be (unpleasantly) surprised if mpirun --mca pml ucx ... and mpirun --mca btl '^openib' ... didn't prevent those messages. Can you confirm?

[ca-taylor]

1. Using just mpirun --mca pml ucx results in the error message and warning.
2. Using just export OMPI_MCA_btl=^vader,tcp,openib prevents the error message and warning.
3. Using both mpirun --mca pml ucx and OMPI_MCA_btl=^vader,tcp,openib_ also prevents the error message and warning.

So I probably tried specifying the UCX PML without explicitly excluding the openib BTL which, in my mind, should also work but I guess not.

[jsquyres]

Ooohhh.... I know why. Because openib is used for more than just MPI point-to-point (i.e., PML-related things): it's also used for MPI one-sided things.

Ok, so just the --mca btl '^openib' works, but not --mca pml ucx. Got it -- I'll adjust and squash.

[jsquyres]

Ok, I did the following:

• removed the -mca pml ucx.. example from the "this works" section
• added -mca pml ucx... section to "this doesn't work section at the end
• Added a "Signed off by" line for @ca-taylor

@ca-taylor Is that ok?

[ca-taylor]

@jsquyres Re: OK - Yes, that should work.