osc/ucx: Move the ucx_mca_register/deregister to open/close

[jjhursey]

• In the scenario where UCX is built into Open MPI, but not used at runtime
  then the memory hooks infrastructure can get into a situation where it has
  an invalid function pointer resulting in a segv.
• Move the opal_common_ucx_mca_register and opal_common_ucx_mca_deregister
  calls to the component open and close functions, just like the UCX PML.

[jjhursey]

This one was tricky and hand to do with component opening/initializing/finalizing/closing, unloading in a scenario that is probably not commonly tested.

From what I can tell, the problem in v4.1.x is that the common/ucx is being unloaded while the opal_common_ucx_mem_release_cb memory hook is still registered. So when a memory release happens in the finalization of the OSC components it encounters an invalid function pointer (the memory it points to is no longer valid). So it segv's.

I'm running in a Docker container without IB. So the default pml_ucx_tls will exclude the ucx PML because there is no device match. If I set export OMPI_MCA_pml_ucx_tls=tcp,self,sysv,posix then the ucx PML is selected, and the test case runs fine without a segv.

The test case was simple: mpirun -np 2 examples/hello_c

Thus the scenario is when the ucx PML is loaded and disqualifies itself (due to the pml_ucx_tls default), and the ucx OSC is loaded and also disqualifies itself. A scenario where UCX is built in, but cannot be used at runtime.

The difference is that the ucx PML calls opal_common_ucx_mca_register and opal_common_ucx_mca_deregister in mca_pml_ucx_component_open and mca_pml_ucx_component_close, respectively. However, usc OSC calls opal_common_ucx_mca_register in component_init instead of open, and (most critically) calls opal_common_ucx_mca_deregister in component_finalize (instead of in a component_close like the PML) which is called during MPI_Finalize.

This probably did not impact main or v5.0.x because they don't rely as strongly on DSOs, unlike the older branches. However, it is good to apply the fix to main and v5.0.x to keep things uniform across the branches.

[janjust]

@jjhursey Thanks for the fix! Can you open the equivalent v4.1 and v5.0 PR?

[jjhursey]

Done:

• v5.0.x PR osc/ucx: Move the ucx_mca_register/deregister to open/close #10764
• v4.1.x PR osc/ucx: Move the ucx_mca_register/deregister to open/close #10765