Find libcuda.so automatically if --with-cuda-lib is not passed

[nsarka]

In newer OpenMPI versions, it's required to pass both --with-cuda and --with-cuda-libdir in order for CUDA to be recognized by the build system. Ideally, just --with-cuda should be enough and the build system should be able to detect where the CUDA libraries are.

This PR allows removes the requirement that --with-cuda-libdir be passed and instead makes the build system search for libcuda.so inside of the directory passed with --with-cuda.

See issue: #12264

[github-actions]

Hello! The Git Commit Checker CI bot found a few problems with this PR:

5bcaba0: Find libcuda.so automatically if --with-cuda-lib i...

• check_signed_off: does not contain a valid Signed-off-by line

Please fix these problems and, if necessary, force-push new commits back up to the PR branch. Thanks!

[hppritcha]

this works for me at nersc perlmutter. I load cudatoolkit module which sets CUDA_HOME=/opt/nvidia/hpc_sdk/Linux_x86_64/23.9/cuda/12.2. In my configure line I have

./configure blah blah --with-cuda=$CUDA_HOME

then when doing the libtool link think its linking in libcuda.so at /opt/nvidia/hpc_sdk/Linux_x86_64/23.9/cuda/12.2/targets/x86_64-linux/lib/stubs/libcuda.so,
but at runtime, because LD_LIBRARY_PATH is set to prepend another location for libcuda.so.1 i get

hpp@login19:~/ompi/install_pr12378/lib/openmpi> (nsarka/cuda-aware-ompi-find-libs-automatically)ldd mca_accelerator_cuda.so 
	linux-vdso.so.1 (0x00007ffcc2304000)
	libopen-pal.so.0 => /global/homes/h/hpp/ompi/install_pr12378/lib/libopen-pal.so.0 (0x00007fc4aeff6000)
	libfabric.so.1 => /opt/cray/libfabric/1.15.2.0/lib64/libfabric.so.1 (0x00007fc4aec00000)
	libxpmem.so.0 => /opt/cray/xpmem/default/lib64/libxpmem.so.0 (0x00007fc4aebd8000)
	libcuda.so.1 => /usr/local/cuda-12.2/compat/libcuda.so.1 (0x00007fc4acf6b000)

so this patch is doing what we'd like on perlmutter but the fixes for protecting for cases where just --with-cuda or --without-cuda need to be handled before I can approve this PR.

[github-actions]

Hello! The Git Commit Checker CI bot found a few problems with this PR:

cfec4a9: Update to check /usr/local/cuda in case with_cuda ...

• check_signed_off: does not contain a valid Signed-off-by line

Please fix these problems and, if necessary, force-push new commits back up to the PR branch. Thanks!

[nsarka]

Hi @hppritcha, I have updated the PR to handle the case where $with_cuda is not a directory.

[nsarka]

Ahh, I did not see the approval before force pushing again. The reason I force pushed was to add redirection to /dev/null in case that dirname command couldn't find the directory. It's not helpful output, just "dirname: missing argument".

[hppritcha]

Ahh, I did not see the approval before force pushing again. The reason I force pushed was to add redirection to /dev/null in case that dirname command couldn't find the directory. It's not helpful output, just "dirname: missing argument".

this tweak looks good to me.