Patch for linking libfabric

[amckinstry]

Linking libfabric breaks on Debian/Ubuntu systems (at least) without the following patch:

Author: Gianfranco Costamagna <[email protected]>

--- openmpi-2.0.1.orig/ompi/mca/mtl/ofi/Makefile.am
+++ openmpi-2.0.1/ompi/mca/mtl/ofi/Makefile.am
@@ -43,7 +43,7 @@ mca_mtl_ofi_la_SOURCES = $(mtl_ofi_sourc
mca_mtl_ofi_la_LDFLAGS = \
$(ompi_mtl_ofi_LDFLAGS) \
-module -avoid-version
-mca_mtl_ofi_la_LIBADD = $(ompi_mtl_ofi_LIBS) \
+mca_mtl_ofi_la_LIBADD = $(ompi_mtl_ofi_LIBS) $(opal_common_libfabric_LIBS) \
$(OPAL_TOP_BUILDDIR)/opal/mca/common/libfabric/lib@OPAL_LIB_PREFIX@mca_common_libfabric.la

noinst_LTLIBRARIES = $(component_noinst)

[ggouaillardet]

@amckinstry thanks for the patch

@rhc54 @jsquyres that is an intersting one ...

we link libmca_common_libfabric.la with -lfabric and naively hope libfabric.so is a dependency.
as shown by ldd, that is true on Centos 7, but not on debian (i tested ubuntu 14.04.3-LTS). if i understand correctly, the reason is libmca_common_libfabric.la does not need libfabric.so at all (!)

indeed, opal/mca/common/libfabric/common_libfabric.s only contains

int mca_common_libfabric_register_mca_variables(void)
{
    return OPAL_SUCCESS;
}

a possible fix is the suggested patch.
an other one is to really make libmca_common_libfabric.la depend on libfabric.so
for example

int mca_common_libfabric_register_mca_variables(void)
{
    if (fi_version() >= FI_VERSION(1,0)) {
        return OPAL_SUCCESS;
    } else {
        return OPAL_ERROR;
   }
}

and an other one is to simple remove opal/mca/common/libfabric (it does not do much today)

any thoughts ?

[rhc54]

There was some logic behind that component, but I honestly don't recall. I'd just use your patch for now.

[jsquyres]

@ggouaillardet Good call -- yes, calling fi_version() should do the trick.

[rashikakheria]

I am able to reproduce this issue using the OMPI v4.0.x on Ubuntu 16.04. I can see that the libmpi.so is not picking libfabric as its dynamic link.

ubuntu@ip-10-0-1-177:~/aws-ofi-nccl$ ldd /home/ubuntu/mpi-install/lib/libmpi.so
	linux-vdso.so.1 =>  (0x00007ffcd41e9000)
	libopen-rte.so.40 => /home/ubuntu/mpi-install/lib/libopen-rte.so.40 (0x00007ff897ea5000)
	libopen-pal.so.40 => /home/ubuntu/mpi-install/lib/libopen-pal.so.40 (0x00007ff897b91000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007ff897989000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff897680000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff897463000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff897099000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff896e95000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007ff896c92000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff898475000)

This causes basic MPI tests like ring to fail with the following error:

/home/ubuntu/ompi/examples/ring_c: symbol lookup error: /home/ubuntu/mpi-install/lib/openmpi/mca_mtl_ofi.so: undefined symbol: fi_dupinfo

Command used to configure OMPI build:

./configure --with-ofi=<absolute-path to libfbaric installation>

Any suggestions on fixing this?

[rashikakheria]

I am able to reproduce this issue using the OMPI v4.0.x on Ubuntu 16.04. I can see that the libmpi.so is not picking libfabric as its dynamic link.

ubuntu@ip-10-0-1-177:~/aws-ofi-nccl$ ldd /home/ubuntu/mpi-install/lib/libmpi.so
	linux-vdso.so.1 =>  (0x00007ffcd41e9000)
	libopen-rte.so.40 => /home/ubuntu/mpi-install/lib/libopen-rte.so.40 (0x00007ff897ea5000)
	libopen-pal.so.40 => /home/ubuntu/mpi-install/lib/libopen-pal.so.40 (0x00007ff897b91000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007ff897989000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff897680000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff897463000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff897099000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff896e95000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007ff896c92000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff898475000)

This causes basic MPI tests like ring to fail with the following error:

/home/ubuntu/ompi/examples/ring_c: symbol lookup error: /home/ubuntu/mpi-install/lib/openmpi/mca_mtl_ofi.so: undefined symbol: fi_dupinfo

Command used to configure OMPI build:

./configure --with-ofi=<absolute-path to libfbaric installation>

Any suggestions on fixing this?

[ggouaillardet]

Sounds like the fix never landed the repository ! Will do tomorrow.

[jsquyres]

libmpi.so is not supposed to link against libfabric.so -- mca_mtl_ofi.so is supposed to link against libfabric.so.

Specifically, if you ldd $libdir/openmpi/mca_mtl_ofi.so, I think you'll see that it links against some libfabric.so on your system. It likely would not have linked/installed, otherwise.

I suspect that the issue you're seeing here is that you are linking against and older version of libfabric that does not have the fi_dupinfo() call. If that really is the case, we should ameliorate this in the Open MPI code somehow (i.e., have configure check libfabric for fi_dupinfo() and if it doesn't have it, either code around it or disqualify that installation of libfabric).

What version of libfabric are you linking against?

You should open a new issue for this -- this existing issue was a different problem that was already resolved / closed.

[rashikakheria]

@jsquyres I checked output for ldd mca_mtl_ofi.so and can see it is missing libfabric linking

ubuntu@ip-10-0-1-177:~/mpi-install/lib/openmpi$ ldd mca_mtl_ofi.so
	linux-vdso.so.1 =>  (0x00007ffd6fad2000)
	libmpi.so.40 => /home/ubuntu/mpi-install/lib/libmpi.so.40 (0x00007fbce6098000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbce5e7b000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbce5ab1000)
	libopen-rte.so.40 => /home/ubuntu/mpi-install/lib/libopen-rte.so.40 (0x00007fbce57fa000)
	libopen-pal.so.40 => /home/ubuntu/mpi-install/lib/libopen-pal.so.40 (0x00007fbce54f1000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbce52e9000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbce4fe0000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fbce659a000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbce4ddc000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fbce4bd9000)

I have confirmed in the past that my libfabric build does have a fi_dupinfo() call. See below:

ubuntu@ip-10-0-1-177:~$ objdump -T ofi-install/lib/libfabric.so | grep fi_dupinfo
00000000000121aa g    DF .text	00000000000004b3  FABRIC_1.2  fi_dupinfo
0000000000017b77 g    DF .text	000000000000046d (FABRIC_1.0) fi_dupinfo
0000000000018100 g    DF .text	000000000000007c (FABRIC_1.1) fi_dupinfo

Also, I am using OFI v1.7.x

I saw that this issue was still open and hence wanted to confirm if it was ever merged. I can open a new issue for this.

[jsquyres]

That is pretty weird to me -- I have no idea how you would have an mca_mtl_ofi.so that does not link against libfabric. For example, here's my build from 4.0.0:

$ ldd ~/bogus/lib/openmpi/mca_mtl_ofi.so | grep fabric
        libfabric.so.1 => /home/jsquyres/libfabric-1.6.1/install/lib/libfabric.so.1 (0x00002aaaab9bd000)

Can you open a new issue and include the stdout from your configure, your config.log, and the stdout from make? (you might need to paste those large files into a gist or something)

[ggouaillardet]

@jsquyres per my previous analysis, libfabric.so is pulled indirectly from the common lib ... expect on ubuntu.

[matcabral]

Hi @rashikakheria, would you please share the details of your setup? I have Ubuntu 16.04.5 LTS (in KVM) but cannot reproduce the issue you are seeing. I'm building from master and I have libfabric v1.7.0.

thanks,

macabral@ubuntu16:/tmp/ompi$ git log -1 |head -1
commit ea40d48

macabral@ubuntu16:/tmp/ompi$ head config.log |grep -e "./configure"
$ ./configure --with-ofi --prefix=/tmp/ompi-master-git

macabral@ubuntu16:/tmp$ ldd /tmp/ompi-master-git/lib/openmpi/mca_mtl_ofi.so
linux-vdso.so.1 => (0x00007ffd0c3ba000)
libmpi.so.0 => /tmp/ompi-master-git/lib/libmpi.so.0 (0x00007f446bc19000)
libopen-rte.so.0 => /tmp/ompi-master-git/lib/libopen-rte.so.0 (0x00007f446b960000)
libopen-pal.so.0 => /tmp/ompi-master-git/lib/libopen-pal.so.0 (0x00007f446b64d000)
libfabric.so.1 => /usr/local/lib/libfabric.so.1 (0x00007f446b3b8000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f446b19b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f446add1000)
/lib64/ld-linux-x86-64.so.2 (0x00007f446c141000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f446abc9000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f446a8c0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f446a6bc000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f446a4b9000)

macabral@ubuntu16:/tmp/mpi_helloworld$ mpirun -np 2 -mca pml cm -mca mtl ofi -mca mtl_ofi_provider_include sockets ./mpi_helloworld
Hello world from host ubuntu16 processor ubuntu16, rank 0 out of 2 processors
Hello world from host ubuntu16 processor ubuntu16, rank 1 out of 2 processors

[rashikakheria]

@matcabral I tried the exact version you mentioned and still see the same issue

ubuntu@ip-10-0-1-51:~/ompi$ git log -1 |head -1
commit ea40d488993e3f1be8b7de943f4a751cfcfe37a6
ubuntu@ip-10-0-1-51:~/ompi$ head config.log |grep -e "./configure"
  $ ./configure --with-ofi --prefix=/home/ubuntu/ompi/install
ubuntu@ip-10-0-1-51:~/ompi$ ldd ./install/lib/openmpi/mca_mtl_ofi.so
	linux-vdso.so.1 =>  (0x00007ffd4cd68000)
	libmpi.so.0 => /home/ubuntu/ompi/install/lib/libmpi.so.0 (0x00007f14f1e0d000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f14f1bf0000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f14f1826000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f14f2335000)
	libopen-rte.so.0 => /home/ubuntu/ompi/install/lib/libopen-rte.so.0 (0x00007f14f156d000)
	libopen-pal.so.0 => /home/ubuntu/ompi/install/lib/libopen-pal.so.0 (0x00007f14f1259000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f14f1051000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f14f0d48000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f14f0b44000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f14f0941000)

I am also using Ubuntu 16.04 LTS. Here is my kernel version:

ubuntu@ip-10-0-1-51:~/ompi$ uname -a
Linux ip-10-0-1-51 4.4.0-1072-aws #82-Ubuntu SMP Fri Nov 2 15:00:21 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Let's follow further discussion on new issue as requested by @jsquyres

[jsquyres]

This is quite odd. I see the following:

ompi/ompi/mca/mtl/ofi/Makefile.am

Lines 78 to 80 in 0601b3e

	mca_mtl_ofi_la_LIBADD = $(top_builddir)/ompi/lib@[email protected] \
	$(ompi_mtl_ofi_LIBS) \
	$(OPAL_TOP_BUILDDIR)/opal/mca/common/ofi/lib@OPAL_LIB_PREFIX@mca_common_ofi.la

which shows that mca_mtl_ofi.la should be linking against both $(ompi_mtl_ofi_LIBS) libmca_common_ofi.la, and

ompi/opal/mca/common/ofi/Makefile.am

Line 74 in 0601b3e

lib@OPAL_LIB_PREFIX@mca_common_ofi_la_LIBADD = $(opal_common_ofi_LIBS)

which shows that libmca_common_ofi.la should be linking against $(opal_common_ofi_LIBS).

In my build:

• $(ompi_mtl_ofi_LIBS) is empty
• $(opal_common_ofi_LIBS) is -lfabric

And:

$ ldd mca_mtl_ofi.so
...
        libmca_common_ofi.so.0 => /home/jsquyres/bogus/lib/libmca_common_ofi.so.0 (0x00002aaaab958000)
        libfabric.so.1 => /home/jsquyres/libfabric-1.6.1/install/lib/libfabric.so.1 (0x00002aaaabb59000)
...

Showing that mca_mtl_ofi.so is linked against both libfabric and the OPAL common OFI library. And just for more fun:

$ ldd libmca_common_ofi.so.0
...
        libfabric.so.1 => /home/jsquyres/libfabric-1.6.1/install/lib/libfabric.so.1 (0x00002aaaaacaf000)
...

Showing that the OPAL common OFI library is linked against libfabric.

[jsquyres]

Blarg. I just deleted my last comment because it was wrong. In both cases, Libtool inserted /path/to/libfabric.so in the linker line. So in both cases, libmca_common_ofi.so was properly linked against libfabric.so.

[jsquyres]

Note that there was a second issue opened for a while and some discussion happened over there -- be sure to see #6360 for some additional content. We closed that issue and will continue the discussion here, just to keep it all together.

[jsquyres]

@rashikakheria Can you do this in your build tree:

$ cd opal/mca/common/ofi
$ rm libmca_common_ofi.la
$ make V=1

and send the output?

[rashikakheria]

Here is the output:

ubuntu@ip-10-0-1-51:~/ompi$ cd opal/mca/common/ofi
ubuntu@ip-10-0-1-51:~/ompi/opal/mca/common/ofi$ ls libmca_common_ofi.la
libmca_common_ofi.la
ubuntu@ip-10-0-1-51:~/ompi/opal/mca/common/ofi$ rm libmca_common_ofi.la
ubuntu@ip-10-0-1-51:~/ompi/opal/mca/common/ofi$ make V=1
/bin/bash ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG -Wall -Wundef -Wno-long-long -Wsign-compare -Wmissing-prototypes -Wstrict-prototypes -Wcomment -pedantic -Werror-implicit-function-declaration -finline-functions -fno-strict-aliasing -mcx16 -pthread  -version-info 0:0:0  -o libmca_common_ofi.la -rpath /home/ubuntu/ompi/install/lib  common_ofi.lo -lfabric  -lrt -lm -lutil
libtool: link: rm -fr  .libs/libmca_common_ofi.la .libs/libmca_common_ofi.lai .libs/libmca_common_ofi.so .libs/libmca_common_ofi.so.0 .libs/libmca_common_ofi.so.0.0.0
libtool: link: gcc -shared  -fPIC -DPIC  .libs/common_ofi.o   -lfabric -lrt -lm -lutil  -O3 -mcx16 -pthread   -pthread -Wl,-soname -Wl,libmca_common_ofi.so.0 -o .libs/libmca_common_ofi.so.0.0.0
libtool: link: (cd ".libs" && rm -f "libmca_common_ofi.so.0" && ln -s "libmca_common_ofi.so.0.0.0" "libmca_common_ofi.so.0")
libtool: link: (cd ".libs" && rm -f "libmca_common_ofi.so" && ln -s "libmca_common_ofi.so.0.0.0" "libmca_common_ofi.so")
libtool: link: ( cd ".libs" && rm -f "libmca_common_ofi.la" && ln -s "../libmca_common_ofi.la" "libmca_common_ofi.la" )
if test -z "libmca_common_ofi.la"; then \
  rm -f "libmca_common_ofi.la"; \
  ln -s "libmca_common_ofi_noinst.la" "libmca_common_ofi.la"; \
fi

[ggouaillardet]

the patch fixes the issue for me (up-to-date ubuntu xenial)

stock master does not depend on libfabric.so

gilles@ubuntu:~$ ldd local/ompi/lib/openmpi/mca_mtl_ofi.so 
	linux-vdso.so.1 =>  (0x00007fff3c5bb000)
	libmpi.so.0 => /home/gilles/local/ompi/lib/libmpi.so.0 (0x00007f4c0365d000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4c03439000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4c0306f000)
	/lib64/ld-linux-x86-64.so.2 (0x000055fe6ec89000)
	libopen-rte.so.0 => /home/gilles/local/ompi/lib/libopen-rte.so.0 (0x00007f4c02d4e000)
	libopen-pal.so.0 => /home/gilles/local/ompi/lib/libopen-pal.so.0 (0x00007f4c029e2000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f4c027da000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4c024d0000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4c022cc000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f4c020c9000)

patched master depends on libfabric.so

gilles@ubuntu:~$ ldd local/ompi.6313/lib/openmpi/mca_mtl_ofi.so 
	linux-vdso.so.1 =>  (0x00007ffdfa1cc000)
	libmpi.so.0 => /home/gilles/local/ompi.6313/lib/libmpi.so.0 (0x00007fce92560000)
	libfabric.so.1 => /home/gilles/local/libfabric-1.7.0/lib/libfabric.so.1 (0x00007fce92293000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fce9206f000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fce91ca4000)
	/lib64/ld-linux-x86-64.so.2 (0x0000563654d0e000)
	libopen-rte.so.0 => /home/gilles/local/ompi.6313/lib/libopen-rte.so.0 (0x00007fce91984000)
	libopen-pal.so.0 => /home/gilles/local/ompi.6313/lib/libopen-pal.so.0 (0x00007fce91618000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fce9140f000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fce91106000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fce90f02000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fce90cfe000)

here is the up-to-date patch

diff --git a/ompi/mca/mtl/ofi/Makefile.am b/ompi/mca/mtl/ofi/Makefile.am
index 2499f85..58c7ce2 100644
--- a/ompi/mca/mtl/ofi/Makefile.am
+++ b/ompi/mca/mtl/ofi/Makefile.am
@@ -5,6 +5,8 @@
 # Copyright (c) 2017      Los Alamos National Security, LLC.  All rights
 #                         reserved.
 # Copyright (c) 2017      IBM Corporation.  All rights reserved.
+# Copyright (c) 2019      Research Organization for Information Science
+#                         and Technology (RIST).  All rights reserved.
 # $COPYRIGHT$
 #
 # Additional copyrights may follow
@@ -18,7 +20,7 @@ EXTRA_DIST = post_configure.sh \
 MAINTAINERCLEANFILES = \
        $(generated_sources)
 
-AM_CPPFLAGS = $(ompi_mtl_ofi_CPPFLAGS) $(opal_common_ofi_CPPFLAGS)
+AM_CPPFLAGS = $(opal_common_ofi_CPPFLAGS)
 
 dist_ompidata_DATA = help-mtl-ofi.txt
 
@@ -55,7 +57,7 @@ mtl_ofi_sources = \
 # files should be added to generated_source_modules, as well as adding
 # their .c variants to generated_sources.
 %.c : %.pm;
-       $(PERL) generate-opt-funcs.pl $@
+       $(PERL) -I$(top_srcdir)/ompi/mca/mtl/ofi  $(top_srcdir)/ompi/mca/mtl/ofi/generate-opt-funcs.pl $@
 
 # Make the output library in this directory, and name it either
 # mca_<type>_<name>.la (for DSO builds) or libmca_<type>_<name>.la
@@ -73,15 +75,15 @@ mcacomponentdir = $(ompilibdir)
 mcacomponent_LTLIBRARIES = $(component_install)
 mca_mtl_ofi_la_SOURCES = $(mtl_ofi_sources)
 mca_mtl_ofi_la_LDFLAGS = \
-        $(ompi_mtl_ofi_LDFLAGS) \
+        $(opal_common_ofi_LDFLAGS) \
         -module -avoid-version
 mca_mtl_ofi_la_LIBADD = $(top_builddir)/ompi/lib@[email protected] \
-       $(ompi_mtl_ofi_LIBS) \
+        $(opal_common_ofi_LIBS) \
         $(OPAL_TOP_BUILDDIR)/opal/mca/common/ofi/lib@OPAL_LIB_PREFIX@mca_common_ofi.la
 
 noinst_LTLIBRARIES = $(component_noinst)
 libmca_mtl_ofi_la_SOURCES = $(mtl_ofi_sources)
 libmca_mtl_ofi_la_LDFLAGS = \
-        $(ompi_mtl_ofi_LDFLAGS) \
+        $(opal_common_ofi_LDFLAGS) \
         -module -avoid-version
-libmca_mtl_ofi_la_LIBADD = $(ompi_mtl_ofi_LIBS)
+libmca_mtl_ofi_la_LIBADD = $(opal_common_ofi_LIBS)

mca_mtl_ofi.so currently relies on libmca_common_ofi.so to access libfabric.so.

common/ofi does not use any symbols from libfabric.so.
on redhat, libmca_common_ofi.so does depend on libfabric.so (because this is what we tell the linker to do). even if we do the same thing on ubuntu, the libfabric.so dependency is skipped by libtool/ld, and hence the failure.

[jsquyres]

@ggouaillardet I think you just hit the nail on the head: common/ofi doesn't actually utilize any libfabric symbols.

This brings up (again) the idea that we should just delete common/ofi, because it never fulfilled its original purpose and just causes indirect problems like this. I think we talked about this in the last week or two on the weekly webex, but I don't think an issue was created for it.

EDIT: Correction -- we talked about this on the webex and I put a comment on #6313.

[ggouaillardet]

@jsquyres I will improve #6313 tomorrow (per Brian's comments)

[jsquyres]

@ggouaillardet I've got a solution in the works. Should have a PR shortly.

[jsquyres]

Please see PR #6363, which is a rollup of all known outstanding OFI configure/linking issues.