Skip to content

Remove openib and affiliated stuff #6270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 4 additions & 62 deletions README
Original file line number Diff line number Diff line change
Expand Up @@ -623,7 +623,6 @@ MPI Functionality and Features
- portals4

(2) The ob1 PML and the following BTLs support MPI_THREAD_MULTIPLE:
- openib (see exception below)
- self
- sm
- smcuda
Expand All @@ -632,10 +631,6 @@ MPI Functionality and Features
- usnic
- vader (shared memory)

The openib BTL's RDMACM based connection setup mechanism is also not
thread safe. The default UDCM method should be used for
applications requiring MPI_THREAD_MULTIPLE support.

Currently, MPI File operations are not thread safe even if MPI is
initialized for MPI_THREAD_MULTIPLE support.

Expand Down Expand Up @@ -794,7 +789,8 @@ Network Support
- In prior versions of Open MPI, InfiniBand and RoCE support was
provided through the openib BTL and ob1 PML plugins. Starting with
Open MPI 4.0.0, InfiniBand support through the openib plugin is both
deprecated and superseded by the ucx PML component.
deprecated and superseded by the ucx PML component. The openib BTL
was removed in Open MPI v5.0.0.

While the openib BTL depended on libibverbs, the UCX PML depends on
the UCX library.
Expand All @@ -809,15 +805,6 @@ Network Support
for OpenSHMEM support, and "--mca osc ucx" for MPI RMA (one-sided)
operations.

- Although the ob1 PML+openib BTL is still the default for iWARP and
RoCE devices, it will reject InfiniBand defaults (by default) so
that they will use the ucx PML. If using the openib BTL is still
desired, set the following MCA parameters:

# Note that "vader" is Open MPI's shared memory BTL
$ mpirun --mca pml ob1 --mca btl openib,vader,self \
--mca btl_openib_allow_ib 1 ...

- The usnic BTL is support for Cisco's usNIC device ("userspace NIC")
on Cisco UCS servers with the Virtualized Interface Card (VIC).
Although the usNIC is accessed via the OpenFabrics Libfabric API
Expand Down Expand Up @@ -850,8 +837,8 @@ Network Support

http://lwn.net/Articles/343351/

- The use of fork() with OpenFabrics-based networks (i.e., the openib
BTL) is only partially supported, and only on Linux kernels >=
- The use of fork() with OpenFabrics-based networks (i.e., the UCX
PML) is only partially supported, and only on Linux kernels >=
v2.6.15 with libibverbs v1.1 or later (first released as part of
OFED v1.2), per restrictions imposed by the OFED network stack.

Expand Down Expand Up @@ -1206,51 +1193,6 @@ NETWORKING SUPPORT / OPTIONS
--with-usnic
Abort configure if Cisco usNIC support cannot be built.

--with-verbs=<directory>
Specify the directory where the verbs (also known as OpenFabrics
verbs, or Linux verbs, and previously known as OpenIB) libraries and
header files are located. This option is generally only necessary
if the verbs headers and libraries are not in default
compiler/linker search paths.

The Verbs library usually implies operating system bypass networks,
such as InfiniBand, usNIC, iWARP, and RoCE (aka "IBoIP").

--with-verbs-libdir=<directory>
Look in directory for the verbs libraries. By default, Open MPI
will look in <verbs_directory>/lib and <verbs_ directory>/lib64,
which covers most cases. This option is only needed for special
configurations.

--with-verbs-usnic
Note that this option is no longer necessary in recent Linux distro
versions. If your Linux distro uses the "rdma-core" package (instead
of a standalone "libibverbs" package), not only do you not need this
option, you shouldn't use it, either. More below.

This option will activate support in Open MPI for disabling a
dire-sounding warning message from libibverbs that Cisco usNIC
devices are not supported (because Cisco usNIC devices are supported
through libfabric, not libibverbs). This libibverbs warning can
also be suppressed by installing the "no op" libusnic_verbs plugin
for libibverbs (see https://github.com/cisco/libusnic_verbs, or
download binaries from cisco.com).

This option is disabled by default for two reasons:

1. It causes libopen-pal.so to depend on libibverbs.so, which is
undesirable to many downstream packagers.
2. As mentioned above, recent versions of the libibverbs library
(included in the "rdma-core" package) do not have the bug that
will emit dire-sounding warnings about usnic devices. Indeed,
the --with-verbs-usnic option will enable code in Open MPI that
is actually incompatible with rdma-core (i.e., cause Open MPI to
fail to compile).

If you enable --with-verbs-usnic and your system uses the rdma-core
package, configure will safely abort with a helpful message telling
you that you should not use --with-verbs-usnic.


RUN-TIME SYSTEM SUPPORT

Expand Down
1 change: 0 additions & 1 deletion VERSION
Original file line number Diff line number Diff line change
Expand Up @@ -114,4 +114,3 @@ libmca_opal_common_ofi_so_version=0:0:0
libmca_opal_common_sm_so_version=0:0:0
libmca_opal_common_ucx_so_version=0:0:0
libmca_opal_common_ugni_so_version=0:0:0
libmca_opal_common_verbs_so_version=0:0:0
Loading