Compilation fails for powerpc with --enable-mpi-cxx #2055

amckinstry · 2016-09-05T13:53:35Z

Hi,

Enabling --enable-mpi-cxx leads to:

make[2]: Entering directory '/home/mckinstry/openmpi-2.0.1/ompi/mpi/cxx'
/bin/bash ../../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1112/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1112/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c  -DOMPI_BUILDING_CXX_BINDINGS_LIBRARY=1 -DOMPI_SKIP_MPICXX=1 -I../../.. -I../../../orte/include -Wdate-time -D_FORTIFY_SOURCE=2   -I/usr//include -I/home/mckinstry/openmpi-2.0.1/opal/mca/event/libevent2022/libevent -I/home/mckinstry/openmpi-2.0.1/opal/mca/event/libevent2022/libevent/include  -DNDEBUG -g -O2 -fdebug-prefix-map=/home/mckinstry/openmpi-2.0.1=. -fstack-protector-strong -Wformat -Werror=format-security -finline-functions -pthread -c -o mpicxx.lo mpicxx.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1112/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1112/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c -DOMPI_BUILDING_CXX_BINDINGS_LIBRARY=1 -DOMPI_SKIP_MPICXX=1 -I../../.. -I../../../orte/include -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr//include -I/home/mckinstry/openmpi-2.0.1/opal/mca/event/libevent2022/libevent -I/home/mckinstry/openmpi-2.0.1/opal/mca/event/libevent2022/libevent/include -DNDEBUG -g -O2 -fdebug-prefix-map=/home/mckinstry/openmpi-2.0.1=. -fstack-protector-strong -Wformat -Werror=format-security -finline-functions -pthread -c mpicxx.cc  -fPIC -DPIC -o .libs/mpicxx.o
In file included from ../../../opal/class/opal_object.h:128:0,
                 from ../../../ompi/errhandler/errhandler.h:32,
                 from mpicxx.cc:37:
../../../opal/threads/thread_usage.h: In function 'int32_t opal_thread_swap_32(volatile int32_t*, int32_t)':
../../../opal/threads/thread_usage.h:134:76: error: 'opal_atomic_swap_32' was not declared in this scope
         return opal_atomic_swap_ ## suffix ((volatile type *) ptr, newvalue); \

This fails because in opal/include/opal/sys/atomic.h:

 #if defined(c_plusplus) || defined(__cplusplus)
/* We no longer support inline assembly for C++ as OPAL is a C-only interface */
#define opal_atomic_swap_32 0
#define OPAL_DEC_INLINE_ASSEMBLY 0
#define OPAL_XLC_INLINE_ASSEMBLY 0

and opal_atomic_swap_32 is conditionally only included if opal_atomic_swap_32 is set

The text was updated successfully, but these errors were encountered:

jsquyres · 2016-09-06T13:30:00Z

@hjelmn Is this related to #2051 and/or #2044?

gpaulsen · 2016-09-06T13:55:05Z

@nysal - can you or someone on your team look at this?

hjelmn · 2016-09-06T14:42:18Z

This isn't PowerPC specific. I used C99 in a header exposed to c++. I need to figure out a way to protect internal headers from c++ until the bindings are removed. will look at this today.

jsquyres · 2016-09-06T15:07:47Z

@hjelmn and I chatted on the call today -- he has an interesting proposal to fix this: require a C++11 compiler for the C++ MPI bindings.

My gut reaction is "we can't do that!", but Nathan raises the valid point that the MPI C++ bindings have been removed from the MPI 3.0 spec. So raising the bar a little to compile the MPI C++ bindings may not be horrible. Need to think about this a little bit...

hjelmn · 2016-09-06T18:51:49Z

Yeah, if we do C++11 I think it is ok to include any of our internal headers to C++. My understanding is many of the C99 features we use in Open MPI are available in C++11.

hjelmn · 2016-09-07T15:09:14Z

I can't get g++ 6.1 to accept C99 subobject naming. Guess I was wrong that it is supported. Working on another workaround now.

This commit removes the #include of errhandler/errhandler.h (which includes opal_object.h). The inlude was only needed to get the extern declaration of ompi_mpi_errors_throw_exceptions. Since we have the ompi_predefined_errhandler_t in mpi.h we can declare the extern in mpicxx.c without needing the C errhandler header. This should fix open-mpi#2055 Signed-off-by: Nathan Hjelm <[email protected]>

This commit adds some glue code to support the C++ bindings and updates the bindings to use the new glue code. This protects our internal headers (which are C99) from C++. This is done as a quick workaround to compilation errors when the legacy C++ bindings are requested. Fixes open-mpi#2055 Signed-off-by: Nathan Hjelm <[email protected]>

This commit adds some glue code to support the C++ bindings and updates the bindings to use the new glue code. This protects our internal headers (which are C99) from C++. This is done as a quick workaround to compilation errors when the legacy C++ bindings are requested. Fixes open-mpi#2055 Signed-off-by: Nathan Hjelm <[email protected]> (cherry picked from commit c6464ca)

sharkcz · 2016-10-26T21:47:07Z

Just tried to build master (it passes) and the v2.0.x branch (it fails the same way as described here) on ppc64 Fedora 25. Has been the backport from master complete?

sharkcz · 2016-10-27T10:13:53Z

For the record - build with --enable-mpi-cxx passes on ppc64 from the master branch even before the"mpi/cxx: isolate internal headers from C++ bindings #2064" fix has been merged, I have tried with commit 7601e78

jsquyres · 2016-10-27T14:55:50Z

@sharkcz I just merged a v2.0.x compile failures with regards to the C++ bindings (#2295). Does this fix the issue for you?

sharkcz · 2016-10-27T15:47:54Z

Still the same problem with the opal_atomics(), I'm currently bisecting the 2.0.x branch to find the problematic change.

jsquyres · 2016-10-27T15:49:47Z

@gpaulsen @jjhursey Can you have a look?

sharkcz · 2016-10-27T17:58:22Z

and this is the result of my bisecting
[f07fddb] atomic/ppc: add atomics for load-link, store-conditional, and swap
with this commit the build fails in my environment
Fedora 25 for ppc64 running on "PowerNV palmetto" machine (Tyan GN70-BP010), Power8 CPU
gcc-6.2.1-2.fc25.ppc64

sharkcz · 2016-10-27T19:24:13Z

and for the record also the error message

Making all in mpi/cxx
make[2]: Entering directory '/home/sharkcz/projects/ompi/ompi/mpi/cxx'
depbase=`echo mpicxx.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ../../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c  -DOMPI_BUILDING_CXX_BINDINGS_LIBRARY=1 -DOMPI_SKIP_MPICXX=1 -I../../.. -I../../../orte/include    -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent/include  -g -Wall -Wundef -Wno-long-long -finline-functions -pthread -MT mpicxx.lo -MD -MP -MF $depbase.Tpo -c -o mpicxx.lo mpicxx.cc &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo intercepts.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ../../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c  -DOMPI_BUILDING_CXX_BINDINGS_LIBRARY=1 -DOMPI_SKIP_MPICXX=1 -I../../.. -I../../../orte/include    -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent/include  -g -Wall -Wundef -Wno-long-long -finline-functions -pthread -MT intercepts.lo -MD -MP -MF $depbase.Tpo -c -o intercepts.lo intercepts.cc &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo comm.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ../../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c  -DOMPI_BUILDING_CXX_BINDINGS_LIBRARY=1 -DOMPI_SKIP_MPICXX=1 -I../../.. -I../../../orte/include    -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent/include  -g -Wall -Wundef -Wno-long-long -finline-functions -pthread -MT comm.lo -MD -MP -MF $depbase.Tpo -c -o comm.lo comm.cc &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c -DOMPI_BUILDING_CXX_BINDINGS_LIBRARY=1 -DOMPI_SKIP_MPICXX=1 -I../../.. -I../../../orte/include -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent/include -g -Wall -Wundef -Wno-long-long -finline-functions -pthread -MT comm.lo -MD -MP -MF .deps/comm.Tpo -c comm.cc  -fPIC -DPIC -o .libs/comm.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c -DOMPI_BUILDING_CXX_BINDINGS_LIBRARY=1 -DOMPI_SKIP_MPICXX=1 -I../../.. -I../../../orte/include -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent/include -g -Wall -Wundef -Wno-long-long -finline-functions -pthread -MT mpicxx.lo -MD -MP -MF .deps/mpicxx.Tpo -c mpicxx.cc  -fPIC -DPIC -o .libs/mpicxx.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1111/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c -DOMPI_BUILDING_CXX_BINDINGS_LIBRARY=1 -DOMPI_SKIP_MPICXX=1 -I../../.. -I../../../orte/include -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent -I/home/sharkcz/projects/ompi/opal/mca/event/libevent2022/libevent/include -g -Wall -Wundef -Wno-long-long -finline-functions -pthread -MT intercepts.lo -MD -MP -MF .deps/intercepts.Tpo -c intercepts.cc  -fPIC -DPIC -o .libs/intercepts.o
In file included from ../../../opal/include/opal_config_bottom.h:251:0,
                 from ../../../opal/include/opal_config.h:2797,
                 from ../../../ompi/include/ompi_config.h:29,
                 from ../../../ompi/communicator/communicator.h:31,
                 from comm.cc:30:
../../../opal/include/opal_stdint.h:58:32: warning: ‘-pedantic’ is not an option that controls warnings [-Wpragmas]
 #pragma GCC diagnostic ignored "-pedantic"
                                ^~~~~~~~~~~
In file included from ../../../opal/include/opal_config_bottom.h:251:0,
                 from ../../../opal/include/opal_config.h:2797,
                 from ../../../ompi/include/ompi_config.h:29,
                 from mpicxx.cc:26:
../../../opal/include/opal_stdint.h:58:32: warning: ‘-pedantic’ is not an option that controls warnings [-Wpragmas]
 #pragma GCC diagnostic ignored "-pedantic"
                                ^~~~~~~~~~~
In file included from ../../../opal/include/opal_config_bottom.h:251:0,
                 from ../../../opal/include/opal_config.h:2797,
                 from ../../../ompi/include/ompi_config.h:29,
                 from intercepts.cc:26:
../../../opal/include/opal_stdint.h:58:32: warning: ‘-pedantic’ is not an option that controls warnings [-Wpragmas]
 #pragma GCC diagnostic ignored "-pedantic"
                                ^~~~~~~~~~~
In file included from ../../../opal/class/opal_free_list.h:28:0,
                 from ../../../ompi/communicator/comm_request.h:16,
                 from ../../../ompi/communicator/communicator.h:35,
                 from comm.cc:30:
../../../opal/class/opal_lifo.h: In function ‘opal_list_item_t* opal_lifo_pop_atomic(opal_lifo_t*)’:
../../../opal/class/opal_lifo.h:191:73: error: ‘opal_atomic_swap_32’ was not declared in this scope
         if (opal_atomic_swap_32((volatile int32_t *) &item->item_free, 1)) {
                                                                         ^
In file included from ../../../opal/class/opal_free_list.h:28:0,
                 from ../../../ompi/communicator/comm_request.h:16,
                 from ../../../ompi/communicator/communicator.h:35,
                 from intercepts.cc:28:
../../../opal/class/opal_lifo.h: In function ‘opal_list_item_t* opal_lifo_pop_atomic(opal_lifo_t*)’:
../../../opal/class/opal_lifo.h:191:73: error: ‘opal_atomic_swap_32’ was not declared in this scope
         if (opal_atomic_swap_32((volatile int32_t *) &item->item_free, 1)) {
                                                                         ^
Makefile:1795: recipe for target 'comm.lo' failed
make[2]: *** [comm.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
Makefile:1795: recipe for target 'intercepts.lo' failed
make[2]: *** [intercepts.lo] Error 1
make[2]: Leaving directory '/home/sharkcz/projects/ompi/ompi/mpi/cxx'
Makefile:3312: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/sharkcz/projects/ompi/ompi'
Makefile:1788: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1

hjelmn · 2016-10-27T21:37:08Z

Yup. We use c99 internally and the c++ was exposed to it. Some compilers do ok but others do not. We fixed it by isolating the legacy c++ bindings from the c99 on master and now 2.x and 2.0.x.

jsquyres · 2016-10-27T21:38:12Z

@hjelmn He's saying it's not fixed.

sharkcz · 2016-10-27T22:07:33Z

Respectively it's fixed in master, but it is not in the 2.0.x branch, haven't tried 2.x branch yet.

sharkcz · 2016-10-28T20:11:42Z

The difference between master and v2.0.x branch is that the problematic opal/include/opal/sys/powerpc/atomic.h header isn't used at all in the master branch. I have saved the preprocessed copies of ompi/mpi/cxx/comm.c if they should be helpful.

hjelmn · 2016-10-28T21:33:04Z

Looks like we have an erroneous include in intercepts.cc still. The powerpc header is not used by default in master because we made (a perhaps poor) decision to use the builtin compiler atomics by default. The powerpc/atomic.h header is legal C99 which is not supported by C++.

hjelmn · 2016-10-28T21:42:35Z

FYI, the C++ bindings in MPI were removed in MPI-3. We will do what is needed to keep them compiling in 2.x but they may not survive to 3.0.

Fixes open-mpi#2055 Signed-off-by: Nathan Hjelm <[email protected]>

hjelmn · 2016-10-28T21:58:38Z

Try patching with the commit in #2317 and see if that does it.

This commit adds some glue code to support the C++ bindings and updates the bindings to use the new glue code. This protects our internal headers (which are C99) from C++. This is done as a quick workaround to compilation errors when the legacy C++ bindings are requested. Fixes open-mpi#2055 Signed-off-by: Nathan Hjelm <[email protected]>

jjhursey · 2017-05-02T16:23:55Z

Per comment on #2317 there is still more work to do (checking against the v2.1.0 tarball with that patch)

hppritcha · 2017-07-03T15:28:51Z

@jjhursey any updates on this issue?

jjhursey · 2017-07-05T17:03:05Z

I can confirm that this is fixed in master by PR #3589. I've started builds to also confirm the fix is correct for the release branches:

v3.0.x : PR v3.0.x: mpi/cxx: remove nonexistent function from cxx glue #3592
v2.x : PR v2.1.x: mpi/cxx: remove nonexistent function from cxx glue #3590
v2.0.x : PR v2.0.x: mpi/cxx: remove nonexistent function from cxx glue #3591

We can close this issue, and I'll followup with the results of the release branch testing.

jjhursey · 2017-07-05T18:16:00Z

I verified correct compilation with --enable-mpi-cxx on all of the active release branches (v2.0.x, v2.x, v3.0.x)

sharkcz · 2017-07-12T12:16:17Z

I can confirm I got a successful build with --enable-mpi-cxx on ppc64 from the v2.0.x branch, thanks to all.

Ref: open-mpi#2055

jsquyres added this to the v2.0.2 milestone Sep 6, 2016

jsquyres added the bug label Sep 6, 2016

jsquyres assigned hjelmn, jjhursey and gpaulsen Sep 6, 2016

gpaulsen assigned nysal Sep 6, 2016

hjelmn mentioned this issue Sep 7, 2016

mpi/cxx: limit exposure of internal header to C++ #2062

Closed

hjelmn mentioned this issue Sep 7, 2016

mpi/cxx: isolate internal headers from C++ bindings #2064

Merged

hjelmn closed this as completed in #2064 Oct 4, 2016

hjelmn mentioned this issue Oct 4, 2016

mpi/cxx: isolate internal headers from C++ bindings #2162

Merged

hjelmn mentioned this issue Oct 4, 2016

mpi/cxx: isolate internal headers from C++ bindings #2163

Merged

hjelmn added a commit to hjelmn/ompi that referenced this issue Oct 28, 2016

ompi/comm: make communicator.h safe for C++ bindings

6a197b1

Fixes open-mpi#2055 Signed-off-by: Nathan Hjelm <[email protected]>

hjelmn mentioned this issue Oct 28, 2016

ompi/comm: make communicator.h safe for C++ bindings #2317

Closed

jjhursey reopened this May 2, 2017

hppritcha modified the milestones: v2.0.4, v2.0.2 Jun 1, 2017

jjhursey closed this as completed Jul 5, 2017

smuzaffar mentioned this issue Sep 15, 2017

openmpi 2.1.1 --enable-mpi-cxx failed to build on aarch64 cms-sw/cmsdist#3421

Closed

MingcongBai added a commit to AOSC-Tracking/ompi that referenced this issue Feb 3, 2024

opal: fix GCC in-line assembly for PowerPC

5cf7b0a

Ref: open-mpi#2055

Compilation fails for powerpc with --enable-mpi-cxx #2055

Compilation fails for powerpc with --enable-mpi-cxx #2055

Comments

amckinstry commented Sep 5, 2016

jsquyres commented Sep 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gpaulsen commented Sep 6, 2016

Uh oh!

hjelmn commented Sep 6, 2016

Uh oh!

jsquyres commented Sep 6, 2016

Uh oh!

hjelmn commented Sep 6, 2016

Uh oh!

hjelmn commented Sep 7, 2016

Uh oh!

sharkcz commented Oct 26, 2016

Uh oh!

sharkcz commented Oct 27, 2016

Uh oh!

jsquyres commented Oct 27, 2016

Uh oh!

sharkcz commented Oct 27, 2016

Uh oh!

jsquyres commented Oct 27, 2016

Uh oh!

sharkcz commented Oct 27, 2016

Uh oh!

sharkcz commented Oct 27, 2016

Uh oh!

hjelmn commented Oct 27, 2016

Uh oh!

jsquyres commented Oct 27, 2016

Uh oh!

sharkcz commented Oct 27, 2016

Uh oh!

sharkcz commented Oct 28, 2016

Uh oh!

hjelmn commented Oct 28, 2016

Uh oh!

hjelmn commented Oct 28, 2016

Uh oh!

hjelmn commented Oct 28, 2016

Uh oh!

jjhursey commented May 2, 2017

Uh oh!

hppritcha commented Jul 3, 2017

Uh oh!

jjhursey commented Jul 5, 2017

Uh oh!

jjhursey commented Jul 5, 2017

Uh oh!

sharkcz commented Jul 12, 2017

Uh oh!

jsquyres commented Sep 6, 2016 •

edited

Loading