Argobots and Qthread configure detection need fixing/improvements

[devreal]

Working with current master (g560ebc5780), I am trying to compile Open MPI with either Argobots or Qthread support and noticed the following:

1. There had been a patch to improve the detection of Argobots in configure (https://github.com/open-mpi/ompi/pull/7675/files) but the current state does not seem to work. I am passing --with-argobots=$HOME/opt/argobots-1.0/ to configure but all I am getting is the following message in config.log:

configure:5285: +++ Configuring MCA framework threads
configure:153773: checking for no configure components in framework threads
configure:153775: result: 
configure:153777: checking for m4 configure components in framework threads
configure:153779: result: argobots, pthreads, qthreads
configure:5300: --- MCA component threads:argobots (m4 configuration macro, priority 30)
configure:153867: checking for MCA component threads:argobots compile mode
configure:153870: result: static 
configure:156050: checking if MCA component threads:argobots can compile
configure:156052: result: no

Any ideas how to get more information on what is going wrong? I don't see a failed compilation attempt anywhere in the logs...

2. There is no way to provide configure with a path to a custom qthread installation. Something along the lines of --with-qthreads=/path/to/qthreads/ should be provided.

[shintaro-iwasaki]

Thank you for taking a look at this issue (I am also keeping track of #8037).

For Argobots, although the Argobots version has some issues on multithreading tests, which is still under investigation, at least compilation and configuration succeeded on our CI environment last weekend (https://jenkins-pmrs.cels.anl.gov/job/ompi-weekly-argobots/compiler=gnu,label=argobots_centos64/24/consoleFull).

The configuration line is as follows:

/tmp/jenkins5540533621955127193.sh:52> ./configure '--with-threads=argobots' \
'--with-argobots=/var/lib/jenkins-slave/workspace/ompi-weekly-argobots/compiler/gnu/label/argobots_centos64/argobots' \
'--with-libevent=/var/lib/jenkins-slave/workspace/ompi-weekly-argobots/compiler/gnu/label/argobots_centos64/libevent' \
'--prefix=/var/lib/jenkins-slave/workspace/ompi-weekly-argobots/compiler/gnu/label/argobots_centos64/ompi' \
'--with-cray-xpmem=no'

where ...argobots_centos64/argobots contains include and lib. This test used a pretty new Open MPI main branch pulled on September 6th. It also used the latest Argobots main branch, but Open MPI + Argobots should be compiled with Argobots 1.0 as well. I do not think --with-cray-xpmem and --with-libevent are required for configuration and compilation.

[devreal]

@shintaro-iwasaki Ahh yes, it was a user error 🤦 I forgot to specify --with-threads=argobots and only provided --with-argobots=.... I shouldn't post late-night bug reports...

The situation with qthreads seems more dire though. I installed current qthreads master from github and set CFLAGS/LDFLAGS accordingly to configure OMPI with qthreads enabled. Compilation fails with two different errors:

../../../opal/mca/threads/qthreads/threads_qthreads_mutex.h:129:64: error: ‘OPAL_SUCCESS’ undeclared (first use in this function); did you mean ‘EXIT_SUCCESS’?
     return 0 == qthread_readFE_nb(NULL, &m->m_lock_qthreads) ? OPAL_SUCCESS
                                                                ^~~~~~~~~~~~

and

../../../opal/mca/threads/qthreads/threads_qthreads_mutex.h: In function ‘opal_mutex_trylock’:
../../../opal/mca/threads/qthreads/threads_qthreads_mutex.h:129:17: warning: implicit declaration of function ‘qthread_readFE_nb’; did you mean ‘qthread_readFE’? [-Wimplicit-function-declaration]
     return 0 == qthread_readFE_nb(NULL, &m->m_lock_qthreads) ? OPAL_SUCCESS
                 ^~~~~~~~~~~~~~~~~
                 qthread_readFE

Indeed, I only see qthread_readFE contained in the headers that were installed by qthreads.

[hppritcha]

@devreal yes qthread configury needs work. i may be able to get to that this week.

[olivier-snl]

@npe9 : Looking at the above compile output, the nonblocking FEBs seem to be at issue. The regular public-facing FEB functions are in include/qthread/qthread.h, but the nb versions are in include/qt_feb.h. Though they are marked API_FUNC, it is still odd for public-facing functions to not be in the include/qthread subdir. Oversight?

[devreal]

I checked and found that qt_feb.h was not installed on my system. Do I need to ask configure to build the nonblocking FEBs?

[olivier-snl]

@devreal qt_feb.h is a Qthreads internal implementation include file, not a public-facing interface include file. What we need to do is move that function into a public-facing interface file, probably qthread.h, since other public feb functions are there. I am hoping to get some feedback from the original developer of the Open MPI - Qthreads interface ( @npe9 ) to see if there is a good reason why it did't get put there before.