Question regarding the requirement for OpenMPI to be compiled with --disable-dlopen

[samfux84]

Hi,

Sorry for bothering you with that many questions.

I could now resolve almost all issues and can consistently build EMAN 2.3 from source, both, using dependencies built with CONDA and using dependencies built without CONDA.

The last missing piece is to make EMAN 2.3 using our centrally provided OpenMPI as it was compiled with support for our batch system (LSF).

Why do you have a requirement to have OpenMPI built with --disable-dlopen ? Unfortunately, our MPI is compiled with --enable-dlopen. This parameter determines, if the OpenMPI plugins are built as separate libraries (DSO) or if they are added to the main MPI library.

I don't understand how this can affect a software using OpenMPI. Shouldn't a software work independent of how OpenMPI is compiled with regards to DSO ?

Best regards and that you for the help you provided so far.

Sam

[sludtke42]

If memory serves, dlopen() is not compatible with Python. That is, Python libraries cannot link to OpenMPI if it has been compiled with this option. This requirement was added probably 6-8 years ago, so it is conceivable that it no longer exists. You're welcome to try it, but you may well run into this issue.

…

-------------------------------------------------------------------------------------- Steven Ludtke, Ph.D. <sludtke@bcm.edu> Baylor College of Medicine Charles C. Bell Jr., Professor of Structural Biology Dept. of Biochemistry and Molecular Biology (www.bcm.edu/biochem) Academic Director, CryoEM Core (cryoem.bcm.edu) Co-Director CIBR Center (www.bcm.edu/research/cibr)

On Jul 11, 2019, at 8:21 AM, Samuel Fux ***@***.***> wrote: Hi, Sorry for bothering you with that many questions. I could now resolve almost all issues and can consistently build EMAN 2.3 from source, both, using dependencies built with CONDA and using dependencies built without CONDA. The last missing piece is to make EMAN 2.3 using our centrally provided OpenMPI as it was compiled with support for our batch system (LSF). Why do you have a requirement to have OpenMPI built with --disable-dlopen ? Unfortunately, our MPI is compiled with --enable-dlopen. This parameter determines, if the OpenMPI plugins are built as separate libraries (DSO) or if they are added to the main MPI library. I don't understand how this can affect a software using OpenMPI. Shouldn't a software work independent of how OpenMPI is compiled with regards to DSO ? Best regards and that you for the help you provided so far. Sam — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#414?email_source=notifications&email_token=ACPKF2SJDYKHBTYTDORTK53P64XVPA5CNFSM4IBFW4TKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G6UFJ2A>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACPKF2SGK53DLX7GNCFPLEDP64XVPANCNFSM4IBFW4TA>.

[samfux84]

@sludtke42 : Thank you for your reply. I already tested yesterday to use our system OpenMPI (built with --enable-dlopen). EMAN 2.3 builds fine, but when running the test with "make test", I get an "undefined symbol error":

/cluster/apps/gcc-4.8.5/cmake-3.9.0-5yzdokxge3hk65gdzdfxw7wdzsv4pnfl/bin/ctest -V -C Release
UpdateCTestConfiguration  from :/scratch/2409628.tmpdir/eman2-2.3/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/scratch/2409628.tmpdir/eman2-2.3/build/DartConfiguration.tcl
Test project /scratch/2409628.tmpdir/eman2-2.3/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: imports

1: Test command: /cluster/apps/python/2.7.14/bin/python "/scratch/2409628.tmpdir/eman2-2.3/tests/test_imports.py"
1: Test timeout computed to be: 9.99988e+06
1: Traceback (most recent call last):
1:   File "/scratch/2409628.tmpdir/eman2-2.3/tests/test_imports.py", line 3, in <module>
1:     from libpyAligner2 import Aligners
1: ImportError: /cluster/apps/python/2.7.14/lib64/python2.7/site-packages/libEM2.so: undefined symbol: _ZN3MPI8Datatype4FreeEv
1/5 Test #1: imports ..........................***Failed    0.11 sec
test 2
    Start 2: test-EMAN2DIR

2: Test command: /cluster/apps/python/2.7.14/bin/python "/scratch/2409628.tmpdir/eman2-2.3/tests/test_EMAN2DIR.py"
2: Test timeout computed to be: 9.99988e+06
2: Traceback (most recent call last):
2:   File "/scratch/2409628.tmpdir/eman2-2.3/tests/test_EMAN2DIR.py", line 4, in <module>
2:     import EMAN2
2:   File "/cluster/apps/python/2.7.14/lib64/python2.7/site-packages/EMAN2.py", line 51, in <module>
2:     from EMAN2_cppwrap import *
2:   File "/cluster/apps/python/2.7.14/lib64/python2.7/site-packages/EMAN2_cppwrap.py", line 35, in <module>
2:     from libpyAligner2 import *
2: ImportError: /cluster/apps/python/2.7.14/lib64/python2.7/site-packages/libEM2.so: undefined symbol: _ZN3MPI8Datatype4FreeEv
2/5 Test #2: test-EMAN2DIR ....................***Failed    0.38 sec

Checking the libEM2.so with readelf, showed that those symbols are not defined in the library:

[sfux@lo-a2-017 libEM]$ readelf -a libEM2.so | grep _ZN3MPI8Datatype4FreeEv
   116: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _ZN3MPI8Datatype4FreeEv
  4457: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _ZN3MPI8Datatype4FreeEv
[sfux@lo-a2-017 libEM]$

I am not really an OpenMPI expert, therefore I cannot judge, if this is caused by the dlopen issue.

Again, thank you for your help.

[sludtke42]

That looks to me more like a problem with using different compilers/versions on different parts of the source code? I could be wrong, but I don't think that's the dlopen issue you're seeing.

…

-------------------------------------------------------------------------------------- Steven Ludtke, Ph.D. <sludtke@bcm.edu> Baylor College of Medicine Charles C. Bell Jr., Professor of Structural Biology Dept. of Biochemistry and Molecular Biology (www.bcm.edu/biochem) Academic Director, CryoEM Core (cryoem.bcm.edu) Co-Director CIBR Center (www.bcm.edu/research/cibr)

On Jul 11, 2019, at 8:52 AM, Samuel Fux ***@***.***> wrote: @sludtke42 <https://github.com/sludtke42> : Thank you for your reply. I already tested yesterday to use our system OpenMPI (built with --enable-dlopen). EMAN 2.3 builds fine, but when running the test with "make test", I get an "undefined symbol error": /cluster/apps/gcc-4.8.5/cmake-3.9.0-5yzdokxge3hk65gdzdfxw7wdzsv4pnfl/bin/ctest -V -C Release UpdateCTestConfiguration from :/scratch/2409628.tmpdir/eman2-2.3/build/DartConfiguration.tcl UpdateCTestConfiguration from :/scratch/2409628.tmpdir/eman2-2.3/build/DartConfiguration.tcl Test project /scratch/2409628.tmpdir/eman2-2.3/build Constructing a list of tests Done constructing a list of tests Updating test list for fixtures Added 0 tests to meet fixture requirements Checking test dependency graph... Checking test dependency graph end test 1 Start 1: imports 1: Test command: /cluster/apps/python/2.7.14/bin/python "/scratch/2409628.tmpdir/eman2-2.3/tests/test_imports.py" 1: Test timeout computed to be: 9.99988e+06 1: Traceback (most recent call last): 1: File "/scratch/2409628.tmpdir/eman2-2.3/tests/test_imports.py", line 3, in <module> 1: from libpyAligner2 import Aligners 1: ImportError: /cluster/apps/python/2.7.14/lib64/python2.7/site-packages/libEM2.so: undefined symbol: _ZN3MPI8Datatype4FreeEv 1/5 Test #1: imports ..........................***Failed 0.11 sec test 2 Start 2: test-EMAN2DIR 2: Test command: /cluster/apps/python/2.7.14/bin/python "/scratch/2409628.tmpdir/eman2-2.3/tests/test_EMAN2DIR.py" 2: Test timeout computed to be: 9.99988e+06 2: Traceback (most recent call last): 2: File "/scratch/2409628.tmpdir/eman2-2.3/tests/test_EMAN2DIR.py", line 4, in <module> 2: import EMAN2 2: File "/cluster/apps/python/2.7.14/lib64/python2.7/site-packages/EMAN2.py", line 51, in <module> 2: from EMAN2_cppwrap import * 2: File "/cluster/apps/python/2.7.14/lib64/python2.7/site-packages/EMAN2_cppwrap.py", line 35, in <module> 2: from libpyAligner2 import * 2: ImportError: /cluster/apps/python/2.7.14/lib64/python2.7/site-packages/libEM2.so: undefined symbol: _ZN3MPI8Datatype4FreeEv 2/5 Test #2: test-EMAN2DIR ....................***Failed 0.38 sec Checking the libEM2.so with readelf, showed that those symbols are not defined in the library: ***@***.*** libEM]$ readelf -a libEM2.so | grep _ZN3MPI8Datatype4FreeEv 116: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _ZN3MPI8Datatype4FreeEv 4457: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _ZN3MPI8Datatype4FreeEv ***@***.*** libEM]$ I am not really an OpenMPI expert, therefore I cannot judge, if this is caused by the dlopen issue. Again, thank you for your help. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#414?email_source=notifications&email_token=ACPKF2QFND5252MJUY7TEYTP643KBA5CNFSM4IBFW4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZWYRRQ#issuecomment-510494918>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACPKF2UKZLUCE6L233ADH3LP643KBANCNFSM4IBFW4TA>.

[shadowwalkersb]

--disable-dlopen shouldn't be needed with OpenMPI 3 and higher, open-mpi/ompi#3705 (comment).