"make install" fails on MacOS with flang-new

[jsquyres]

On main, on macOS 15.3.1 with homebrew-installed flang-new, I ran into an interesting error during make install:

cd ompi/mpi/fortran/use-mpi-f08
$ make install
Making install in profile
make[2]: warning: -jN forced in submake: disabling jobserver mode.
make[2]: Nothing to be done for `install-exec-am'.
make[2]: Nothing to be done for `install-data-am'.
make[2]: warning: -jN forced in submake: disabling jobserver mode.
make[2]: Nothing to be done for `install-data-am'.
 /opt/homebrew/bin/gmkdir -p '/Users/jsquyres/bogus/lib'
 /bin/sh ../../../../libtool   --mode=install /usr/bin/install -c   libmpi_usempif08.la '/Users/jsquyres/bogus/lib'
libtool: install: /usr/bin/install -c .libs/libmpi_usempif08.0.dylib /Users/jsquyres/bogus/lib/libmpi_usempif08.0.dylib
install: .libs/libmpi_usempif08.0.dylib: No such file or directory
make[2]: *** [install-libLTLIBRARIES] Error 71
make[1]: *** [install-am] Error 2
make: *** [install-recursive] Error 1

I note that make completed successfully.

When looking in the .libs dir, .libs/libmpi_usempif08.0.dylib is, indeed, not there:

$ ls -l .libs/lib*
lrwxr-xr-x  1 jsquyres  staff    24 Mar 12 11:15 .libs/libmpi_usempif08.dylib@ -> libmpi_usempif08.0.dylib
lrwxr-xr-x  1 jsquyres  staff    22 Mar 12 11:15 .libs/libmpi_usempif08.la@ -> ../libmpi_usempif08.la
-rw-r--r--  1 jsquyres  staff  1298 Mar 12 11:15 .libs/libmpi_usempif08.lai

I'm not sure what happened here. Forcing the build again in this directory:

$ rm *.la
$ make V=1
Making all in profile
make[1]: Nothing to be done for `all'.
/bin/sh ../../../../libtool  --tag=FC   --mode=link flang-new -I../../../../ompi/mpi/fortran/use-mpi-f08/mod -I../../../../ompi/include -I../../../../ompi/include -I../../../../ompi/mpi/fortran/use-mpi -I../../../../ompi/mpi/fortran/use-mpi-ignore-tkr -Imod -Ibindings -I../../../.. -I../../../..  -DOMPI_BUILD_MPI_PROFILING=0  -version-info 0:0:0 -Wl,-flat_namespace -Wl,-commons,use_dylibs   -o libmpi_usempif08.la -rpath /Users/jsquyres/bogus/lib abort_f08.lo accumulate_f08.lo add_error_class_f08.lo add_error_code_f08.lo add_error_string_f08.lo aint_add_f08.lo aint_diff_f08.lo allgather_f08.lo allgather_init_f08.lo allgatherv_f08.lo allgatherv_init_f08.lo alloc_mem_f08.lo allreduce_f08.lo allreduce_init_f08.lo alltoall_f08.lo alltoall_init_f08.lo alltoallv_f08.lo alltoallv_init_f08.lo alltoallw_f08.lo alltoallw_init_f08.lo barrier_f08.lo barrier_init_f08.lo bcast_f08.lo bcast_init_f08.lo bsend_f08.lo bsend_init_f08.lo buffer_attach_f08.lo buffer_detach_f08.lo cancel_f08.lo cart_coords_f08.lo cart_create_f08.lo cartdim_get_f08.lo cart_get_f08.lo cart_map_f08.lo cart_rank_f08.lo cart_shift_f08.lo cart_sub_f08.lo close_port_f08.lo comm_accept_f08.lo comm_call_errhandler_f08.lo comm_compare_f08.lo comm_connect_f08.lo comm_create_errhandler_f08.lo comm_create_f08.lo comm_create_from_group_f08.lo comm_create_group_f08.lo comm_create_keyval_f08.lo comm_delete_attr_f08.lo comm_disconnect_f08.lo comm_dup_f08.lo comm_dup_with_info_f08.lo comm_idup_f08.lo comm_idup_with_info_f08.lo comm_free_f08.lo comm_free_keyval_f08.lo comm_get_attr_f08.lo comm_get_errhandler_f08.lo comm_get_info_f08.lo comm_get_name_f08.lo comm_get_parent_f08.lo comm_group_f08.lo comm_join_f08.lo comm_rank_f08.lo comm_remote_group_f08.lo comm_remote_size_f08.lo comm_set_attr_f08.lo comm_set_errhandler_f08.lo comm_set_info_f08.lo comm_set_name_f08.lo comm_size_f08.lo comm_spawn_f08.lo comm_spawn_multiple_f08.lo comm_split_f08.lo comm_split_type_f08.lo comm_test_inter_f08.lo compare_and_swap_f08.lo dist_graph_create_adjacent_f08.lo dist_graph_create_f08.lo dist_graph_neighbors_count_f08.lo dist_graph_neighbors_f08.lo dims_create_f08.lo errhandler_free_f08.lo error_class_f08.lo error_string_f08.lo exscan_f08.lo exscan_init_f08.lo f_sync_reg_f08.lo fetch_and_op_f08.lo file_call_errhandler_f08.lo file_close_f08.lo file_create_errhandler_f08.lo file_delete_f08.lo file_get_amode_f08.lo file_get_atomicity_f08.lo file_get_byte_offset_f08.lo file_get_errhandler_f08.lo file_get_group_f08.lo file_get_info_f08.lo file_get_position_f08.lo file_get_position_shared_f08.lo file_get_size_f08.lo file_get_type_extent_f08.lo file_get_view_f08.lo file_iread_at_f08.lo file_iread_f08.lo file_iread_at_all_f08.lo file_iread_all_f08.lo file_iread_shared_f08.lo file_iwrite_at_f08.lo file_iwrite_f08.lo file_iwrite_at_all_f08.lo file_iwrite_all_f08.lo file_iwrite_shared_f08.lo file_open_f08.lo file_preallocate_f08.lo file_read_all_begin_f08.lo file_read_all_end_f08.lo file_read_all_f08.lo file_read_at_all_begin_f08.lo file_read_at_all_end_f08.lo file_read_at_all_f08.lo file_read_at_f08.lo file_read_f08.lo file_read_ordered_begin_f08.lo file_read_ordered_end_f08.lo file_read_ordered_f08.lo file_read_shared_f08.lo file_seek_f08.lo file_seek_shared_f08.lo file_set_atomicity_f08.lo file_set_errhandler_f08.lo file_set_info_f08.lo file_set_size_f08.lo file_set_view_f08.lo file_sync_f08.lo file_write_all_begin_f08.lo file_write_all_end_f08.lo file_write_all_f08.lo file_write_at_all_begin_f08.lo file_write_at_all_end_f08.lo file_write_at_all_f08.lo file_write_at_f08.lo file_write_f08.lo file_write_ordered_begin_f08.lo file_write_ordered_end_f08.lo file_write_ordered_f08.lo file_write_shared_f08.lo finalized_f08.lo finalize_f08.lo free_mem_f08.lo gather_f08.lo gather_init_f08.lo gatherv_f08.lo gatherv_init_f08.lo get_accumulate_f08.lo get_address_f08.lo get_count_f08.lo get_elements_f08.lo get_elements_x_f08.lo get_f08.lo get_library_version_f08.lo get_processor_name_f08.lo get_version_f08.lo graph_create_f08.lo graphdims_get_f08.lo graph_get_f08.lo graph_map_f08.lo graph_neighbors_count_f08.lo graph_neighbors_f08.lo grequest_complete_f08.lo grequest_start_f08.lo group_compare_f08.lo group_difference_f08.lo group_excl_f08.lo group_free_f08.lo group_from_session_pset_f08.lo group_incl_f08.lo group_intersection_f08.lo group_range_excl_f08.lo group_range_incl_f08.lo group_rank_f08.lo group_size_f08.lo group_translate_ranks_f08.lo group_union_f08.lo iallgather_f08.lo iallgatherv_f08.lo iallreduce_f08.lo ialltoall_f08.lo ialltoallv_f08.lo ialltoallw_f08.lo ibarrier_f08.lo ibcast_f08.lo ibsend_f08.lo iexscan_f08.lo igather_f08.lo igatherv_f08.lo improbe_f08.lo imrecv_f08.lo ineighbor_allgather_f08.lo ineighbor_allgatherv_f08.lo ineighbor_alltoall_f08.lo ineighbor_alltoallv_f08.lo ineighbor_alltoallw_f08.lo info_create_f08.lo info_create_env_f08.lo info_delete_f08.lo info_dup_f08.lo info_free_f08.lo info_get_f08.lo info_get_nkeys_f08.lo info_get_nthkey_f08.lo info_get_valuelen_f08.lo info_get_string_f08.lo info_set_f08.lo init_f08.lo initialized_f08.lo init_thread_f08.lo intercomm_create_f08.lo intercomm_create_from_groups_f08.lo intercomm_merge_f08.lo iprobe_f08.lo irecv_f08.lo ireduce_f08.lo ireduce_scatter_f08.lo ireduce_scatter_block_f08.lo irsend_f08.lo iscan_f08.lo iscatter_f08.lo iscatterv_f08.lo isend_f08.lo isendrecv_f08.lo isendrecv_replace_f08.lo issend_f08.lo is_thread_main_f08.lo lookup_name_f08.lo mprobe_f08.lo mrecv_f08.lo neighbor_allgather_f08.lo neighbor_allgather_init_f08.lo neighbor_allgatherv_f08.lo neighbor_allgatherv_init_f08.lo neighbor_alltoall_f08.lo neighbor_alltoall_init_f08.lo neighbor_alltoallv_f08.lo neighbor_alltoallv_init_f08.lo neighbor_alltoallw_f08.lo neighbor_alltoallw_init_f08.lo op_commutative_f08.lo op_create_f08.lo open_port_f08.lo op_free_f08.lo pack_external_f08.lo pack_external_size_f08.lo pack_f08.lo pack_size_f08.lo parrived_f08.lo pcontrol_f08.lo pready_f08.lo pready_range_f08.lo pready_list_f08.lo precv_init_f08.lo probe_f08.lo psend_init_f08.lo publish_name_f08.lo put_f08.lo query_thread_f08.lo raccumulate_f08.lo recv_f08.lo recv_init_f08.lo reduce_f08.lo reduce_init_f08.lo reduce_local_f08.lo reduce_scatter_f08.lo reduce_scatter_init_f08.lo reduce_scatter_block_f08.lo reduce_scatter_block_init_f08.lo register_datarep_f08.lo request_free_f08.lo request_get_status_f08.lo rget_f08.lo rget_accumulate_f08.lo rput_f08.lo rsend_f08.lo rsend_init_f08.lo scan_f08.lo scan_init_f08.lo scatter_f08.lo scatter_init_f08.lo scatterv_f08.lo scatterv_init_f08.lo send_f08.lo send_init_f08.lo sendrecv_f08.lo sendrecv_replace_f08.lo session_call_errhandler_f08.lo session_create_errhandler_f08.lo session_get_errhandler_f08.lo session_get_info_f08.lo session_get_nth_pset_f08.lo session_get_num_psets_f08.lo session_get_pset_info_f08.lo session_init_f08.lo session_finalize_f08.lo session_set_errhandler_f08.lo ssend_f08.lo ssend_init_f08.lo startall_f08.lo start_f08.lo status_f082f_f08.lo status_f2f08_f08.lo status_set_cancelled_f08.lo status_set_elements_f08.lo status_set_elements_x_f08.lo testall_f08.lo testany_f08.lo test_cancelled_f08.lo test_f08.lo testsome_f08.lo topo_test_f08.lo type_commit_f08.lo type_contiguous_f08.lo type_create_darray_f08.lo type_create_f90_complex_f08.lo type_create_f90_integer_f08.lo type_create_f90_real_f08.lo type_create_hindexed_f08.lo type_create_hvector_f08.lo type_create_indexed_block_f08.lo type_create_hindexed_block_f08.lo type_create_keyval_f08.lo type_create_resized_f08.lo type_create_struct_f08.lo type_create_subarray_f08.lo type_delete_attr_f08.lo type_dup_f08.lo type_free_f08.lo type_free_keyval_f08.lo type_get_attr_f08.lo type_get_contents_f08.lo type_get_envelope_f08.lo type_get_extent_f08.lo type_get_extent_x_f08.lo type_get_name_f08.lo type_get_true_extent_f08.lo type_get_true_extent_x_f08.lo type_indexed_f08.lo type_match_size_f08.lo type_set_attr_f08.lo type_set_name_f08.lo type_size_f08.lo type_size_x_f08.lo type_vector_f08.lo unpack_external_f08.lo unpack_f08.lo unpublish_name_f08.lo waitall_f08.lo waitany_f08.lo wait_f08.lo waitsome_f08.lo win_allocate_f08.lo win_allocate_shared_f08.lo win_attach_f08.lo win_call_errhandler_f08.lo win_complete_f08.lo win_create_dynamic_f08.lo win_create_errhandler_f08.lo win_create_f08.lo win_create_keyval_f08.lo win_delete_attr_f08.lo win_detach_f08.lo win_fence_f08.lo win_flush_f08.lo win_flush_all_f08.lo win_flush_local_f08.lo win_flush_local_all_f08.lo win_free_f08.lo win_free_keyval_f08.lo win_get_attr_f08.lo win_get_errhandler_f08.lo win_get_group_f08.lo win_get_info_f08.lo win_get_name_f08.lo win_lock_f08.lo win_lock_all_f08.lo win_post_f08.lo win_set_attr_f08.lo win_set_errhandler_f08.lo win_set_info_f08.lo win_set_name_f08.lo win_shared_query_f08.lo win_start_f08.lo win_sync_f08.lo win_test_f08.lo win_unlock_f08.lo win_unlock_all_f08.lo win_wait_f08.lo mpi-f08.lo sizeof_f08.lo profile/psizeof_f08.lo profile/libmpi_usempif08_pmpi.la ../../../../ompi/mpiext/ftmpi/use-mpi-f08/libmpiext_ftmpi_usempif08.la ../../../../ompi/mpi/fortran/mpif-h/libmpi_mpifh.la ../../../../ompi/libmpi.la mod/libusempif08_internal_modules.la base/libusempif08_ccode.la /Users/jsquyres/git/o2/3rd-party/libevent-2.1.12-stable-ompi/libevent_core.la /Users/jsquyres/git/o2/3rd-party/libevent-2.1.12-stable-ompi/libevent_pthreads.la /Users/jsquyres/git/o2/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /Users/jsquyres/git/o2/3rd-party/openpmix/src/libpmix.la
libtool: link: rm -fr  .libs/libmpi_usempif08.dylib .libs/libmpi_usempif08.la .libs/libmpi_usempif08.lai
libtool: link: (cd ".libs" && rm -f "libmpi_usempif08.dylib" && ln -s "libmpi_usempif08.0.dylib" "libmpi_usempif08.dylib")
libtool: link: ( cd ".libs" && rm -f "libmpi_usempif08.la" && ln -s "../libmpi_usempif08.la" "libmpi_usempif08.la" )

The needed file is still not in .libs:

$ ls -l .libs/lib*
lrwxr-xr-x  1 jsquyres  staff    24 Mar 12 11:15 .libs/libmpi_usempif08.dylib@ -> libmpi_usempif08.0.dylib
lrwxr-xr-x  1 jsquyres  staff    22 Mar 12 11:15 .libs/libmpi_usempif08.la@ -> ../libmpi_usempif08.la
-rw-r--r--  1 jsquyres  staff  1298 Mar 12 11:15 .libs/libmpi_usempif08.lai

Here's my flang-new version:

$ flang-new --version
Homebrew flang-new version 19.1.7
Target: arm64-apple-darwin24.3.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/flang/19.1.7_1/libexec
Configuration file: /opt/homebrew/Cellar/flang/19.1.7_1/libexec/flang.cfg
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin24.cfg

I don't have time to investigate this at the moment.

Has anyone else seen this?

[ggouaillardet]

Indeed, libtool does not even invoke the compiler and hence does not create the library.

libtool: link: rm -fr  .libs/libmpi_usempif08.dylib .libs/libmpi_usempif08.la .libs/libmpi_usempif08.lai
libtool: link: (cd ".libs" && rm -f "libmpi_usempif08.dylib" && ln -s "libmpi_usempif08.40.dylib" "libmpi_usempif08.dylib")
libtool: link: creating libmpi_usempif08.la
libtool: link: ( cd ".libs" && rm -f "libmpi_usempif08.la" && ln -s "../libmpi_usempif08.la" "libmpi_usempif08.la" )

I re-ran autogen.pl with the brew provided autotools, but had no luck.

I will try to have a look.

[ggouaillardet]

There are a few things going on here...

on Darwin, libtool can only build Fortran libraries with ifort, nagfor and gfortran:

  case $cc_basename in
     ifort*|nagfor*) _lt_dar_can_shared=yes ;;
     *) _lt_dar_can_shared=$GCC ;;
  esac

then autotools try to generate a shared library even if it somehow knows it cannot, it silently fails and we ultimately end up with the reported error message.

I was somehow able to build Open MPI after manually hacking the generated libtool file

[jsquyres]

@ggouaillardet Yoinks. Ok. Do we need another libtool patch? (similar to the config/ltmain*.diff patches that autogen.pl applies)

[jsquyres]

This was fixed on both main and v5.0.x.