[VMR] linux/mac-arm64 cross build fails w/ CoreCLR runtime

[steveisok]

Describe the Bug

When triggering a building targeting arm64 from a linux or mac x64 host, the VMR build will fail at runtime with the following error:

      [ 29%] Building CXX object nativeaot/Runtime/Full/CMakeFiles/Runtime.ServerGC.dir/__/EHHelpers.cpp.o
      [ 29%] Building CXX object pal/src/CMakeFiles/coreclrpal.dir/thread/threadsusp.cpp.o
      /Users/steve/dev/dotnet-vmr/src/runtime/artifacts/source-build/self/src/src/coreclr/pal/src/synchmgr/synchmanager.cpp:4633:6: error: "Don't know how to get hi-res current time on this platform"
          #error "Don't know how to get hi-res current time on this platform"
           ^
      [ 30%] Building CXX object nativeaot/Runtime/Full/CMakeFiles/Runtime.WorkstationGC.dir/__/MethodTable.cpp.o
      1 error generated.
      make[2]: *** [pal/src/CMakeFiles/coreclrpal.dir/synchmgr/synchmanager.cpp.o] Error 1
      make[2]: *** Waiting for unfinished jobs....

Steps to Reproduce

• Apply the patch @directhex made in [VMR] Add cross build support #3671 (comment)

• Build the VMR

For Mac:

./prep.sh 
./build.sh --online -- /p:OverrideTargetRid=osx-arm64

For Linux:

docker run -e CROSSCOMPILE=1 -e ROOTFS_DIR=/crossrootfs/arm64 -w pwd--platform linux/amd64 -vpwd:pwd -it mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm64 
./build.sh --online -- /p:OverrideTargetRid=linux-arm64

Other Information

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[steveisok]

/cc @jkoritzinsky

[directhex]

This is the error I get with Linux:

      Executing "/home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/src/coreclr/build-runtime.sh" -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1" -arm64 -release -ci -portablebuild=false -keepnativesymbols -os linux -hostarch x64 -hostos linux -outputrid linux-arm64 -component crosscomponents
      Commencing CoreCLR Repo build
      __OutputRid: linux-arm64
      Setting up directories for build
      Checking prerequisites...
      Commencing build of "  crosscomponents " target in "CoreCLR component" for linux.arm64.Release in /home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/artifacts/obj/coreclr/linux.arm64.Release/x64
      Invoking "/home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/eng/native/gen-buildsys.sh" "/home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/src/coreclr" "/home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/artifacts/obj/coreclr/linux.arm64.Release/x64" x64 linux clang Release ""  -DCLR_CMAKE_TARGET_ARCH=arm64 -DCLR_CMAKE_PGO_INSTRUMENT=0 -DCLR_CMAKE_OPTDATA_PATH= -DCLR_CMAKE_PGO_OPTIMIZE=0 -DCLI_CMAKE_FALLBACK_OS="linux" -DFEATURE_DISTRO_AGNOSTIC_SSL=1 -DCLR_CROSS_COMPONENTS_BUILD=1  -DCLR_CMAKE_KEEP_NATIVE_SYMBOLS=true
      /home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/artifacts/obj/coreclr/linux.arm64.Release/x64 /home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/src/coreclr
      Not searching for unused variables given on the command line.
      loading initial cache file /home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/eng/native/tryrun.cmake
      -- The C compiler identification is Clang 16.0.0
      -- The CXX compiler identification is Clang 16.0.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - failed
      -- Check for working C compiler: /usr/local/bin/clang-16
      -- Check for working C compiler: /usr/local/bin/clang-16 - broken
      CMake Error at /usr/share/cmake-3.21/Modules/CMakeTestCCompiler.cmake:69 (message):
        The C compiler
      
          "/usr/local/bin/clang-16"
      
        is not able to compile a simple test program.
      
        It fails with the following output:
      
          Change Dir: /home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/artifacts/obj/coreclr/linux.arm64.Release/x64/CMakeFiles/CMakeTmp
          
          Run Build Command(s):/usr/bin/make -f Makefile cmTC_f4a38/fast && /usr/bin/make  -f CMakeFiles/cmTC_f4a38.dir/build.make CMakeFiles/cmTC_f4a38.dir/build
          make[1]: Entering directory '/home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/artifacts/obj/coreclr/linux.arm64.Release/x64/CMakeFiles/CMakeTmp'
          Building C object CMakeFiles/cmTC_f4a38.dir/testCCompiler.c.o
          /usr/local/bin/clang-16 --target=x86_64-linux-gnu --gcc-toolchain=/crossrootfs/arm64/usr --sysroot=/crossrootfs/arm64    -MD -MT CMakeFiles/cmTC_f4a38.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_f4a38.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_f4a38.dir/testCCompiler.c.o -c /home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/artifacts/obj/coreclr/linux.arm64.Release/x64/CMakeFiles/CMakeTmp/testCCompiler.c
          Linking C executable cmTC_f4a38
          /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_f4a38.dir/link.txt --verbose=1
          /usr/local/bin/clang-16 --target=x86_64-linux-gnu --gcc-toolchain=/crossrootfs/arm64/usr --sysroot=/crossrootfs/arm64 -Wl,--rpath-link=/crossrootfs/arm64/lib/x86_64-linux-gnu -Wl,--rpath-link=/crossrootfs/arm64/usr/lib/x86_64-linux-gnu -Wl,--rpath-link=/crossrootfs/arm64/lib/x86_64-linux-gnu -Wl,--rpath-link=/crossrootfs/arm64/usr/lib/x86_64-linux-gnu -fuse-ld=lld  CMakeFiles/cmTC_f4a38.dir/testCCompiler.c.o -o cmTC_f4a38 
          ld.lld: error: cannot open Scrt1.o: No such file or directory
          ld.lld: error: cannot open crti.o: No such file or directory
          ld.lld: error: cannot open crtbeginS.o: No such file or directory
          ld.lld: error: unable to find library -lgcc
          ld.lld: error: unable to find library -lgcc_s
          ld.lld: error: unable to find library -lc
          ld.lld: error: unable to find library -lgcc
          ld.lld: error: unable to find library -lgcc_s
          ld.lld: error: cannot open crtendS.o: No such file or directory
          ld.lld: error: cannot open crtn.o: No such file or directory
          clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
          make[1]: *** [CMakeFiles/cmTC_f4a38.dir/build.make:100: cmTC_f4a38] Error 1
          make[1]: Leaving directory '/home/directhex/Projects/dotnet/src/runtime/artifacts/source-build/self/src/artifacts/obj/coreclr/linux.arm64.Release/x64/CMakeFiles/CMakeTmp'
          make: *** [Makefile:127: cmTC_f4a38/fast] Error 2
          
          
      
        
      
        CMake will not be able to correctly generate this project.
      Call Stack (most recent call first):
        CMakeLists.txt:9 (project)
      
      
      -- Configuring incomplete, errors occurred!

I don't know enough about CoreCLR to know what crosscomponents is, but if it's needed, it indicates a need for bi-arch Docker images to satisfy the attempts to compile x64 code for the x64 host on the arm64 build (a valid need in theory, but every instance needs auditing)

[steveisok]

@jtschuster @agocke looping you guys in too

[jtschuster]

I have a build finished and the arm64 sdk runs on my machine with qemu.
I've made a PR on my fork with the diff for reference: jtschuster/dotnet#1

I have dotnet/dotnet cloned at /src/dotnet in wsl, then ran

> docker run -e ROOTFS_DIR=/crossrootfs/arm64 -w /src/dotnet --platform linux/amd64 -v \\wsl.localhost\Ubuntu\src\dotnet:/src/dotnet -it mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm64
$ ./prep.sh && ./build.sh --online -- /p:OverrideTargetRid=linux-arm64 /p:OverrideTargetArch=arm64