superpmi collection: linux arm64/arm32 fail to create CORE_ROOT

[BruceForstall]

Some, but not all, Linux arm32/arm64 jobs are failing in the "Create Core_Root" step with:

  /usr/bin/ld.bfd: unrecognised emulation mode: aarch64linux
  Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu elf_l1om elf_k1om i386pep i386pe

or:

  Generating native code
  /usr/bin/ld.bfd: unrecognised emulation mode: armelf_linux_eabi
  Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu elf_l1om elf_k1om i386pep i386pe

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[JulieLeeMSFT]

@BruceForstall PTAL.

[BruceForstall]

Internally, @agocke said, "I made a change to AOT the xunit log checker during the build that could be affected by linker configuration. Ensure that your pipeline is passing “-cross” during the build — that should force usage of lld instead of bfd"

Perhaps the fix will be similar to #107229

[BruceForstall]

@agocke @jkoritzinsky Unfortunately, simply adding $(crossArg) to the right line didn't work:

https://dev.azure.com/dnceng/internal/_git/dotnet-runtime/pullrequest/42627?_a=files

Apparently $(crossArg) is not set when this line in pipelines\coreclr\templates\superpmi-collect-job.yml is processed. I'm flummoxed by the insane complexity of the YML files these days: do either of you have a suggestion why the setting in pipelines\common\templates\runtimes\xplat-job.yml or pipelines\common\global-build-job.yml doesn't get there?

[BruceForstall]

(fwiw, I see an extra space compared to before, so I'm at least confident my change is getting testing:

https://dev.azure.com/dnceng/internal/_build/results?buildId=2534438&view=logs&j=bd44f093-79ea-522b-8283-93e7f1fb7120&t=80630a4a-c80e-5644-b7ae-37eda359a6be

versus (before):

https://dev.azure.com/dnceng/internal/_build/results?buildId=2533665&view=logs&j=bd44f093-79ea-522b-8283-93e7f1fb7120&t=80630a4a-c80e-5644-b7ae-37eda359a6be
)

[agocke]

Sorry, I don't know how the superpmi jobs are supposed to work. If the parameter is empty, it's likely not being passed a value from above. I'd just trace all the calls upwards until you find the nearest place with a crossArg definition and pass it all the way down.

[BruceForstall]

Well, I managed to get -cross passed. but now it fails with:

/mnt/vss/_work/1/s/src/tests/build.sh checked arm -cross generatelayoutonly  /p:LibrariesConfiguration=Release /p:UsePublishedCrossgen2=false
========================== Starting Command Output ===========================
/usr/bin/bash --noprofile --norc /mnt/vss/_work/_temp/91de1ffe-1518-46b3-ad97-05d1a65ea3c5.sh
Error: rootfsDir has been passed, but the location is not valid.

https://dev.azure.com/dnceng-public/public/_build/results?buildId=802916&view=logs&j=83516c17-6666-5250-abde-63983ce72a49&t=6135b38d-96c0-5932-8da0-4571a0c51203

Apparently someone isn't setting ROOTFS_DIR. In fact, it looks like no containers are being used. It doesn't look like containers were ever being used in these pipelines before. I've spent hours trying to trace how containers get set up and don't see why they aren't already being used.

[agocke]

How was this pipeline working at all? It sounds like you were building for ARM on a different architecture with no rootfs or cross building support? How could that work?