First install with `JULIA_AMDGPU_DISABLE_ARTIFACTS` leads to broken config

[vchuravy]

Using the following Dockerfile:

FROM rocm/rocm-terminal:5.3

RUN sudo apt-get update && \
    sudo apt-get dist-upgrade -y && \
    sudo apt-get install -y wget vim

RUN wget https://julialang-s3.julialang.org/bin/linux/x64/1.9/julia-1.9.0-linux-x86_64.tar.gz && \
    tar xf julia-1.9.0-linux-x86_64.tar.gz && \
    rm julia-1.9.0-linux-x86_64.tar.gz

ENV JULIA_AMDGPU_DISABLE_ARTIFACTS=1
# Broken due to https://github.com/JuliaLang/julia/issues/49912
# ENV JULIA_CPU_TARGET="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)"

RUN julia-1.9.0/bin/julia -e 'import Pkg; Pkg.add("AMDGPU")'

Running it with:

 docker run -it --device=/dev/kfd --device=/dev/dri --group-add video ID

julia> using AMDGPU
┌ Warning: HSA runtime is unavailable, compilation and runtime functionality will be disabled.
│ Reason: unknown
└ @ AMDGPU ~/.julia/packages/AMDGPU/9VGcR/src/AMDGPU.jl:268
┌ Warning: LLD is unavailable, compilation functionality will be disabled.
│ Reason: unknown
└ @ AMDGPU ~/.julia/packages/AMDGPU/9VGcR/src/AMDGPU.jl:281
┌ Warning: Device libraries are unavailable, device intrinsics will be disabled.
│ Reason: unknown
└ @ AMDGPU ~/.julia/packages/AMDGPU/9VGcR/src/AMDGPU.jl:294
┌ Warning: HIP library is unavailable, HIP integration will be disabled.
│ Reason: unknown
└ @ AMDGPU ~/.julia/packages/AMDGPU/9VGcR/src/AMDGPU.jl:309

julia> AMDGPU.versioninfo()
Using ROCm provided by: System
HSA Runtime (MISSING)
ld.lld (MISSING)
ROCm-Device-Libs (MISSING)
HIP Runtime (MISSING)
rocBLAS (MISSING)
rocSOLVER (MISSING)
rocALUTION (MISSING)
rocSPARSE (MISSING)
rocRAND (MISSING)
rocFFT (MISSING)
MIOpen (MISSING)

After removing cache-file:

julia> AMDGPU.versioninfo()
Using ROCm provided by: System
HSA Runtime (ready)
- Path: /opt/rocm-5.3.0/lib/libhsa-runtime64.so
- Version: 1.1.0
ld.lld (ready)
- Path: /opt/rocm/llvm/bin/ld.lld
ROCm-Device-Libs (ready)
- Path: /opt/rocm/amdgcn/bitcode
HIP Runtime (ready)
- Path: /opt/rocm-5.3.0/lib/libamdhip64.so
rocBLAS (MISSING)
rocSOLVER (MISSING)
rocALUTION (MISSING)
rocSPARSE (MISSING)
rocRAND (MISSING)
rocFFT (MISSING)
MIOpen (MISSING)
HSA Agents (2):
- CPU-XX [AMD Ryzen 7 3700X 8-Core Processor]
- GPU-849e7f8e4045a584 [AMD Radeon RX 6800 XT (gfx1030)]

This might be due to:

AMDGPU.jl/deps/bindeps.jl

Lines 191 to 194 in b80dc31

	if !ispath("/dev/kfd")
	@debug "/dev/kfd not available, cannot use ROCm Runtime."
	@goto populate
	end

is this necessary? We should be able to do the discovery on systems without KFD.

[vchuravy]

@jpsamaroo this is the docker issue I mentioned on the call.

[luraess]

Seems suspiciously similar to the issue we experience on LUMI when trying to precompile AMDGPU.jl on login nodes where AMD GPU hardware is missing...

Our current workaround there is the have export JULIA_PKG_PRECOMPILE_AUTO=0 set and manually trigger recompilation on compute nodes.