[RISC-V] Need to configure target RISC-V extension for coreclr build

[fuad1502]

There is a need to enable certain extensions for RISC-V coreclr builds. For example, emitLoadImmediate makes heavy use of bit count intrinsics.

runtime/src/coreclr/jit/emitriscv64.cpp

Line 1312 in 456e1fe

void emitter::emitLoadImmediate(emitAttr size, regNumber reg, ssize_t imm)

We can increase the JIT throughput of this function by enabling Zbb extension. This is only one example where we could improve RISC-V JIT throughput by enabling certain extensions. Other extensions, such as Zba and Zicond can improve throughput at various places.

Currently, the compiler (clang) uses the default configuration for RISC-V target, which "only" enables the rv64gc "profile".

We can use CLR_ADDITIONAL_COMPILER_OPTIONS to pass in -march=rv64gc_zbb to clang:

runtime/eng/native/configurecompiler.cmake

Lines 785 to 787 in 11afd86

	if(CLR_CMAKE_HOST_UNIX)
	add_compile_options(${CLR_ADDITIONAL_COMPILER_OPTIONS})
	endif(CLR_CMAKE_HOST_UNIX)

But I was wondering if we should introduce a better way to enable these extensions, akin to what we have for ARM:

runtime/eng/native/configurecompiler.cmake

Lines 753 to 756 in 11afd86

	if(CLR_CMAKE_HOST_UNIX_ARM)
	if (NOT DEFINED CLR_ARM_FPU_TYPE)
	set(CLR_ARM_FPU_TYPE vfpv3)
	endif(NOT DEFINED CLR_ARM_FPU_TYPE)

There are several alternative proposals that I would like to suggest:

• Introduce RISCV_EXTENSIONS CMake option. User can pass in Zba_Zicond_....
• Introduce RISCV_PROFILE CMake option. User can pass in rv22, rv23, etc.
• Introduce options for each extension that might be useful, e.g. RISCV_ZBB=1, adding it only when a need arises.
• Change the default profile to RVA20/22/23.

cc @dotnet/samsung

[fuad1502]

Any thoughts? @tomeksowi @clamp03 @am11 @filipnavara @jakobbotsch

Would this be something that you think is useful to add?

[tomeksowi]

RISC-V ELF has mappings with optional ISA strings, so one option is to use that and let it be chosen at runtime (never tried it).

Another would be to build JIT/VM/etc shared objects for rv64gc/rva20, 22, 23 and only the corerun/dotnet executables would be rv64gc-only and load the appropriate .so.

It would be interesting to know how other platforms handle enabling ISA extensions for JIT/VM/etc shared objects.

EDIT: I'm not opposed to a build-time option, just thinking ahead that the wider user base will likely be using a pre-packaged .NET rather than customized build from source.

[am11]

Change the default profile to RVA20/22/23.

The current baseline for Ubuntu 24.04 (first Ubuntu release officially supporting riscv64) is RVA20. We can increase it to that, as MSFT supported architectures align with "some" release of distro as a baseline https://github.com/dotnet/core/blob/c95d43ae54af9076a5034f7acd422c8d2204f1a3/release-notes/9.0/supported-os.md#linux.

Ubuntu 25.10 is planning to drop even RAV20 and making RVA22 as their baseline https://ubuntu.com/download/risc-v

We plan to upgrade the ISA level above RVA20 with the 25.10 release. Please, install the 24.04 LTS release if you need long term support for RVA20 hardware.

I think making RVA23 (ratified few months ago) as baseline to run dotnet would be a bit overambitious at this point.

We can increase the JIT throughput of this function by enabling Zbb extension.

Do you have some stats? Like wall clock time dotnet run in a moderately complex app (doing some substantial work at the startup forcing JIT to compile many methods) when runtime is built with base vs. RVA23 profile.

Note that the default profile for AMD64 is also SSE2, and it is also very slow compared to AVX512, e.g. #114597 (comment), the difference between v3 and v4 for float->long etc. is massive: https://godbolt.org/z/nYx43dfE5. Usually the hardware predictions on newer, more capable machine, optimize these type of things and JIT throughput doesn't matter too much in real-world apps.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[fuad1502]

@tomeksowi good point, those are good options for supporting all hardware. If we don't mind the increase in size, I think both options will be good moving forward for RISC-V .NET packages 👍

@am11

The current baseline for Ubuntu 24.04 (first Ubuntu release officially supporting riscv64) is RVA20. We can increase it to that

Sounds good. But RVA20 still does not have Zbb. I think for our particular use case, it would be better if we can enable it through a build configuration option. But if it looks like something that not many would find useful, we can add it internally only 🙏

Do you have some stats? Like wall clock time dotnet run in a moderately complex app (doing some substantial work at the startup forcing JIT to compile many methods) when runtime is built with base vs. RVA23 profile.

I run MicroBenchmark for the following C# code:

[Benchmark]
public void MethodCall() => Fun1();

[MethodImpl(MethodImplOptions.NoInlining)]
static void Fun1()
{
    Fun2();
    Fun2();
    // repeated 50 times
    Fun2();
}

[MethodImpl(MethodImplOptions.NoInlining)]
static void Fun2() 
{
}

When jitted, it will call emitLoadImmediate many times: (Yes, the immediate is created again and again, I was thinking if it's possible to CSE the synthesized pointer, that's for another time though)

G_M30329_IG02:  ;; offset=0x0010
            lui            t6, -220227
            addiw          t6, t6, 1959
            slli           t6, t6, 29
            srli           t6, t6, 17           ;; load imm: hex=0x00007CA3BD7A7000 dec=137042700431360
            ld             a0, -1152(t6)
            jalr           a0           // Program:Fun2()
            lui            t6, -220227
            addiw          t6, t6, 1959
            slli           t6, t6, 29
            srli           t6, t6, 17           ;; load imm: hex=0x00007CA3BD7A7000 dec=137042700431360
            ld             a0, -1152(t6)
            jalr           a0           // Program:Fun2()
            ;; repeated 50 times

Using runtime with rv64gc profile:

OverheadJitting  1: 1 op, 877763.00 ns, 877.7630 us/op
WorkloadJitting  1: 1 op, 1973030.00 ns, 1.9730 ms/op

OverheadJitting  2: 16 op, 1628024.00 ns, 101.7515 us/op
WorkloadJitting  2: 16 op, 1612524.00 ns, 100.7828 us/op

Using runtime with rv64gc_Zba_Zbb profile:

OverheadJitting  1: 1 op, 843012.00 ns, 843.0120 us/op
WorkloadJitting  1: 1 op, 1797527.00 ns, 1.7975 ms/op

OverheadJitting  2: 16 op, 1546023.00 ns, 96.6264 us/op
WorkloadJitting  2: 16 op, 1573523.00 ns, 98.3452 us/op

Not sure if it's a substantial enough improvement for throughput though.

I ran it on:

processor       : 0
hart            : 2
isa             : rv64imafdc_zicntr_zicsr_zifencei_zihpm_zca_zcd_zba_zbb
mmu             : sv39
uarch           : sifive,u74-mc
mvendorid       : 0x489
marchid         : 0x8000000000000007
mimpid          : 0x4210427
hart isa        : rv64imafdc_zicntr_zicsr_zifencei_zihpm_zca_zcd_zba_zbb

[am11]

Sounds good. But RVA20 still does not have Zbb

Sorry I wasn't clear. I meant RVA22 will be the minimal requirement of future version of Ubuntu (25.10 onwards) and we normally make a distro version as the basis of kernel+libc baseline.

If the intention is to add a build option just for folks building runtime, then it's less appealing as you can already do stuff like CLR_CC="clang -march=...." CLR_CXX="clang -march=...." ./build.sh ... without changing the code.

Thanks for the stats. How much of that difference would you say is due to emitter::emitLoadImmediate?

• If that's the major hot path and the goal is to only optimize one function, then we can do something more surgical, e.g. multi-version target attribute (https://godbolt.org/z/W1rb1nGed); not yet fully supported in riscv64 https://github.com/riscv-non-isa/riscv-c-api-doc/blob/6e0246f909625d61471377bb211031a56d0a0dcb/src/c-api.adoc#__attribute__targetattr-string so we can manually do the same thing by placing this function in a separate .cpp file and compile it twice with -march=rv64gc as emitLoadImmediate and rv64gc_zbb as emitLoadImmediate_zbb, and make the caller check CPU feature to decide which method to call at run-time (under #ifdef TARGET_RISCV64).

• Otherwise, if it's going to duplicate the whole coreclr runtime to get slight JIT throughput, I'm not sure if it's worth the hassle.

[jakobbotsch]

Thanks for the stats. How much of that difference would you say is due to emitter::emitLoadImmediate?

I would be very surprised if any noticeable impact is coming from emitLoadImmediate. I would wager a guess that there are thousands upon thousands of places throughout the JIT and runtime that all benefit from compiling with a different instruction set.