Address APX requirements for GC and VM stubs

[anthonycanino]

There are a number of asm stubs in some of the GC, PInvoke, and NativeAOT code.

I'm working to establish if APX extended GPRs (EGPRS) need to be added to some of the state saving --- for what I currently see, most of the registers that are manipulated in some of this code, src/coreclr/nativeaot/Runtime/amd64/PInvoke.asm and src/coreclr/nativeaot/Runtime/amd64/GcProbe.asm only need to address non-volatile registers --- all of the EGPRs are volatile.

My question is, is there any other asm points in the code, perhaps on gc transitions etc., where register state may need to be explictly saved for Windows? I am aware most of the state saving is done with xsave, but looking to make sure all dependencies are handled.

[anthonycanino]

Also, are there any similar requirements with gcinfo? I see that some of the debugging dump code dumps all registers.

[anthonycanino]

@dotnet/avx512-contrib

@jkotas @janvorli @MichalStrehovsky is this something you can help me gain some clarity on?

[BruceForstall]

Also, are there any similar requirements with gcinfo? I see that some of the debugging dump code dumps all registers.

It looks like gc info encoding/decoding is resilient to variable number of registers (other platforms that use the common gc info encoder have different numbers of registers). You will probably find cases like

runtime/src/coreclr/gcinfo/gcinfoencoder.cpp

Line 967 in 09c5809

_ASSERTE(regNum >= 0 && regNum <= 16);

where there is an assert that the register number is <= 16. It is good that the new eGPRs are defined with numbers in the JIT consecutive above the previous GPRs. There must be code where the GC info is decoded to get a register number, and takes that register number and extracts the corresponding register value from a saved VM context record, to use that value as a GC root during GC processing. I don't know where that lookup is.

[anthonycanino]

Thanks helpful thanks Bruce. Looks to have some encoding from the JIT here:

https://github.com/dotnet/runtime/blob/main/src/coreclr/jit/gcencode.cpp#L4639-L4661

I will have to dig more to understand the decode side during GC.

[BruceForstall]

Thanks helpful thanks Bruce. Looks to have some encoding from the JIT here:

Yes, and note that the JIT register numbers are assumed to be equivalent to the register numbers used by the VM.

[jkotas]

There must be code where the GC info is decoded to get a register number, and takes that register number and extracts the corresponding register value from a saved VM context record

This code is here:

runtime/src/coreclr/vm/gcinfodecoder.cpp

Lines 1682 to 1715 in 8d06783

	OBJECTREF* GcInfoDecoder::GetRegisterSlot(
	int regNum,
	PREGDISPLAY pRD
	)
	{
	_ASSERTE(regNum >= 0 && regNum <= 30);
	_ASSERTE(regNum != 18); // TEB
	#ifdef FEATURE_NATIVEAOT
	PTR_uintptr_t* ppReg = &pRD->pX0;
	return (OBJECTREF*)*(ppReg + regNum);
	#else
	DWORD64 **ppReg;
	if(regNum <= 17)
	{
	ppReg = &pRD->volatileCurrContextPointers.X0;
	return (OBJECTREF*)*(ppReg + regNum);
	}
	else if(regNum == 29)
	{
	return (OBJECTREF*) pRD->pCurrentContextPointers->Fp;
	}
	else if(regNum == 30)
	{
	return (OBJECTREF*) pRD->pCurrentContextPointers->Lr;
	}
	ppReg = &pRD->pCurrentContextPointers->X19;
	return (OBJECTREF*)*(ppReg + regNum-19);
	#endif
	}

You will also need to expand the context tracking structures. You can use Arm64 handling or X0-X17 registers as a blue-print:

runtime/src/coreclr/inc/regdisp.h

Line 173 in 8d06783

PDWORD64 X0;

. NativeAOT variant: https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/Runtime/regdisplay.h#L132-L142 .

An alternative to doing all this work would be avoid storing GC tracked references in these registers. SIMD registers are on a plan like that. I assume that you do not want to do this - just mentioning this option for completeness.

[BruceForstall]

fyi, the x64 GetRegisterSlot is here:

runtime/src/coreclr/vm/gcinfodecoder.cpp

Lines 1424 to 1443 in 8d06783

	OBJECTREF* GcInfoDecoder::GetRegisterSlot(
	int regNum,
	PREGDISPLAY pRD
	)
	{
	_ASSERTE(regNum >= 0 && regNum <= 16);
	_ASSERTE(regNum != 4); // rsp
	#ifdef FEATURE_NATIVEAOT
	PTR_uintptr_t* ppRax = &pRD->pRax;
	if (regNum > 4) regNum--; // rsp is skipped in NativeAOT RegDisplay
	#else
	// The fields of KNONVOLATILE_CONTEXT_POINTERS are in the same order as
	// the processor encoding numbers.
	ULONGLONG **ppRax = &pRD->pCurrentContextPointers->Rax;
	#endif
	return (OBJECTREF*)*(ppRax + regNum);
	}

An alternative to doing all this work would be avoid storing GC tracked references in these registers.

That's possibly a reasonable short-term solution if necessary, but it's likely we want to treat the eGPR as equivalent to the current GPR in all ways, long-term.