Large Page Support in memory allocated by the GC

[mjsabby]

Large Pages (a.k.a Huge Pages on Linux) is a feature where the operating system is able to establish memory regions larger than the native page size (often 4K) to improve performance of the application requesting these large pages.

When a virtual-to-physical address translation occurs, a cache called the Translation lookaside buffer (TLB) is first consulted (often in parallel) to check if a physical translation for the virtual address being accessed is available to avoid doing a page-table walk which can be expensive.

A 4GB virtual address space is made up of 1048576 4K virtual pages. It could also be thought of to be represented as 2048 2MB virtual pages. Most modern os and cpus can operate on 4K or larger page sizes (2MB/4MB/1GB in x86_64).

On Windows VirtualAlloc must be called with MEM_RESERVE | MEM_COMMIT | MEM_LARGE_PAGES for the flAllocationType parameter. It is also required that process acquire the SeLockMemoryPrivilege privilege. Large Pages can also significantly increase the chance of a memory allocation failure because now the OS has to find physically contiguous memory.

What this means for coreclr and the GC is that when the GC does a VirtualReserve at the beginning of the program it must now commit this memory as well.

[mjsabby]

cc @Maoni0 I'll try to do the naive find and replace of reserves in the GC code to reserve+commit, it failed when I tried last but will update here.

[Maoni0]

happy to take a look at what you have whenever you are ready.

[Alois-xx]

@mjsabby: I am sure if large pages would be a good thing for the GC since large page memory is not pageable anymore. It will lock the working set in memory and task manager will report a working set and commit memory which is 512 times too little.
Besides that I have found no perf gain when using large pages for the purpose of faster page faults.
See https://aloiskraus.wordpress.com/2016/10/03/windows-10-memory-compression-and-more/ for a more detailed discussion about large pages and their drawbacks.
When the machine is running for longer times (a few weeks) then the OS memory becomes quite fragmented (see https://stackoverflow.com/questions/46976548/cannot-allocate-e-g-3gb-large-pages) where on my 32 GB machine the largest block which I can allocate was only 3,4 GB.

[jkotas]

@Alois-xx We do know that TLB is a bottleneck for very large workloads in some cases and we believe that this option can help mitigate it. This feature is not for everybody. It is unlikely to going to be enabled by default because of the issues you have mentioned.

[mjsabby]

@Alois-xx: The benefit is that since each page table entry is addressing a much larger (512 times) memory footprint, the total amount of vmem that can be addressed without data TLB misses increases. Also when you have to fill the TLB the number of page table entries (8 bytes each) you need to iterate are 512 times less. So for a 4GB virtual address space, you'll only have to iterate 16KB of physical memory to fill the TLB whereas when using 4K pages you'll have to iterate through 2MB of physical memory. Of course all of this matters on your application memory access pattern.

Not surprisingly using Large Pages is quite advantageous in the guest operating system in a virtualized environment where the page walk is even more expensive and hitting the TLB more critical for performance.

As was already pointed this will be an opt-in setting.

[Alois-xx]

@mjsabby: Thanks for additional context. I was under the impression you wanted to change the allocator for the GC in general. For special workloads this can be ok. Do you have performance numbers how much you can gain from enabling large pages? What are large workloads for the GC in this context where it can make sense to use large pages? I know about TLB misses in theory but unless you look with Intel VTune into highly optimized code you will never spot TLB misses. How did you find this optimization?

[Maoni0]

TLB misses are a very real perf problem and have been studied quite a bit. there are plenty of papers that reported investigations of (and various mechanisms to mitigate) perf impact of TLB misses. a casual search resulted in 1, 2, 3. numbers that people quote are generally a few percent to 20%.

enabling large page support in GC had been on my list for a while now and one of the problems was with the OS write watch we were using on Windows - we used the OS write watch to record modified pages and didn't want to get modified pages that were 2mb large (too coarse). but since we've switched to using our own write watch implementation we no longer have that problem. so now is a good time to experiment with this.

as folks had already pointed out this would only be turned on via a config.

[Alois-xx]

@Maoni0: Thanks for the additional links. That is an interesting topic .