Fix Metadata aggregator: removing cumulative sum for GUIDs heap offset

[fabrimaz]

Removing cumulative sum for GUID heap offsets
Fixes #113910

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

[fabrimaz]

Run unit-tests on this library:

=== TEST EXECUTION SUMMARY ===
System.Reflection.Metadata.Tests Total: 852, Errors: 0, Failed: 0, Skipped: 1, Time: 5,376s

[tmat]

The comment on line https://github.com/dotnet/runtime/pull/115268/files#diff-463bcba9d713f9c49493793a38b92567be4f344112abe18ebd08a34d3a5a1659R13 should be updated

[tmat]

It'd be good to have some tests that demonstrate correctness. There are some existing tests for MetadataAggregator, so you can update them. Make sure the test fails without your change and passes with the change.

[fabrimaz]

It'd be good to have some tests that demonstrate correctness. There are some existing tests for MetadataAggregator, so you can update them. Make sure the test fails without your change and passes with the change.

Hello! The linked comment was updated :)

The method "CalculateHeapSizes(..)" I edited is private and its result is assigned to a private field directly in the constructor of the class.
Even if I try to unit-test that constructor, I wouldn't be able to easily access the private field from unit-test class to assert its values.

What would you suggest? Should I split the mentioned method in a separate class to enhance testability?

Or else making the method 'internal'?

[fabrimaz]

Ping @tmat

[fabrimaz]

ping @tmat

[tmat]

See https://github.com/dotnet/runtime/blob/main/src/libraries/System.Reflection.Metadata/tests/Metadata/Ecma335/MetadataAggregatorTests.cs

[ericstj]

@fabrimaz as @tmat mentions the action here is to add a test to https://github.com/dotnet/runtime/blob/main/src/libraries/System.Reflection.Metadata/tests/Metadata/Ecma335/MetadataAggregatorTests.cs. It looks like there are already tests that call TestGenerationHandle which should be able to observe the broken behavior called out in #113910.

[fabrimaz]

@ericstj I might be completely wrong but feels like all the existing unit-tests are building _heapSize in a simplified way through a fake internal ctor, hence not replicating our 'issue'.

To unit-test our behaviour, I need to use at least a baseReader and a list of MetadataReader deltas.
Going in that direction and keeping you posted
@tmat

[fabrimaz]

By chance, do you have any clue on how to create a minimal MetadataReader with an allocated GUID? Couldn't find a good option among existing unit-tests for MetadataReader

[fabrimaz]

@dotnet-policy-service agree

[fabrimaz]

@ericstj @tmat
glad to tell that I managed to build meaningful unit-tests for the specific case.
Needed to create MetadataReader manually to fully test the behaviour.

Please, give it a review and let me know your opinion about :)

[tmat]

You're returning reader that accesses memory at pointer p but the target array can move once fixed block is exited. You need to keep the array pinned while using the reader.

[fabrimaz]

I refactored according to your suggestions

[tmat]

We should be able to read 3 guids since there is one in baseline, and one in each delta, no?

[fabrimaz]

@tmat in a real case yes, not in this test though.
Our new implementation removes cumulative sum and relies on each delta-reader GUID heap size, relying on deltareaders the copying of previous GUIDs and adding of existing new ones.

With new implementation, each generation here allocates a single GUID, hence only that one at index 1 could be read.
Tried to allocate more GUIDs on manually created delta-readers but couldn't find a working example.

[fabrimaz]

With cumulative it was
Gen0: #1
Gen1: #1 #2
Gen2: #1 #2 #3

With new implementation
Gen0: #1
Gen1: #1
Gen2: #1

It's ok because all existing GUIDs are actually copied in next generation of metadata reader normally.

[fabrimaz]

Ping @tmat @ericstj