Move portable RID graph into runtime and clean-up

[ViktorHofer]

See discussion in #86391 for more details

1. Move portable RID graph into runtime
2. Allow updates to both the non-portable and portable RID graphs under source build.
3. Clean-up project and remove hacks

cc @elinor-fung @akoeplinger @jkotas @am11

[jkotas]

Suggested change

	The RID graphs are frozen and shouldn't be updated anymore. Build from source automatically adds the non-portable distro RID encoded via the `OutputRID` property into the RID graph which allows build tools to target that RID.
	The RID graphs should be only updated with new base OSes. The RID graphs shouldn't be updated with new OS flavor- and version-specific RIDs anymore. Build from source automatically adds the non-portable distro RID encoded via the `OutputRID` property into the RID graph which allows build tools to target that RID.

[jkotas]

dotnet/designs#260 (comment)

[ViktorHofer]

Makes sense, thanks. Do we somewhere already capture that we don't need to update the RID graph for building an unknown Unix distribution as part of source build? cc @tmds

[tmds]

The RID graphs shouldn't be updated with new OS flavor- and version-specific RIDs anymore.

More specifically: the build automatically takes care of adding the non-portable rid for the OS during source-build.

[ViktorHofer]

The one difficulty that I don't think many are aware of is that the libraries TFM infrastructure reads from the RID graph (portable by default) to calculate the compatibility mapping when referencing other projects. That's what powers TFMs like net8.0-unix.

As the Microsoft.NETCore.Platforms package builds too late, we can't leverage the source built updated RID graph. Therefore we still require manual additions to the RID graph just for the sake of being able to target RIDs in our libraries.

This problem would go away if libraries would use OS runtime detection instead of at build time. Looking at the haiku PR, there aren't just a handful of these libraries. Collapsing build time platforms is general goodness as it reduces the build graph (makes evaluation, restore, build, pack, ... faster).

[tmds]

As the Microsoft.NETCore.Platforms package builds too late, we can't leverage the source built updated RID graph. Therefore we still require manual additions to the RID graph just for the sake of being able to target RIDs in our libraries.

At least for .NET 8, it doesn't seem necessary for the non-portable rid that gets source-built to be in the graph.

This problem would go away if libraries would use OS runtime detection instead of at build time. Looking at the haiku PR, there aren't just a handful of these libraries. Collapsing build time platforms is general goodness as it reduces the build graph (makes evaluation, restore, build, pack, ... faster).

Yea, we shouldn't create rid specific libs when they are not needed. I imagine most of them exists for rid specific assets, so they are needed.

Haiku is different from adding the non-portable rid like we're doing with source-build.
haiku a direct base of unix. It's more similar to a portable rid than to a non-portable rid.

Maybe some changes are possible to reduce the verbosity/repetition.

[ViktorHofer]

Yea, we shouldn't create rid specific libs when they are not needed. I imagine most of them exists for rid specific assets, so they are needed.

System.Data.Odbc is a prime example of a library that targets way too many platforms. I would imagine that all the Unix derived platforms could be collapsed into a single build by using runtime checks (which would then get optimized away when the linker is used in the consuming app).

[tmds]

System.Data.Odbc is a prime example of a library that targets way too many platforms. I would imagine that all the Unix derived platforms could be collapsed into a single build by using runtime checks (which would then get optimized away when the linker is used in the consuming app).

I took a quick look. It seems the split is mostly for finding the platform specific name of the Odbc32 library. Perhaps we can do something with NativeLibrary instead. @ViktorHofer may be you can create an issue to investigate it further.

[ViktorHofer]

Exactly. ODBC is tracked via #53900 but there are probably also inbox libraries that would benefit from a build platform simplification.

[ViktorHofer]

@dsplaisted @agocke @elinor-fung can you please review the changes? Is that we want?

@tmds do we actually want to update both RID graphs for source build or would the portable one be enough?

[tmds]

@tmds do we actually want to update both RID graphs for source build or would the portable one be enough?

We don't need to update the non-portable graph, as the plan is to remove it, right? (#90000).

[ViktorHofer]

We don't need to update the non-portable graph, as the plan is to remove it, right?

I'm not sure about that anymore as

runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj

Line 15 in cc89f38

<UseRidGraph>true</UseRidGraph>

depends on it. cc @vitek-karas who made that change

[vitek-karas]

It's true that we will need to keep the non-portable RID graph as it was in .NET 8 around, as it needs to be included in the Microsoft.NETCore.App framework. My change was the most expedient way to fix a code integration problem.

I must admit I don't know where SDK gets the RID graph from when it generates a self-contained app. Wherever that is, we probably need to keep it there for now.

[tmds]

as it needs to be included in the Microsoft.NETCore.App framework

This doesn't cause something to get included/excluded. It's a choice between the legacy graph and the new portable graph.

If you need to set this, then something is broken when using the new portable graph.

Do you know what opting into the legacy graph is supposed to fix?

[ViktorHofer]

I must admit I don't know where SDK gets the RID graph from when it generates a self-contained app. Wherever that is, we probably need to keep it there for now.

The SDK gets it from runtime via the Microsoft.NETCore.Platforms non-shipping transport package (the project file that is changed in this PR).

[vitek-karas]

If you need to set this, then something is broken when using the new portable graph.

Do you know what opting into the legacy graph is supposed to fix?

Even .NET 8+ apps can opt into the "old" behavior where the host reads the RID graph from .deps.json (instead of relying on the computed fallback list). If the app is self-contained, then SDK needs to write the RID graph into the .deps.json of the app (and Viktor just replied as to where that RID graph comes from). If the app is framework dependent, then the RID graph needs to come from the core framework - which is Microsoft.NETCore.App. So Microsoft.NETCore.App.deps.json needs to contain the full RID graph (the "old" one) even in .NET 8+. By default host will ignore it, but if the app opts into the backward compat behavior host will read it (just like it does in 7).

[ViktorHofer]

Even .NET 8+ apps can opt into the "old" behavior where the host reads the RID graph from

It sounded like we are considering removing that switch in .NET 9 or future. I assume that the need for keeping the non portable RID graph alive would then go away?

[vitek-karas]

It sounded like we are considering removing that switch in .NET 9 or future. I assume that the need for keeping the non portable RID graph alive would then go away?

That's a good point. I must admit don't know if we already made a plan to deprecate the old RID graph. @elinor-fung would know best.

[tmds]

That's a good point. I must admit don't know if we already made a plan to deprecate the old RID graph. @elinor-fung would know best.

#90000 was created for removing the legacy rid graph.

[vitek-karas]

Looks like I'm out of the loop then. If we want to remove the RID graph from the framework I would expect test failures in the host tests (that was the reason I added the change to the framework build).

[elinor-fung]

It sounded like we are considering removing that switch in .NET 9 or future. I assume that the need for keeping the non portable RID graph alive would then go away?

We don't have a specific release set for removing the switch. I think .NET 9 is a bit too soon though - my thought had been having at least a release of warning that the switch would go away. But yes, once the switch is removed, the non-portable graph could go way.

[tmds]

If the legacy graph is maintained for .NET 9, then we should to update it as part of source-build too, to maintain the legacy behavior.

[ViktorHofer]

If the legacy graph is maintained for .NET 9, then we should to update it as part of source-build too, to maintain the legacy behavior.

Agreed. Perfect, that's already the current state of the PR. Just waiting for some reviews.

[ghost]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

---

See discussion in #86391 for more details

1. Move portable RID graph into runtime
2. Allow updates to both the non-portable and portable RID graphs under source build.
3. Clean-up project and remove hacks

cc @elinor-fung @akoeplinger @jkotas @am11

Author:	ViktorHofer
Assignees:	ViktorHofer
Labels:	area-Infrastructure-libraries
Milestone:	-

[ViktorHofer]

@dsplaisted can you please follow-up on deleting the non portable RID graph from the sdk repo (probably including the task) and consuming it from the Microsoft.NETCore.Platforms transport package?

edit tracked by dotnet/sdk#35750