Support installing workloads in user folder

[dsplaisted]

Currently, workloads are installed in the dotnet root folder. We should also support installing them into a different, user-local folder. Benefits would include:

• Support source-built .NET.
  • The fallback workaround for source-built .NET would involve manually extracting various packages to specific folders, and setting some environment variables to tell the .NET SDK to load workloads from those folders.
• Support installing workloads in Docker containers, where there are not permissions to write to the dotnet root folder
• Better support for .NET SDKs installed via a package manager such as rpm or apt. Currently we expect that you will be able to install workloads in this case via sudo, but it may be an anti-pattern to elevate permissions to write to a folder that is owned by a package manager.

However, we do need to make sure to consider the security implications of having a root-installed .NET SDK load workloads which are installed with non-root permissions.

It's also possible that we want to represent workloads themselves as packages managed by rpm or apt. In this case we might not need to also support installing packages to user folders. (However, for this to work, we would need to be able to source-build all workloads available on Linux in order to make them available to source-built .NET as source-built packages).

Once we have settled on this as the right approach and reviewed the security implications, we should schedule this work as soon as we are able after the currently committed .NET 6 workloads work. The current estimate is 6.0.200 though we should start as soon as we can.

[dsplaisted]

FYI @tmds @omajid, I've filed this issue based on our discussion last week.

[tmds]

I think consuming the workloads from a user folder is as secure as consuming them from an administrator installed .NET root folder.

In both cases they are as trustworthy as the user.

[tmds]

I had a look at the workload implementation. It seems 3 folders under the DOTNET_ROOT are written to: metadata, sdk-manifests, and packs.

For this feature, there needs to be a user folder location for these, e.g. ~/.dotnet/{metadata,sdk-manifests,packs}.

The sdk already supports probing multiple locations for sdk-manifests and packs. This probing could include the user-specific folder by default.

The metadata folder tracks information about the installation. So for user-folder installation, the user-specific metadata folder can be used.

Then for installation, we need to target the user-specific folders instead of the DOTNET_ROOT folders.

@dsplaisted does that sound right?

[dsplaisted]

Yes, that sounds right.

The biggest challenge I see is if we want to support installing and updating workloads both in the DOTNET_ROOT and in user folders for the same installation. That may not be feasible.

[tmds]

For source-build .NET we don't want anyone to install to the DOTNET_ROOT except actual packages.
There is no need to support installing/updating both to both locations.
This also means the most appropriate default locations for the dotnet workload command is the user folder.

How does this compare to Windows?
With .NET 6, what workloads would a user install using the workload install command?
Are these File-based or MSI-based?
When a user downloads and extracts .NET from a tarball, will workloads be added File-based or MSI-based?

[dsplaisted]

Any supported workload might be installed with the dotnet-install command. On Windows for .NET 6, this includes Blazor AOT as well as ios/android/.NET MAUI workloads. Whether the workloads will be installed as file-based or MSI-based depends on how the .NET SDK was installed. If it was installed with Visual Studio or the standalone installer, then workloads will be MSI based. If it was from a tarball or zip, then it will be file-based.

For .NET 6 I think it will be fine to disallow local user worklad installs together with DOTNET_ROOT workload installs. However, once we take existing ASP.NET functionality and put it in a workload, people may want docker images to continue to include the ASP.NET workload, but they might want to allow installing local workloads on top of that.

[tmds]

Whether the workloads will be installed as file-based or MSI-based depends on how the .NET SDK was installed. If it was installed with Visual Studio or the standalone installer, then workloads will be MSI based. If it was from a tarball or zip, then it will be file-based.

How does the SDK differentiate between these type of installs? Is the data in sdk-manifests different? Or is it handled elsewhere?

Could we, at the same level, introduce a setting that tells the SDK it should install workloads to the user-folder? Then we can enable this setting for source-build .NET.

once we take existing ASP.NET functionality and put it in a workload

Is this for .NET 7? I was not aware. Has this been mentioned?
Probably we'll want to be able to source-build this and deliver it through a package.
Otherwise we're unable to provide it in an image (because we can't ship what we haven't built ourselves).

[dsplaisted]

How does the SDK differentiate between these type of installs?

There will be an msi file in the <DOTNET ROOT>/metadata/workloads/<SdkFeatureBand>/installertype/ folder for MSI-based installs. This is in the "Choosing the installer abstraction" section of the design PR.

So yes, we could decide to support a userlocal or similar file in that folder, which source-built .NET could include.

once we take existing ASP.NET functionality and put it in a workload

Is this for .NET 7? I was not aware. Has this been mentioned?
Probably we'll want to be able to source-build this and deliver it through a package.
Otherwise we're unable to provide it in an image (because we can't ship what we haven't built ourselves).

It's mentioned here, though when that was written the plan was for iOS and Android workloads to ship with .NET 5.0 and for further workloads (ASP.NET, Windows Forms / WPF).

Yes, anything that is currently part of the source-built .NET SDK and gets factored out into a workload will need to be source-buildable as a workload.

[tmds]

There will be an msi file in the /metadata/workloads//installertype/ folder for MSI-based installs. This is in the "Choosing the installer abstraction" section of the design PR.

I see. So this is for the entire sdk. I thought it might be that some workloads would be file-based and others MSI-based.

So yes, we could decide to support a userlocal or similar file in that folder, which source-built .NET could include.

The full path would be <DOTNET ROOT>/metadata/workloads/<SdkFeatureBand>/installertype/userlocal?

So when this file exists, we make the SDK install to a user folder instead of the DOTNET_ROOT.

If that sounds good to you, I can look at implementing it.

[marcpopMSFT]

To clarify here, the presence of the userlocal file would cause source build to install AND read workloads from a hardcoded user location, correct? This location would not be controllable by the end user and it would be a location that didn't require elevation but was protected for that user?

That seems like a reasonable approach to me. I'd avoid making it user configurable as that expands the security risks and the location needs to be protected from other users putting files there. @dsplaisted , we'd want to settle on the locations we'd use for linux, mac, and windows for Microsoft produced SDKs and run those user locations by our security team to get signoff.

[tmds]

To clarify here, the presence of the userlocal file would cause source build to install AND read workloads from a hardcoded user location, correct?

Yes. A source-built SDK would include the userlocal file somewhere under the <DOTNET ROOT>. That file indicates to the SDK it must install workloads to a user location instead of the <DOTNET ROOT>.

This location would not be controllable by the end user

The location would be in a directory somewhere under SpecialFolder.MyDocuments or SpecialFolder.LocalApplicationData. These locations can be changed using environment variables, but that is a well-known pattern.

that didn't require elevation but was protected for that user?

Yes, this is expected for these locations.

You may consider to add the userlocal file also to the Linux SDK packages Microsoft builds for .NET. That avoids the sudo anti-pattern which is now needed to add workloads to these SDKs.
It also makes the experience the same between source-built and Microsoft-built SDK packages.

[dsplaisted]

The userlocal file seems we're talking about seems like a good solution for source-built .NET SDKs.

It won't work well for mixing and matching DOTNET_ROOT and user local installs. But the primary issue there is supporting installation and updates to the different location. So for source-built .NET SDKs I think it should still work, as the base version of the source-built SDK could still deliver (source-built) workloads into DOTNET_ROOT, where they would still get resolved from, but updates would come from updating the .NET SDK.

That would mean that even if ASP.NET was factored out into a separate workload, from the source-built SDK perspective it would basically still be part of the .NET SDK.

[tmds]

The userlocal file seems we're talking about seems like a good solution for source-built .NET SDKs.

Yes, and even for Microsoft Linux packages I think it is more appropriate than the sudo install.

we'd want to settle on the locations we'd use for linux, mac, and windows for Microsoft produced SDKs and run those user locations by our security team to get signoff.

@marcpopMSFT @dsplaisted did they approve?

[dsplaisted]

@tmds @omajid I've created a tool to help create a local workload installation. You can find it here: https://github.com/dsplaisted/dotnet-local-workload. I took a stab at creating a tool instead of writing up instructions on how to manually install workloads, as it seemed like the manual instructions would be a pain to follow.

The tool is very basic and kind of hacked up but it worked for me in a very basic test. If we can get the user local workload support into the .NET 6.0 release then we may not need the tool much at all. Still, it would be nice if you can check it out and let us know how it works.

[dsplaisted]

@tmds If you are interested in starting to work on this, you can go ahead without waiting for our security team signoff. We don't think it's likely that they would totally veto the feature, it's more likely that they would have feedback on which folder we use for local folder installation.

Let us know if you need any guidance or anything.