NuGetFallbackFolder adds unnecessary bloat to build containers

[natemcmaster]

In .NET Core 2.0, the CLI started shipping a feature that adds an offline package feed called the NuGetFallbackFolder. This adds unnecessary bloat to SDK images in the following ways:

• ~660 MB of xmldocs are stored in ~/.dotnet/NuGetFallbackFolder. xmldocs are unnecessary in build images.
• first run of dotnet-restore creates about 5,700 duplicate files which adds about 326 MB. This duplication is caused by copying files from ~/.dotnet/NuGetFallbackFolder to ~/.nuget/packages
• Any users that build inside the image and don't clear out the ~/.nuget/packages folder end up with a bloated image. Here is the resulting duplication between the nuget cache and fallback folders:

  fdupes -r -m ~/.dotnet/NuGetFallbackFolder/ ~/.nuget/packages/
  11439 duplicate files (in 2809 sets), occupying 876.3 megabytes
  

Suggestions

• Exclude xmldocs from NuGet fallback folder
• Disable the offline package cache feature altogether.
• Warmup the NuGet cache by running dotnet-restore as a part of the image creation.

Or some combination of the ideas above.

Details
How I measured the numbers
Using: microsoft/dotnet-nightly:2.0.0-preview1-sdk

• Measure the effect of a dotnet-restore
  1. Install fdupes tool. apt-get update -qq && apt-get install -y fdupes
  2. mkdir /app && cd /app
  3. ls ~/.nuget/. The ~/.nuget/packages/ folder does not yet exist.
  4. dotnet new web (automatically launches dotnet restore)
  5. fdupes -r -m ~/.dotnet/NuGetFallbackFolder/ ~/.nuget/packages/
  6. Count files added to nuget cache: find ~/.nuget/packages -type f | wc -l
• Count xmldoc files
  find ~/.dotnet -name "*.xml" -type f | wc -l
• Count xmldoc filesize:
  find ~/.dotnet/ -name "*.xml" -type f -print0 | du -sb --files0-from=- | awk '{ total += $1} END { print total/1024/1024 }' (number is in MB)

cc @emgarten @rrelyea @dotnet/dotnet-cli @glennc

[emgarten]

This will improve once the fallback folder is used as an actual fallback folder.

Long term to solve this I'd like to see either a dotnet or NuGet command that takes a set of project.assets.json files and prunes out all unneeded files from the package folders. This would include assemblies from unused frameworks, all dlls and files not listed in the assets file, and resource dlls that aren't used.

[natemcmaster]

This will improve once the fallback folder is used as an actual fallback folder.

@emgarten is this just a preview1 limitation that will be fixed before 2.0.0 RTM?

Also FYI the ~/.dotnet/NuGetFallbackFolder on Linux takes up about 1.0 GB.

[emgarten]

@emgarten is this just a preview1 limitation that will be fixed before 2.0.0 RTM?

That is the current goal

[tmds]

Does this help?

ENV NUGET_XMLDOC_MODE=skip
ENV DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1

[natemcmaster]

The images already set NUGET_XMLDOC_MODE=skip, but it's not honored by the first-time experience. If we decided to disable the offline package cache, setting DOTNET_SKIP_FIRST_TIME_EXPERIENCE is the best way to do that.

[tmds]

Is the /opt/dotnet/store also used during dotnet restore to avoid duplicating those packages to ~/.nuget/packages?

[natemcmaster]

If you meant /usr/share/dotnet/store, that is the runtime store feature that's new to 2.0. These are a subset of files from the ~/.nuget/packages folder that have been crossgened for faster boot time, similar to the "optimization cache" feature in .NET Core 1.x. The subset doesn't include all assets required to build an app.

[tmds]

The subset doesn't include all assets required to build an app.

Ah. So this doesn't get used by dotnet restore and the sdk image won't contain a store since it would only bloat the image.

[MichaelSimons]

@emgarten, What is the latest status of the underlying issue? Can you link to the product issue?

[emgarten]

@MichaelSimons it is tracked here: https://github.com/dotnet/cli/issues/6507

[natemcmaster]

It looks like the fallback folder behavior is working now. However, I think there is still an opportunity here to trim some bloat. If I've done my bash/math right, here are the (uncompressed) size of files in /usr/share/dotnet

[MichaelSimons]

I logged a cli issue to add support for this scenario. There are numerous use cases which would benefit from this trimming capability. Our Dockerfiles should not have to define this trimming logic.