Packages that share dependencies with Microsoft.AspNetCore.App cannot reference patch versions

[natemcmaster]

All NuGet packages (not just ours, but the entire world) which share a dependency with Microsoft.AspNetCore.App cannot not reference versions of those dependencies higher than the default implicit Microsoft.AspNetCore.App version without introducing a version conflict. When a version conflict exists, the following errors will often occur.

error NU1107: Version conflict detected for Microsoft.AspNetCore.Authorization. Reference the package directly from the project to resolve this issue.
error NU1107:  MyApp-> Microsoft.AspNetCore.Mvc.Testing 2.1.1 -> Microsoft.AspNetCore.Mvc.Core 2.1.1 -> Microsoft.AspNetCore.Authorization.Policy 2.1.1 -> Microsoft.AspNetCore.Authorization (>= 2.1.1)
error NU1107:  MyApp-> Microsoft.AspNetCore.App 2.1.0 -> Microsoft.AspNetCore.Authorization (= 2.1.0).

This makes updates in NuGet UI impossible, and updates by hand very difficult. Also, it is not clear from the error messages how a customer should resolve the issue.

Solutions

UPDATE: jump to more detailed analysis: #1180 (comment)

• Lift version constraints on Microsoft.AspNetCore.App
• Introduce a new target framework for aspnetcore
• Ensure packages that share dependencies with Microsoft.AspNetCore.App never reference anything higher than the base line (see Details)

Microsoft.AspNetCore.App 2.1.0 baseline

The following table contains a list of all dependencies of Microsoft.AspNetCore.App 2.1.0. Package authors should adhere to these versions

Table: (open details)

Package ID	Version
Microsoft.AspNet.WebApi.Client	5.2.4
Microsoft.AspNetCore	2.1.0
Microsoft.AspNetCore.Antiforgery	2.1.0
Microsoft.AspNetCore.Authentication	2.1.0
Microsoft.AspNetCore.Authentication.Abstractions	2.1.0
Microsoft.AspNetCore.Authentication.Cookies	2.1.0
Microsoft.AspNetCore.Authentication.Core	2.1.0
Microsoft.AspNetCore.Authentication.Facebook	2.1.0
Microsoft.AspNetCore.Authentication.Google	2.1.0
Microsoft.AspNetCore.Authentication.JwtBearer	2.1.0
Microsoft.AspNetCore.Authentication.MicrosoftAccount	2.1.0
Microsoft.AspNetCore.Authentication.OAuth	2.1.0
Microsoft.AspNetCore.Authentication.OpenIdConnect	2.1.0
Microsoft.AspNetCore.Authentication.Twitter	2.1.0
Microsoft.AspNetCore.Authentication.WsFederation	2.1.0
Microsoft.AspNetCore.Authorization	2.1.0
Microsoft.AspNetCore.Authorization.Policy	2.1.0
Microsoft.AspNetCore.Connections.Abstractions	2.1.0
Microsoft.AspNetCore.CookiePolicy	2.1.0
Microsoft.AspNetCore.Cors	2.1.0
Microsoft.AspNetCore.Cryptography.Internal	2.1.0
Microsoft.AspNetCore.Cryptography.KeyDerivation	2.1.0
Microsoft.AspNetCore.DataProtection	2.1.0
Microsoft.AspNetCore.DataProtection.Abstractions	2.1.0
Microsoft.AspNetCore.DataProtection.Extensions	2.1.0
Microsoft.AspNetCore.Diagnostics	2.1.0
Microsoft.AspNetCore.Diagnostics.Abstractions	2.1.0
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore	2.1.0
Microsoft.AspNetCore.HostFiltering	2.1.0
Microsoft.AspNetCore.Hosting	2.1.0
Microsoft.AspNetCore.Hosting.Abstractions	2.1.0
Microsoft.AspNetCore.Hosting.Server.Abstractions	2.1.0
Microsoft.AspNetCore.Html.Abstractions	2.1.0
Microsoft.AspNetCore.Http	2.1.0
Microsoft.AspNetCore.Http.Abstractions	2.1.0
Microsoft.AspNetCore.Http.Connections	1.0.0
Microsoft.AspNetCore.Http.Connections.Common	1.0.0
Microsoft.AspNetCore.Http.Extensions	2.1.0
Microsoft.AspNetCore.Http.Features	2.1.0
Microsoft.AspNetCore.HttpOverrides	2.1.0
Microsoft.AspNetCore.HttpsPolicy	2.1.0
Microsoft.AspNetCore.Identity	2.1.0
Microsoft.AspNetCore.Identity.EntityFrameworkCore	2.1.0
Microsoft.AspNetCore.Identity.UI	2.1.0
Microsoft.AspNetCore.JsonPatch	2.1.0
Microsoft.AspNetCore.Localization	2.1.0
Microsoft.AspNetCore.Localization.Routing	2.1.0
Microsoft.AspNetCore.MiddlewareAnalysis	2.1.0
Microsoft.AspNetCore.Mvc	2.1.0
Microsoft.AspNetCore.Mvc.Abstractions	2.1.0
Microsoft.AspNetCore.Mvc.Analyzers	2.1.0
Microsoft.AspNetCore.Mvc.ApiExplorer	2.1.0
Microsoft.AspNetCore.Mvc.Core	2.1.0
Microsoft.AspNetCore.Mvc.Cors	2.1.0
Microsoft.AspNetCore.Mvc.DataAnnotations	2.1.0
Microsoft.AspNetCore.Mvc.Formatters.Json	2.1.0
Microsoft.AspNetCore.Mvc.Formatters.Xml	2.1.0
Microsoft.AspNetCore.Mvc.Localization	2.1.0
Microsoft.AspNetCore.Mvc.Razor	2.1.0
Microsoft.AspNetCore.Mvc.Razor.Extensions	2.1.0
Microsoft.AspNetCore.Mvc.Razor.ViewCompilation	2.1.0
Microsoft.AspNetCore.Mvc.RazorPages	2.1.0
Microsoft.AspNetCore.Mvc.TagHelpers	2.1.0
Microsoft.AspNetCore.Mvc.ViewFeatures	2.1.0
Microsoft.AspNetCore.NodeServices	2.1.0
Microsoft.AspNetCore.Owin	2.1.0
Microsoft.AspNetCore.Razor	2.1.0
Microsoft.AspNetCore.Razor.Design	2.1.0
Microsoft.AspNetCore.Razor.Language	2.1.0
Microsoft.AspNetCore.Razor.Runtime	2.1.0
Microsoft.AspNetCore.ResponseCaching	2.1.0
Microsoft.AspNetCore.ResponseCaching.Abstractions	2.1.0
Microsoft.AspNetCore.ResponseCompression	2.1.0
Microsoft.AspNetCore.Rewrite	2.1.0
Microsoft.AspNetCore.Routing	2.1.0
Microsoft.AspNetCore.Routing.Abstractions	2.1.0
Microsoft.AspNetCore.Server.HttpSys	2.1.0
Microsoft.AspNetCore.Server.IISIntegration	2.1.0
Microsoft.AspNetCore.Server.Kestrel	2.1.0
Microsoft.AspNetCore.Server.Kestrel.Core	2.1.0
Microsoft.AspNetCore.Server.Kestrel.Https	2.1.0
Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions	2.1.0
Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets	2.1.0
Microsoft.AspNetCore.Session	2.1.0
Microsoft.AspNetCore.SignalR	1.0.0
Microsoft.AspNetCore.SignalR.Common	1.0.0
Microsoft.AspNetCore.SignalR.Core	1.0.0
Microsoft.AspNetCore.SignalR.Protocols.Json	1.0.0
Microsoft.AspNetCore.SpaServices	2.1.0
Microsoft.AspNetCore.SpaServices.Extensions	2.1.0
Microsoft.AspNetCore.StaticFiles	2.1.0
Microsoft.AspNetCore.WebSockets	2.1.0
Microsoft.AspNetCore.WebUtilities	2.1.0
Microsoft.CodeAnalysis.Razor	2.1.0
Microsoft.EntityFrameworkCore	2.1.0
Microsoft.EntityFrameworkCore.Abstractions	2.1.0
Microsoft.EntityFrameworkCore.Analyzers	2.1.0
Microsoft.EntityFrameworkCore.Design	2.1.0
Microsoft.EntityFrameworkCore.InMemory	2.1.0
Microsoft.EntityFrameworkCore.Relational	2.1.0
Microsoft.EntityFrameworkCore.SqlServer	2.1.0
Microsoft.EntityFrameworkCore.Tools	2.1.0
Microsoft.Extensions.Caching.Abstractions	2.1.0
Microsoft.Extensions.Caching.Memory	2.1.0
Microsoft.Extensions.Caching.SqlServer	2.1.0
Microsoft.Extensions.Configuration	2.1.0
Microsoft.Extensions.Configuration.Abstractions	2.1.0
Microsoft.Extensions.Configuration.Binder	2.1.0
Microsoft.Extensions.Configuration.CommandLine	2.1.0
Microsoft.Extensions.Configuration.EnvironmentVariables	2.1.0
Microsoft.Extensions.Configuration.FileExtensions	2.1.0
Microsoft.Extensions.Configuration.Ini	2.1.0
Microsoft.Extensions.Configuration.Json	2.1.0
Microsoft.Extensions.Configuration.KeyPerFile	2.1.0
Microsoft.Extensions.Configuration.UserSecrets	2.1.0
Microsoft.Extensions.Configuration.Xml	2.1.0
Microsoft.Extensions.DependencyInjection	2.1.0
Microsoft.Extensions.DependencyInjection.Abstractions	2.1.0
Microsoft.Extensions.DiagnosticAdapter	2.1.0
Microsoft.Extensions.FileProviders.Abstractions	2.1.0
Microsoft.Extensions.FileProviders.Composite	2.1.0
Microsoft.Extensions.FileProviders.Embedded	2.1.0
Microsoft.Extensions.FileProviders.Physical	2.1.0
Microsoft.Extensions.FileSystemGlobbing	2.1.0
Microsoft.Extensions.Hosting	2.1.0
Microsoft.Extensions.Hosting.Abstractions	2.1.0
Microsoft.Extensions.Http	2.1.0
Microsoft.Extensions.Identity.Core	2.1.0
Microsoft.Extensions.Identity.Stores	2.1.0
Microsoft.Extensions.Localization	2.1.0
Microsoft.Extensions.Localization.Abstractions	2.1.0
Microsoft.Extensions.Logging	2.1.0
Microsoft.Extensions.Logging.Abstractions	2.1.0
Microsoft.Extensions.Logging.Configuration	2.1.0
Microsoft.Extensions.Logging.Console	2.1.0
Microsoft.Extensions.Logging.Debug	2.1.0
Microsoft.Extensions.Logging.EventSource	2.1.0
Microsoft.Extensions.Logging.TraceSource	2.1.0
Microsoft.Extensions.ObjectPool	2.1.0
Microsoft.Extensions.Options	2.1.0
Microsoft.Extensions.Options.ConfigurationExtensions	2.1.0
Microsoft.Extensions.Primitives	2.1.0
Microsoft.Extensions.WebEncoders	2.1.0
Microsoft.Net.Http.Headers	2.1.0

[natemcmaster]

Current thinking on the baseline dependency version approach:

New dependency versioning rules

• Any package with <dependency> on a package inside the .App sharedfx MUST NOT reference anything other than 2.1.0
• No longer cascade dependencies updates

Customer Impact

• All projects not referencing Microsoft.AspNetCore.App (e.g. .NET Framework, .NET Core console apps)
  • Users must explicitly declare all dependencies, including transitive dependencies, to get latest patch versions, including security updates
• Projects referencing Microsoft.AspNetCore.App
  • Implicit version remains 2.1.0.
  • No changes to plans for updates to portable apps: security updates should be rolled out by deploying the new aspnet shared runtime
  • No changes to plans for self-contained apps: the web SDK will update for each release to bump the implicit Microsoft.AspNetCore.App version to latest 2.1.x.

Infrastructure impact

We will need significant changes to our build system.

• Drop our version coherence check
• Drop our cascading versions
• Drop ProdCon version updates to partner teams that use anything but Microsoft.AspNetCore.App
• Drop our CI automation for updating versions in dependencies.props for now
  • Improve our automation to understand how to split fx dependencies vs test dependencies
• Override default dotnet pack behavior
• Update all test projects to contain the complete closure of dependencies. They cannot rely on transitive dependencies
• All aspnet repos must become aware of what is in Microsoft.AspNetCore.App in order to select the right dependency version
• Split dependencies.props into two sections:
  • Dependency versions to test
  • Dependency to reference in nuspec

cc @muratg @DamianEdwards @davidfowl

[natemcmaster]

Thought about this more. We probably shouldn't handcraft a nuspec to force downgrade dependencies in generated nuspec to "2.1.0" for ProjectReferences. It would lead to a mismatch of assembly reference and dependency versions

e.g.

Kestrel.nupkg 2.1.1
  <dependency id=Kestrel.Core version=2.1.0 />

lib/netstandard2.0/Kestrel.dll
   // reference Microsoft.AspNetCore.Server.Kestrel.Core, Version=2.1.1.0

warning MSB3277: There was a conflict between "Microsoft.AspNetCore.Server.Kestrel.Core, Version=2.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60" and "Microsoft.AspNetCore.Server.Kestrel.Core, Version=2.1.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60".

Alternatives

• Replace all ProjectReferences with PackageReferences
• Don't rev assembly version

cc @davidfowl @anurse @Tratcher

[DamianEdwards]

Revving assembly version is required for servicing, so that's a non-starter.

[natemcmaster]

Didn't know that. I thought we only had to rev file version for servicing.

[DamianEdwards]

The runtime's loading policy on Core uses assembly versioning for tracking and substituting patched versions. On NetFx the same is used for tracking and the GAC is used for substitution, which uses assembly versioning.

[natemcmaster]

Oh right.

Seems like the only viable option is replacing all ProjectReferences with PackageReferences. This is unfortunate -- it introduces a bunch of pain for devs working on patches.

[davidfowl]

Repos aren't patch families anymore so each project needs to burn in the version.

[natemcmaster]

Just kicking around other ideas. I haven't fully tested, but I believe an approach using MSBuild targets to fake the behavior of a custom target framework may work.

Packages in the AspNetCore.App sharedfx (such as Microsoft.Extension.Options) look like this

Microsoft.Extension.Options.2.1.999.nupkg
lib/netcoreapp2.1/_._
lib/netstandard2.0/Microsoft.Extension.Options.dll  (version = 2.1.999.0)
build/netcoreapp2.1/Microsoft.Extension.Options.targets
build/netcoreapp2.1/ref/Microsoft.Extension.Options.targets (assembly version = 2.1.0.0)

<!-- build/netcoreapp2.1/Microsoft.Extension.Options.targets -->
<Project>
  <ItemGroup Condition="'$(_AspNetCoreAppSharedFxIsEnabled)' == 'true'">
    <Reference Include="$(MSBuildThisFileDirectory)ref\Microsoft.Extension.Options.dll">
        <Visible>false</Visible>
        <NuGetPackageId>Microsoft.Extension.Options</NuGetPackageId>
        <NuGetPackageVersion>2.1.999</NuGetPackageVersion>
        <Facade>true</Facade>
        <Private>false</Private>
     </Reference>
   </ItemGroup>
  <ItemGroup Condition="'$(_AspNetCoreAppSharedFxIsEnabled)' != 'true'">
    <Reference Include="$(MSBuildThisFileDirectory)..\lib\netstandard2.0\Microsoft.Extension.Options.dll">
        <Visible>false</Visible>
        <NuGetPackageId>Microsoft.Extension.Options</NuGetPackageId>
        <NuGetPackageVersion>2.1.999</NuGetPackageVersion>
        <Private>false</Private>
     </Reference>
   </ItemGroup>
</Project>

[natemcmaster]

Failed to add this, but along with ⬆️ that suggestion, I mean to add -- we could lift the max version constraint on Microsoft.AspNetCore.App dependencies. It would require bumping the implicit version to 2.1.1 to make up for doing this wrong in 2.1.0, but it may worth some short-term pain for long-term gains.

[natemcmaster]

Ok, I played with this more. Here are my findings. To make the comparison easier, I'm going to describe solutions to the original problem and scenarios.

Solutions

(details below)

Name	Description
(A) Do nothing	Sorry world.
(B) Update implicit metapackage version	Update the Microsoft.AspNetCore.App version
(C) Baseline dependencies	All packages should use the 2.1.0 version of any dependency that is shared with Microsoft.AspNetCore.App
(D) MSBuild References	Use targets embedded in the package to manipulate manually the <Reference> items
(E) Platform manifest	Write a platform manifest that lists which assemblies are in the shared framework

Scenarios

Name	Description
Transitive dependencies	If users update only top-level packages, their transitive dependencies are updated, too
.NET Framework	Package can be used on .NET Framework projects
NETCore.App	Package can be used on a non-web project that targets Microsoft.NETCore.App
AspNetCore.App	Package can be used on a web project using the Microsoft.AspNetCore.App shared framework
Tooling	Visual Studio, CLI, etc. tooling works well

Comparison

_	Transitive dependencies	.NET Framework	NETCore.App	AspNetCore.App	Tooling
(A) Do nothing	✅	✅	✅	❌ users lose advantages of trimming	❌ users must override all conflicts
(B) Update implicit metapackage version	✅	n/a	n/a	🤙 deployment to servers that are missing the updated runtime will fail	✅
(C) Baseline dependencies	❌ users must manually list and update the complete closure	✅	✅	✅	❌ No updates are shown for transitive dependencies. Users will not easily be able to see updates are available
(D) MSBuild References	✅	n/a	🤙 Host cannot resolve assembly from NuGet cache, but otherwise ✅	✅	🤙 Requires an update to AspnetCore.App to avoid NuGet conflicts ❌ This does not work with P2P references because PrivateAssets=Build by default.
(E) Platform manifest	✅	n/a	✅	✅	🤙 Requires an update to AspnetCore.App to avoid NuGet conflicts, otherwise ✅

Solution details

Update implicit metapackage version (B)

• On each update to our packages and Microsoft.AspNetCore.App, we ensure the .NET Core SDK updates the implicit package version for Microsoft.AspNetCore.App
• Down sides: users may see this error when deploying to a server without the update installed

The specified framework 'Microsoft.AspNetCore.App', version '2.1.1' was not found.
  - Check application dependencies and target a framework version installed at:
      C:\Users\namc\.dotnet\x64\
  - Installing .NET Core prerequisites might help resolve this problem:
      http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409
  - The .NET Core framework and SDK can be installed from:
      https://aka.ms/dotnet-download
  - The following versions are installed:
      2.1.0 at [C:\Users\namc\.dotnet\x64\shared\Microsoft.AspNetCore.App]

Baseline dependencies (C)

• All packages (not just ours) which share a dependency with Microsoft.AspNetCore.App MUST NOT reference patch versions

See also #1180 (comment)

MSBuild References (D)

• Create a new version of Microsoft.AspNetCore.App without maximum version constraints
• Add lib/netcoreapp2.1/_._ to packages in Microsoft.AspNetCore.App so by default this reference is a no-op on .NET Core projects.
• Use MSBuild targets to add appropriate <Reference> items to add-back the reference conditionally checking for Microsoft.AspNetCore.App's presence

See also this comment: #1180 (comment)

Platform manifest (E)

This approach makes it impossible to override what is in the shared framework.

• Create a new version of Microsoft.AspNetCore.App without maximum version constraints
• Update the SDK to include a platform manifest for Microsoft.AspNetCore.App with contents like this

Microsoft.AspNet.WebApi.Client.dll|Microsoft.AspNet.WebApi.Client|99.99.99.99|99.99.99.99
Microsoft.AspNetCore.dll|Microsoft.AspNetCore|99.99.99.99|99.99.99.99
Microsoft.AspNetCore.Antiforgery.dll|Microsoft.AspNetCore.Antiforgery|99.99.99.99|99.99.99.99
Microsoft.AspNetCore.Authentication.dll|Microsoft.AspNetCore.Authentication|99.99.99.99|99.99.99.99
Microsoft.AspNetCore.Authentication.Abstractions.dll|Microsoft.AspNetCore.Authentication.Abstractions|99.99.99.99|99.99.99.99
....

• Microsoft.NET.Sdk will filter out the reference from the NuGet package in favor of the one in the platform

  Encountered conflict between 'Platform:Microsoft.Extensions.Options.dll' and 'Runtime:C:\Users\namc.nuget\packages\microsoft.extensions.options\2.1.1\lib\netstandard2.0\Microsoft.Extensions.Options.dll'. Choosing 'Platform:Microsoft.Extensions.Options.dll' because AssemblyVersion '99.99.99.99' is greater than '2.1.1.0'.

• This does not filter what the compiler gets, so we need to add ref/netcoreapp2.1/*.dll files with assembly version 2.1.0.0 so apps crash due to forced downgrade