Improve NuGet package debugging and symbols experience

[diverdan92]

Currently the NuGet package debugging and symbols experience is not streamlined.

Some areas that need improvement are:

1. Optimize the NuGet package and symbol authoring story to encourage ####production of symbols
2. Support for the new portable PDB format for .NET Core and .NET Standard applications
3. Better integration in Visual Studio tooling.

This issue will be used for discussion of the spec here.

[maartenba]

Would love to see this! Have had an offline discussion with some other folks, and came up with a dream workflow:

The main issue today is that when running “nuget.exe pack -Symbols” on a .NET Standard project, SourceLink is not enabled by default. Which makes a LOT of sense as not everyone is keen on sharing their sources. However: when people do want to have sources support, it’s usually too late and discovered during a debugging/profiling/dump analysis situation where symbols and sources would have been handy…

Going out on a limb, perhaps the ideal workflow would be something like this (trying to shape a flow and feature):

Package creation/publishing phase

• User, when packaging using nuget.exe pack -Symbols for Portable PDB
  • If the Portable PDB does not contain embedded sources + sourcelink
    • Check whether source_link.json is available. If not:
      • Generate one based on the current folder and make sure the Portable PDB contains it (may need a recompile… so maybe generating one should be a default of csc.exe instead of a NuGet feature)
        • The mapping would be “”: “%sourceservertemplate%/” (where documentname is actually filled out, and %sourceservertemplate% is a placeholder that is embedded into the file.
• User can then publish NuPkg and symbols NuPkg to wherever they want. If that “wherever” supports symbols, then yay!

Symbols consumption phase (user wants to fetch symbols and sources)

• If the Portable PDB is embedded in the assembly and has embedded sources etc, use the VS SourceLink feature to show sources
• If the Portable PDB must be downloaded, then download it from the configured symbol server using the traditional semantics of PDB ID and PDB name
  • After download, if sources are embedded, use the embedded sources
  • If sources are not embedded but the source_link.json is, and has a URL for document mappings, use the URL
  • If sources are not embedded but the source_link.json is, and has a URL that contains %sourceservertemplate%, interesting things happen!
    • 1. The debugger/VS/… would reach out to the symbol servers configured in VS and fetch, for example, http://symbolserver.local/services.json. This services.json would contain the actual value for %sourceservertemplate%. The value would be cached for the entire debugging session.
    • 2. The debugger/VS/… uses the %sourceservertemplate% placeholder in the URL that was in source_link.json, then fetches the sources from there.

That seems, for Portable PDB, to give the entire spectrum of embedded + sourcelink, to sourcelink with url mappings, to no sourcelink at all but still have debuggability.

[diverdan92]

@maartenba -

a. The debugger/VS/… would reach out to the symbol servers configured in VS and fetch, for example, http://symbolserver.local/services.json. This services.json would contain the actual value for %sourceservertemplate%. The value would be cached for the entire debugging session.
b. The debugger/VS/… uses the %sourceservertemplate% placeholder in the URL that was in source_link.json, then fetches the sources from there.

The way you envision this working, where would the source be uploaded to support these scenarios? It seems like the package author has not uploaded the source in this case?

[maartenba]

That comes from the creation phase:

• User, when packaging using nuget.exe pack -Symbols for Portable PDB
  • If the Portable PDB does not contain embedded sources + sourcelink
    • Check whether source_link.json is available. If not:
      • Generate one based on the current folder and make sure the Portable PDB contains it (may need a recompile… so maybe generating one should be a default of csc.exe instead of a NuGet feature)
      • The mapping would be “”: “%sourceservertemplate%/” (where documentname is actually filled out, and %sourceservertemplate% is a placeholder that is embedded into the file.

This is essentially the current NuGet behavior, where the generated package contains sources and a sourcelink file that holds a placeholder URL. That way, a server can serve up symbols based on the embedded sources.

Imagine the following scenario that you can do today:

• User runs nuget pack -Symbols and generates a .nupkg and .symbols.nupkg.
• User uploads both to server, and consumes only .nupkg in their project
• All of a sudden, debugging is required! They configure a symbol server that serves up the Portable PDB that was in .symbols.nupkg, and expect things to work. Except: they do not!
  • The Portable PDB contains some info, but not sources. It also does not contain a link to the sources. In other words: the user will not be able to do debugging.
  • If instead, NuGet had added the template URL to sourcelink (and VS could work with it), VS could query sources using that template URL, fetching the source files that were in the .symbols.nupkg

Another option is embedding the sources in the Portable PDB, which would be awesome, except for one thing: there is no user control possible. E.g. if I am a tools vendor and I want only my team to have debugging possibilities and not my clients, how do I prevent one group from using the symbols embedded in the Portabe PDB? For this scenario, having to reach out to the server will make it possible to limit access to PDB and source files.

[tmat]

@maartenba We are working on making it super easy to enable source link in builds for both public and private repos, as well as embed sources to PDBs that not tracked by a source control server (e.g. sources generated during the build). Our goal is to reduce the number of packages that were not built with source link or embedded sources. That said, we understand that there will be cases where it will be necessary to add sources after the fact. The scenario is valid and we have some ideas how to address it in future. I don't think NuGet is the right tool to implement this support though (by creating source link files or recompiling PDBs).

[anangaur]

When publishing packages, both the .symbols.nupkg and the .nupkg will be published to NuGet.org. We will make the nuget and dotnet push CLI automatically push both .nupkgs and .symbols.nupkgs (if they exist). We will add a parameter to the dotnet and nuget CLIs to opt out of automatic pushing of .symbols packages.

I would say we should opt in to publish the symbols and by default only the .nupkg is pushed not the .symbols.nupkg.

We are already taking the change to not auto-push package. (See #6113). Lets not randomize :)