OpenAPI example crashes at startup after update to VS 2022 v17.9.0

[bkoelman]

After updating to VS 2022 v17.9.0, launching project JsonApiDotNetCoreExample (F5) crashes the OpenAPI integration at startup. Oddly, it does not happen without the debugger attached, or when the OAS .json file is generated during build, or when run from the command line:

cd src\Examples\JsonApiDotNetCoreExample
dotnet run --framework=net80

All tests are green, both locally and in GitHub Actions. I've rewinded the openapi branch several PR merges back in time; same issue.

The exception thrown on F5 is the following:

System.InvalidOperationException: This operation is only valid on generic types.
   at System.RuntimeType.GetGenericTypeDefinition()
   at JsonApiDotNetCore.OpenApi.JsonApiOperationIdSelector.GetDocumentType(ApiDescription endpoint) in D:\Bart\Source\Repos\JsonApiDotNetCore\src\JsonApiDotNetCore.OpenApi\JsonApiOperationIdSelector.cs:line 90

I've traced it down to the following:

• JsonApiOperationIdSelector.GetDocumentType() expects a generic type, such as ResourceCollectionResponseDocument<Person>, but it gets System.Void instead. This means that our generic-type expansion does not execute.
• Generic type expansion rewrites an endpoint like /people/{id}/relationships/{relationshipName} to /people/{id}/relationships/ownedTodoItems and /people/{id}/relationships/assignedTodoItems. And it sets the controller return type of GetAsync() to ResourceCollectionResponseDocument<Person>, which is how it is defined in JSON:API. This all happens in JsonApiActionDescriptorCollectionProvider.
• We politely register our JsonApiActionDescriptorCollectionProviderin the IoC container using TryAddSingleton<IApiDescriptionGroupCollectionProvider>(), so developers can override it.
• Only when running in the debugger, an existing registration already exists, so that ours won't be added.
• Tracing it back, the registration is already present immediately after returning from WebApplicationBuilder builder = WebApplication.CreateBuilder(args);. How would it get there?
• Searching the sources of ASP.NET and debugging with symbols loaded (added breakpoints inside the disassembled sources), I found that services.AddEndpointsApiExplorer() executes, which registers the default provider during WebApplication.CreateBuilder.
• When running under the debugger, Visual Studio sets the environment variables DOTNET_STARTUP_HOOKS and APIDISCOVERY_FILEPATH, resulting in the execution of Microsoft.WebTools.ApiEndpointDiscovery.HostingStartup.Configure before the assembly entry point (Main) executes, which calls IWebHostBuilder.ConfigureServices(). This is to populate the View > Other Windows > Endpoints Explorer tool window, which shows all OpenAPI endpoints in the IDE. See also https://developercommunity.visualstudio.com/t/WebConfig-invalid-after-using-Visual-St/10555985?pageSize=15&sort=active&topics=windows+10.0 and https://github.com/dotnet/runtime/blob/main/docs/design/features/host-startup-hook.md.

The fix is to not politely register, but forcibly overwrite the existing IApiDescriptionGroupCollectionProvider singleton.