IIS InProcess Hosting - Debugging Errors

[RickStrahl]

I'm running into a problem with .NET Core 3.1 application that I'm hosting on IIS. I'm testing locally on my local machine and the application runs fine in out of process.

I have logging on and I get a trace log, but the failure is in the guts of the runtime without any hint why it's failing:

Exception Info:
Object reference not set to an instance of an object.
---
   at Microsoft.AspNetCore.Hosting.WebHostBuilderIISExtensions.<>c__DisplayClass0_0.<UseIIS>b__2(IISServerOptions options)
   at Microsoft.Extensions.Options.ConfigureNamedOptions`1.Configure(String name, TOptions options)
   at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
   at Microsoft.Extensions.Options.OptionsManager`1.<>c__DisplayClass5_0.<Get>b__0()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Microsoft.Extensions.Options.OptionsCache`1.GetOrAdd(String name, Func`1 createOptions)
   at Microsoft.Extensions.Options.OptionsManager`1.Get(String name)
   at Microsoft.Extensions.Options.OptionsManager`1.get_Value()
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpServer..ctor(IISNativeApplication nativeApplication, IHostApplicationLifetime applicationLifetime, IAuthenticationSchemeProvider authentication, IOptions`1 options, ILogger`1 logger)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
   at WebConnectionWebServer.Program.Main(String[] args) in C:\WebConnection\WebConnectionWebServer\WebConnectionWebServer\Program.cs:line 38
Microsoft.AspNetCore.Server.IIS

My code stops at the call to .Build().

builder
  .Build()  <---  Last of my code
  .Run();

The Web.Config is simple and as I mentioned the app starts fine with OutOfProcess hosting:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <!-- check the path to the dll - in Project the below works   
         in a plain publish output use `.\WebConnectionWebServer.dll`  -->
    <handlers>
          <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
    </handlers>
    <aspNetCore processPath="dotnet" 
    arguments="C:\WebConnection\WebConnectionWebServer\WebConnectionWebServer\bin\Publish\WebConnectionWebServer.dll" 
    stdoutLogEnabled="true" stdoutLogFile=".\logs\stdout" hostingModel="inprocess">
      <environmentVariables>
        <environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Production" />
        <environmentVariable name="WEBCONNECTION_USELIVERELOAD" value="False" />
        <environmentVariable name="WEBCONNECTION_OPENBROWSER" value="False" />
        <environmentVariable name="WEBCONNECTION_SHOWURLS" value="False" />
      </environmentVariables>
    </aspNetCore>
  </system.webServer>
</configuration>

The IIS AppPool runs under System so permissions should not be a problem in this scenario. I'm loading the DLL (not an EXE which would be nice if it worked with a standalone Exe BTW).

My Problem

I have no idea what else to look at to debug this. Based on the error message, this is getting into the IIS hosting bits of the builder and failing there, but there's not enough information for me to see anything beyond that.

Further technical details

• ASP.NET Core 3.1 SDK installed
• 3.1.0 Runtimes for all frameworks

[davidfowl]

You can try setting ASPNETCORE_DETAILEDERRORS=1 or ASPNETCORE_ENVIRONMENT=Development to see a more detailed error message.

[jkotalik]

I believe you are hitting #17973.

[RickStrahl]

@jkotalik Yup - that's totally it!

I thought wasn't referencing any of those dependencies directly as this application is very light on its dependencies, so it's using primarily the ASP.NET Framework reference.

However I noticed that when I built the DotnetTool nuget package was referencing the .NET Core 2.2 in process hosting dll which I thought was weird.

Sure enough checked my project and for reasons I don't remember I had a ref to:

<PackageReference Include="Microsoft.AspNetCore.Server.IIS" Version="2.2.6" />

Removed that and now InProcess hosting is working.

Thanks. I would have never figured that one out on my own.

[jkotalik]

Yeah, it a tough one that we didn't even recognize as a regression until a few weeks ago. Glad that fixed the issue.