Https configuration issues with Kestrel in aspnetcore 2.0

[Jonathan34]

I faced the issue https://github.com/dotnet/cli/issues/6516 and was advised to move to the docker image aspnetcore 2 rather than the dotnet.

dotnet --info

NET Command Line Tools (2.0.0-preview1-005977)

Product Information:
 Version:            2.0.0-preview1-005977
 Commit SHA-1 hash:  414cab8a0b

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.12
 OS Platform: Darwin
 RID:         osx.10.12-x64
 Base Path:   /usr/local/share/dotnet/sdk/2.0.0-preview1-005977/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0-preview1-002111-00
  Build    : 1ff021936263d492539399688f46fd3827169983

startup.cs

 WebHost.CreateDefaultBuilder(args).UseStartup<Startup>().Build().Run();

When I run my app locally, I set the ASPNETCORE_ENV to http://localhost:5000 using vscode tasks.
When I deploy (to Google App Engine) via Dockerfile, i set the following (notice http vs https):

EXPOSE 8080/tcp
ENV ASPNETCORE_URLS https://*:8080

This was good because I could control the port and http/https using environment variable only.
Now Kestrel returns:

System.InvalidOperationException: HTTPS endpoints can only be configured using KestrelServerOptions.Listen().

It seems I have to hardcode this value in Program.cs which I can't really do (flexibility is lost).

I deploy several services in docker containers that are ALL listening on port 8080 (Google App Engine requirement). App Engine provides the https certificate automatically so i do now own it (its probably just a reverse proxy by the way but it redirect to https).
But when I run them locally, I override the env var and specify some custom port 5000, 5001 so I can run them all at the same time...

How would you recommend I now handle this case using the aspnetcore 2.0 image?
The Kestrel ListenOptionsHttpsExtensions seem to require a certificate or a password which I do not have as it is managed by Google...

PS: If i use this code

.UseKestrel(options =>
                {
                    options.Listen(IPAddress.Any, 8080);
                }

and overrides using an environment variable on top, it won't:

Microsoft.AspNetCore.Server.Kestrel:Warning: Overriding address(es) http://*:5001. Binding to endpoints defined in UseKestrel() instead.
      Overriding address(es) http://*:5001. Binding to endpoints defined in UseKestrel() instead.

[Drawaes]

Google can't provide a certificate automatically. If they are reverse proxying with ssl termination then kestrel will run in http

[Jonathan34]

I assumed the same, i will revert everything back to http://localhost and will see if things are working properly

[igorkrug]

Does this help?

BTW can somebody confirm that using hosting.json that way is still the "correct" way of doing it?
I can update my answer on stackoverflow if it isn't.

[Jonathan34]

Partially, the order of precedence is not clear though, UseUrls, KestrelOptions.Listen(), ASPNETCORE_ENV, and hosting.json. What order are options read from?

[Tratcher]

I assume this is 1.1? It's changed some for 2.0. @CesarBS ?

[igorkrug]

I did the update of some projects from 1.1 to 2.0 and I'm using hosting.json in 2.0
All I did was call the UseConfiguration method after CreateDefaultBuilder, like this

public static IWebHost BuildWebHost(string[] args) => 
	WebHost.CreateDefaultBuilder(args)
		.UseConfiguration(new ConfigurationBuilder()
			.SetBasePath(Directory.GetCurrentDirectory())
			.AddJsonFile("hosting.json", optional: true)
			.Build()
		)
		.UseStartup<Startup>()
		.Build();

But I don't now if it is the correct way of doing it or if I should set the urls in another way.

[cesarblum]

@Jonathan34 After we changed the way Kestrel binds to addresses and ports - the new Listen() APIs -, you can only set up HTTPS endpoints via the new APIs. They are not supported in UseUrls() anymore, nor via environment variables.

If you want to be able to override your HTTP endpoints using the ASPNETCORE_URLS environment variable, you have to call PreferHostingUrls(true) on the web host builder e.g.:

var host = new WebHostBuilder()
    .UseKestrel(options =>
    {
        ...
    })
    .PreferHostingUrls(true)

If you don't want to set your HTTPS endpoints programmatically, you can use one of the ASP.NET Core metapackages. When using a metapackage you can configure Kestrel's endpoints (including HTTPS endpoints) using a configuration file. I recommend you look at the MetaPackages's repo SampleApp (https://github.com/aspnet/MetaPackages/tree/dev/samples/SampleApp) to learn how to set that up.

[Jonathan34]

@CesarBS I see. I ended up listening to non https at the end as https seems to require we provide a certificate (which i don't have). That should be fine since Google handles this https connection from the outside.
You can close the ticket if you want

[cesarblum]

@Jonathan34 Indeed, not having the certificate would prevent you from setting HTTPS up in Kestrel.

I've just edited my previous post with a note on our metapackages, which enable you to use external configuration for HTTPS endpoints.

[raffaeler]

@CesarBS I can't find any documentation about the kestrel configuration changes and this is very frustrating. Could you please point everyone to a complete explanation about the configurations (by code and by config files) ?

Thank you

[halter73]

@raffaeler Kestrel configuration by config files still needs to be done manually at this point. aspnet/Announcements#212 covers the new Kestrel binding/configuration APIs. If you aren't doing SSL termination in Kestrel, you can continue to call WebHostBuilder.UseUrls() like you do in Kestrel 1.x.

[raffaeler]

@halter73 Are you saying that everyone should craft his own config file, or that something will come in the 2.0 RTM?

[halter73]

@raffaeler Correct. I know it's not ideal. We started work on this feature, but we had to cut it due to some design questions (aspnet/Options#195). We don't want to release an API that we would have to break in the next version. We're now targeting the release of the feature for 2.1.

In the meantime, if you want to see how we were considering implementing it, so you can do something similar, you can look at #1878. You don't need to have a separate conf file just for Kestrel. I would recommend adding a section to appsettings.json.

[raffaeler]

I ended up writing this code to load the configuration from file:

The changes to Program.cs:

public static IWebHost BuildWebHost(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
        .UseKestrel(SetHost)
        .UseStartup<Startup>()
        .Build();

where SetHost is the following method ("RafHost" is the configuration section):

private static void SetHost(Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerOptions options)
{
    var configuration = (IConfiguration)options.ApplicationServices.GetService(typeof(IConfiguration));
    var host = configuration.GetSection("RafHost").Get<Host>();
    foreach (var endpointKvp in host.Endpoints)
    {
        var endpointName = endpointKvp.Key;
        var endpoint = endpointKvp.Value;
        if (!endpoint.IsEnabled)
        {
            continue;
        }

        var address = IPAddress.Parse(endpoint.Address);
        options.Listen(address, endpoint.Port, opt =>
        {
            if (endpoint.Certificate != null)
            {
                switch (endpoint.Certificate.Source)
                {
                    case "File":
                        opt.UseHttps(endpoint.Certificate.Path, endpoint.Certificate.Password);
                        break;
                    default:
                        throw new NotImplementedException($"The source {endpoint.Certificate.Source} is not yet implemented");
                }

                //opt.UseConnectionLogging();
            }
        });

        options.UseSystemd();   //?
    }
}

The configuration in appsettings.json is the following:

"RafHost": {
  "Endpoints": {
    "Http": {
      "IsEnabled":  true,
      "Address": "0.0.0.0",
      "Port": "5000"
    },

    "Https": {
      "IsEnabled": true,
      "Address": "0.0.0.0",
      "Port": "5443",
      "Certificate": {
        "Source": "File",
        "Path": "your-certificate.pfx",
        "Password": "(this should be a key to recover the password from a secure location)"
      }
    }
  }
}

and finally the configuration classes:

public class Host
{
    public Dictionary<string, Endpoint> Endpoints { get; set; }
}

public class Endpoint
{
    public bool IsEnabled { get; set; }
    public string Address { get; set; }
    public int Port { get; set; }
    public Certificate Certificate { get; set; }
}

public class Certificate
{
    public string Source { get; set; }
    public string Path { get; set; }
    public string Password { get; set; }
}

I hope this can help someone.