Skip to content

SslSessionsCache growth on Linux with MTLS using many client certificates #71401

@RobinsonWM

Description

@RobinsonWM

Description

If an application acting as a client creates many mutually-authenticated TLS connections to a server, with each connection using a different client certificate, the application's resident memory grows linearly.

If I run the same code on Windows and Linux, I only see the memory growth on Linux.

This cache appears to grow without bound:

private static readonly ConcurrentDictionary<SslCredKey, SafeCredentialReference> s_cachedCreds =

There are several IDisposables involved in generating a certificate and making an HTTP request, and I believe I am disposing all of them.

Reproduction Steps

https://gist.github.com/RobinsonWM/cae7244762173127d85f9b842cc3c1ba

The code in this gist reproduces the issue reliably for me. I was running it by starting a container, docker run mcr.microsoft.com/dotnet/sdk:6.0, copying in the code, and then running dotnet run.

The code starts an HTTPS server and then makes requests against it very quickly, each from a new HttpClient with a unique certificate. The test app periodically logs out the result of GC.GetTotalMemory(true) and SslSessionsCache.s_cachedCreds.Count, showing that both increase linearly on Linux but not on Windows.

This test application does muddy up the water a bit since it's running both the client and the server in the same process. I did that to make it easy to reproduce this issue. When I originally encountered this issue, it was in an application that was running only the client; the server was a separate process not written in .NET.

Expected behavior

Over time, if load is steady, the application's resident memory usage is also steady.

Actual behavior

The application works fine for a long time, but its resident memory usage grows linearly.

Regression?

I don't know if this is a regression

Known Workarounds

The same workaround for #65563 works for this issue, except for this issue it would need to be run on a timer.

private static void ClearCache()
{
    var sslAssembly = Assembly.GetAssembly(typeof(SslStream));
    var sslSessionCacheClass = sslAssembly.GetType("System.Net.Security.SslSessionsCache");
    var cachedCredsInfo = sslSessionCacheClass.GetField("s_cachedCreds", BindingFlags.NonPublic | BindingFlags.Static);
    var cachedCreds = cachedCredsInfo.GetValue(null);
    cachedCreds.GetType().GetMethod("Clear", BindingFlags.Public | BindingFlags.Instance).Invoke(cachedCreds, null);
}

Configuration

.NET 6.0 on Linux x64, with the Docker image mcr.microsoft.com/dotnet/runtime-deps:6.0
I have observed this issue on Linux x64. The same code run on Windows x64 works and does not exhibit the problem. I believe this issue is specific to Linux.

Other information

I realize this use case may sound contrived. I ran into this issue while developing a load-generation application used as part of a performance test. In order to simulate multiple users when generating load, I needed my HTTPS requests to use certificates generated on-the-fly, one for each user.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions