Use forTls parameter to enable TLS-compatible certificate key types while maintaining platform compatibility

[Copilot]

This change addresses the performance vs security tradeoff in dynamically generated test certificates by using a runtime parameter approach to select appropriate key types and sizes for TLS vs crypto test contexts while ensuring platform compatibility.

Problem

The shared CertificateAuthority class was using both:

1. A hardcoded 1024-bit RSA key size for all test certificates
2. Explicit RSA key factory forcing in Configuration.Certificates.Dynamic.cs

While the hardcoded size provided fast test execution for cryptography tests, it was too small for networking tests where larger key sizes are needed for TLS compatibility. Additionally, the explicit key factory prevented the use of other key types (ECDsa, MLDsa, SlhDsa) that the test infrastructure supports. Furthermore, Windows S/Channel doesn't support ML-DSA certificates yet, creating platform compatibility issues.

Solution

Implemented a forTls parameter approach that differentiates between test contexts while enabling appropriate key type selection based on platform capabilities:

• TLS tests (forTls: true): Use 2048-bit RSA keys and platform-appropriate key types
  • Windows: Only RSA (2048-bit) and ECDsa (excludes ML-DSA due to S/Channel incompatibility)
  • Non-Windows: RSA (2048-bit), ECDsa, MLDsa, and SlhDsa
• Crypto tests (default): Use 1024-bit RSA keys for speed with full key type support on all platforms
• All tests: Maintain deterministic key type selection based on hash of test parameters

Changes

• Added bool forTls = false parameter to CertificateAuthority.BuildPrivatePki methods
• Created s_tlsVariantKeyFactories static field and BuildTlsVariantFactories() method
• Implemented platform-specific key type filtering in BuildTlsVariantFactories:
  • Windows: RSASize(2048) and ECDsa only
  • Non-Windows: RSASize(2048), ECDsa, MLDsa, and SlhDsa
• Updated certificate generation logic to use appropriate key factories based on forTls parameter
• Modified Configuration.Certificates.Dynamic.cs to use forTls: true for networking certificate generation
• Removed NETWORKING_TESTS preprocessor directive approach
• Reverted all .csproj changes as they are no longer needed

Implementation

The runtime selection logic uses the forTls parameter to choose between key factory arrays:

KeyFactory[] keyFactories = forTls ? s_tlsVariantKeyFactories : s_variantKeyFactories;
keyFactory = keyFactories[hash[0] % keyFactories.Length];

The BuildTlsVariantFactories method ensures platform compatibility:

// On Windows: exclude ML-DSA due to S/Channel incompatibility
// On other platforms: include all supported key types with 2048-bit RSA

This approach provides optimal performance for crypto tests, secure and compatible networking test certificates, and broader key type coverage while respecting platform limitations.

Addresses #116365.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[rzikm]

@copilot instead of CRYPTO_TESTS directive, use NETWORKING_TESTS, and set it from relevant networking projects. Comment that increased size of the RSA key in networking test is required as some TLS implementations don't work with weaker keys.

Also, there is no need to add tests.

[Copilot]

@copilot instead of CRYPTO_TESTS directive, use NETWORKING_TESTS, and set it from relevant networking projects. Comment that increased size of the RSA key in networking test is required as some TLS im...

Switched to NETWORKING_TESTS directive instead of CRYPTO_TESTS as requested. Added the directive to System.Net.Security.Tests.csproj and removed it from System.Security.Cryptography.Tests.csproj. Updated the comment to explain that networking tests require larger RSA keys as some TLS implementations don't work with weaker keys. Test files have been removed as requested. Changes made in b6b5db8.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• http://www.microsoft.com:80/pki/certs/MicCodSigPCA_08-31-2010.crt
  • Triggering command: /home/REDACTED/work/runtime/runtime/artifacts/bin/testhost/net10.0-linux-Debug-x64/dotnet exec --runtimeconfig System.Security.Cryptography.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.Tests.deps.json /home/REDACTED/.nuget/packages/microsoft.dotnet.xunitconsoleREDACTED/2.9.3-beta.25420.121/build/../tools/net/xunit.console.dll System.Security.Cryptography.Tests.dll -xml testResults.xml -nologo -notrait category=OuterLoop -notrait category=failing (http block)
• http://www.microsoft.com:80/pki/mscorp/Microsoft%20RSA%20TLS%20CA%2001.crt
  • Triggering command: /home/REDACTED/work/runtime/runtime/artifacts/bin/testhost/net10.0-linux-Debug-x64/dotnet exec --runtimeconfig System.Security.Cryptography.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.Tests.deps.json /home/REDACTED/.nuget/packages/microsoft.dotnet.xunitconsoleREDACTED/2.9.3-beta.25420.121/build/../tools/net/xunit.console.dll System.Security.Cryptography.Tests.dll -xml testResults.xml -nologo -notrait category=OuterLoop -notrait category=failing (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[rzikm]

@copilot

You also need to remove the explicit keyfactory so that random key type is being used in tests.

runtime/src/libraries/Common/tests/System/Net/Configuration.Certificates.Dynamic.cs

Line 185 in 44a6dd2

keyFactory: CertificateAuthority.KeyFactory.RSASize(keySize),

[Copilot]

@copilot

You also need to remove the explicit keyfactory so that random key type is being used in tests.
...

Removed the explicit keyFactory parameter from BuildPrivatePki call. Now uses the default random key type selection which respects the NETWORKING_TESTS conditional compilation for RSA key sizes.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• http://www.microsoft.com:80/pki/mscorp/Microsoft%20RSA%20TLS%20CA%2001.crt
  • Triggering command: /home/REDACTED/work/runtime/runtime/artifacts/bin/testhost/net10.0-linux-Debug-x64/dotnet exec --runtimeconfig /home/REDACTED/work/runtime/runtime/artifacts/bin/System.Security.Cryptography.Tests/Debug/net10.0-unix/System.Security.Cryptography.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/runtime/runtime/artifacts/bin/System.Security.Cryptography.Tests/Debug/net10.0-unix/System.Security.Cryptography.Tests.deps.json /home/REDACTED/work/runtime/runtime/artifacts/bin/System.Security.Cryptography.Tests/Debug/net10.0-unix/testhost.dll --port 33245 --endpoint 127.0.0.1:033245 --role client --parentprocessid 43134 --datacollectionport 41421 --telemetryoptedin false (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[rzikm]

@copilot you need to pass the NETWORKING_TESTS directive to all networking tests that use CertificateAuthority.cs. You are missing Http, Mail and System.Net.Security.Unit.Tests.

[Copilot]

@copilot you need to pass the NETWORKING_TESTS directive to all networking tests that use CertificateAuthority.cs. You are missing Http, Mail and System.Net.Security.Unit.Tests.

Added the NETWORKING_TESTS directive to the missing networking test projects: System.Net.Http.Functional.Tests, System.Net.Mail.Functional.Tests, and System.Net.Security.Unit.Tests in 45a2b47.

[Copilot]

Pull Request Overview

This change addresses performance vs security tradeoffs in test certificate generation by introducing a runtime parameter approach to select appropriate key types and sizes for different test contexts. The main goal is to enable TLS-compatible certificate key types while maintaining platform compatibility.

• Adds a forTls parameter to certificate generation methods to differentiate between TLS and crypto test contexts
• Implements platform-specific key type filtering to exclude ML-DSA on Windows due to S/Channel incompatibility
• Uses 2048-bit RSA keys for TLS tests and maintains 1024-bit for crypto tests for performance

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
CertificateAuthority.cs	Adds forTls parameter and BuildTlsVariantFactories() method for platform-aware key type selection
Configuration.Certificates.Dynamic.cs	Updates networking certificate generation to use forTls: true instead of hardcoded RSA key size

[bartonjs]

@rzikm There are untagged failures here. Since this is actually changing how some of these tests work it might be related. Can you take a look? (Authentication failed because the remote party sent a TLS alert: 'UserCanceled' doesn't quite sound like the right sort of problem to be created by this, but I'm not familiar enough with SslStream errors to say)

[rzikm]

---- System.Security.Authentication.AuthenticationException : Authentication failed, see inner exception.
-------- Interop+OpenSsl+SslException : Using SSL certificate failed with OpenSSL error - unknown certificate type.

This is weird, seems like Alpine and CentOS only are affected

I will find out which cert type is that

[rzikm]

    [Signature Algorithm]
      SLH-DSA-SHA2-128f(2.16.840.1.101.3.4.3.21)
    
    [Public Key]
      Algorithm: SLH-DSA-SHA2-128f
      Length: 
      Key Blob: 1C2B1FCE46F23F2E85CA780C9D452A12B2E03B4F523B803C6DFE19ECB49267EC
      Parameters: 

[helixbot@e84af088e33d /]$ openssl version
OpenSSL 3.5.1 1 Jul 2025 (Library: OpenSSL 3.5.1 1 Jul 2025)

Seems like issue with the new PQC certs

[rzikm]

Got it, the OpenSSL provider does not advertise support for SLH-DSA-* family in its TLS-SIGALG capability, only ML-DSA

https://github.com/openssl/openssl/blob/0c679f556669e32499a827a081afe3bcf973c9ad/providers/common/capabilities.c#L315-L320

I think we need to wait with testing these cert in TLS until they are advertised by the common OpenSSL provider. in the meanwhile, we can keep ML-DSA.

[rzikm]

Disabled SLH-DSA for linux for now, Opened #119573 to track that (and so that the issue is discoverable if anyone hits it)

[rzikm]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[bartonjs]

Looks like maybe QUIC on Alpine/CentOS doesn't agree that ML-DSA works?

[rzikm]

Right, the libssl linked to MsQuic does not support even ML-DSA as the fork is pre-3.5.0 OpenSSL. This is rather unfortunate, because now we are in a territory where ML-DSA certs work with HTTP/1.1 and HTTP/2, but will fail with HTTP/3.... MsQuic recently did some effort to ditch the fork and use (dynamically-linked?) OpenSSL 3.5.0 but I don't know if that made into any releases, or whether it would work with .NET runtime, cc @ManickaP.

[ManickaP]

It's in the main only and there is an open issue against it: microsoft/msquic#5210 - statically linking, including crypto - so even if it got released, we cannot consume it.

[rzikm]

/ba-g Test failure in System.Numerics.Tensors.Tests is unrelated

[rzikm]

@bartonjs Thanks a lot for helping with this.