Runtime developer can run Networking tests without Loopback server capability (on mobile/browser platforms)

[lewing]

Some platforms don't support listening to incoming requests but do support making outgoing requests. Most of the existing http functional tests expect to be able to start up a loopback server to respond to incoming request which means the tests cannot run on the more restricted platforms.

---

Plan:

• Make xharness extensible by loading kestrel middlewares - [wasm][testing] hosting webSocket echo server in xharness process xharness#593
• Bring outer loop tests into inner loop via xharness on Mono platforms
  • Move existing kestrel echo server into runtime repo
    • this PR is copy of davidsh's corefx-net-endpoint, with minor improvements Refresh network tests server code to Kestrel #52642
    • this server/code should be stay owned by NCL team after merge
    • [out of scope] there are known gaps as compared to obsolete CoreFxNetCloudService as documented in Gaps in kestrel version of networking test echo server (as compared to obsolete CoreFxNetCloudService) #52693
  • move [OuterLoop("Uses external servers")] tests from outer loop on Mono platforms
    • conditional [OuterLoop] by platform - implemented conditional OuterLoopAttribute arcade#7370
    • enable network tests on mono Outerloop for browser [wasm] Enable more networking tests #53027
    • conditional OuterLoop for browser [wasm][testing] hosting echo server in xharness process #52923
    • enable more tests by avoiding crypto [wasm][testing] Avoid using crypto in WASM HTTP unit tests #53925
    • [TODO]: 199x add condition [OuterLoop("Uses external servers", TestPlatforms.Android)]
    • [Consider]: 199x add condition [OuterLoop("Uses external servers", TestPlatforms.iOS)]
      • this will require TCP tunnel via USB
    • For helix on windows we will need to send windows chromeDriver as part the helix payload.
• Bring LoopbackServer tests to Mobile platforms
  • Hosting it in Xharness via middleware binary assembly
  • This remote LoopbackServer code should not go into their Azure server above.
    • per agreement with @karelz. They don't want to expose loopback to public internet.
    • therefore this will be separate project and kestrel middleware for xharness.
  • Enable WASM tests [WIP][wasm][testing] remote loop network tests via xharness and websocket #54289
  • [Consider]: Enable LoopbackServer - Socket unit tests on Android, iOS
    • on iOS this will require TCP tunnel via USB
  • [Consider]: Enable LoopbackServer - Certificate unit tests on Android, iOS

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[akoeplinger]

Some of the http cert and websocket tests use an echo server hosted in Azure, presumably we could do the same for the basic http scenarios:

runtime/src/libraries/Common/tests/System/Net/Configuration.Http.cs

Line 44 in 6072e4d

public static string EchoClientCertificateRemoteServer => GetValue("DOTNET_TEST_HTTPHOST_ECHOCLIENTCERT", "https://corefx-net-tls.azurewebsites.net/EchoClientCertificate.ashx");

[wfurt]

If the tests can run under platform emulator using containers may be better - like we do for stress and enterprise authentication. Dependency on external servers was problematic in the past and all the tests using them are marked as Outerloop.

[stephentoub]

Some of the http cert and websocket tests use an echo server hosted in Azure

Some, yes. Two problems:

1. Going off-box results in significantly more instability of the tests, which is why most of our networking tests that do so have been relegated to outerloop, and why we've pushed to move any tests that don't require going remote to not do so. Even so, there have been discussions in the past about making it configurable.
2. Lots of test use shared in-memory state to coordinate between the client and server, in some cases just as a way to pass data back and forth, but in other cases actually synchronizing execution to ensure proper ordering for the what the test is validating. That's much harder (if not impossible) to do well with a remote server.

[wfurt]

for the state, can we use the new connection hook and use transport other than TCP socket? Pipe, memory stream or something else?

[halter73]

I'm not sure what these tests are using for the server-side, but I've started using the new SocketsHttpHandler connection hook and an in-memory stream (based on System.IO.Pipelines) in some Kestrel tests.

https://github.com/dotnet/aspnetcore/blob/925926c6156e56c6414ad004e5f1e54ced8c29c4/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2EndToEndTests.cs#L49-L61

[stephentoub]

Yes, we could use the connection callback for SocketsHttpHandler to use something other than sockets for testing... but that would only apply to SocketsHttpHandler. It wouldn't address the browser-based handler that is currently the crux of @lewing's issue. If we were to expose similar callbacks on HttpClientHandler (which wraps the browser-based handler), then maybe we could, but I don't know if the browser-based handler would even be able to support it... I suspect not... but maybe in that case supplying such a connection callback would actually opt you out of the browser-based handler? I'm waving my hands.

[lewing]

We've resolved some other blocking issues for testing the networking side in CI for Browser but we still need some way to test actual network requests even if they are going to a local endpoint.