Add Route Short Circuit middleware

[Tratcher]

Background and Motivation

Browsers and bots often probe servers for well known paths like favicon.ico. Normally the middleware pipeline runs to completion before rejecting a request as 404.

Services can reduce resource usage and log noise by filtering out known requests early in the pipeline. We can provide a middleware to do this efficiently.

Proposed API

Project/Assembly: Microsoft.AspNetCore.Routing

namespace Microsoft.Extensions.DependencyInjection;

+ public static class RouteShortCircuitIServiceCollectionExtensions
{
+     public static IServiceCollection AddRouteShortCircuit(this IServiceCollection services, Action<RouteShortCircuitOptions> configure) { }
}

namespace Microsoft.AspNetCore.Builder;

+ public static class RouteShortCircuitIApplicationBuilderExtensions
+ {
+     public static IApplicationBuilder UseRouteShortCircuit(this IApplicationBuilder builder) { }
+ }

namespace Microsoft.AspNetCore.Routing;

+ public class RouteShortCircuitOptions
+ {
+     // Exact path matches, empty by default
+     public ISet<string> Paths { get; } = new HashSet<string>(StringComparer.OridinalIgnoreCase);
+     // Any path that starts with these, empty by default
+     public ISet<string> PathsStartWith { get; } = new HashSet<string>(StringComparer.OridinalIgnoreCase);
+     public RequestDelegate OnRejected { get; set; }
+ }

Usage Examples

services.AddRouteShortCircuit(options =>
{
    options.Paths.Add("/favicon.ico");
    options.OnRejected(context => { /* metrics */ });
});
...
app.UseRouteShortCircuit();

Alternative:

Build this into UseRouting (#43642) and have it terminate immediately with a 404 if such an endpoint is matched.

We could also allow short circuiting to specify an endpoint/delegate to execute to produce a custom response.

Project/Assembly: Microsoft.AspNetCore.Routing

namespace Microsoft.Extensions.DependencyInjection;

+ public static class RouteShortCircuitEndpointConventionBuilderExtensions
{
+     public static IEndpointConventionBuilder ShortCircuit(this IEndpointConventionBuilder builder) { }
}

Thanks for contacting us.

We're moving this issue to the .NET 8 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

• The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
• The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
• Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

[martincostello]

Could this be extended to support configuring file extensions too?

I'm thinking of a use case where you get bots probing for PHP stuff and WordPress sites etc., and you could just blanket short-circuit all requests for *.php in addition to paths that start with /wp/(?).

[davidfowl]

I wonder if this should be built on top of routing instead. This middleware would the short circuit if the appropriate endpoint matched with specific metadata. Another ref count for #43642.

Strawman would look like;

app.MapMetadata("/{path}.php").ShortCircuit(); // Add the short circuit metadata to this route pattern

// This middleware short circuits the request requests
app.UseRouteShortCircuit();

[Tratcher]

Why even have UseRouteShortCircuit? Why not have UseRouting do it?

The original intent was to avoid the costs of static files lookups, route tables, logging & telemetry, etc.. Building into the route table limits the effectiveness of the feature, though it does make match definitions easier.

[davidfowl]

Agreed @Tratcher! Something we can measure though, right?

[tekian]

@davidfowl Routing comes later in the pipeline, right? After authentication, throttling, telemetry, etc. Ideally, we'd want to zero the costs for these requests. We don't need to attempt to authenticate them, we don't need to consider throttling them, we are not interested in capturing telemetry for them. Short circuiting as soon as the requests hits the middleware pipeline does that.

[Tratcher]

@tekian these days we're putting routing almost first, second only to exception handlers. Auth and throttling come later because they want to consider which endpoint the request was routed to.

Telemetry is custom middleware in this case? That goes wherever you put it.

[tekian]

@Tratcher I'm not aware of a change that would put routing almost first. Can you share more information on this? Usually what I have seen is that services assemble route-agnostic middlewares in-between exception handling and routing. That way, they can apply shared policies such as validate certificate issuer of a remote party validate a request has a token or apply per-IP throttling policy. For that, we don't need to know an endpoint or a user associated with the request. And then, per-endpoint or per-user, we may choose to apply additional authorization or throttling as filters. Short circuit was intended to stop even shared policies from applying.

[Tratcher]

When you use the new WebApplicationBuilder class it automatically adds UseDeveloperExceptionPage and UseRouting at the start of the pipeline if you did not specify them.

aspnetcore/src/DefaultBuilder/src/WebApplicationBuilder.cs

Lines 148 to 165 in 83d6c56

	app.UseDeveloperExceptionPage();
	}
	// Wrap the entire destination pipeline in UseRouting() and UseEndpoints(), essentially:
	// destination.UseRouting()
	// destination.Run(source)
	// destination.UseEndpoints()
	// Set the route builder so that UseRouting will use the WebApplication as the IEndpointRouteBuilder for route matching
	app.Properties.Add(WebApplication.GlobalEndpointRouteBuilderKey, _builtApplication);
	// Only call UseRouting() if there are endpoints configured and UseRouting() wasn't called on the global route builder already
	if (_builtApplication.DataSources.Count > 0)
	{
	// If this is set, someone called UseRouting() when a global route builder was already set
	if (!_builtApplication.Properties.TryGetValue(EndpointRouteBuilderKey, out var localRouteBuilder))
	{
	app.UseRouting();

I understand wanting to minimize the costs. We should see what the cost of moving routing up front would be for apps like yours to see if this approach makes sense.

[davidfowl]

@tekian That makes sense, and it sounds like avoiding those things would still happen.