WebProxy.GetSystemProxy

[TonyValenti]

Background and motivation

When troubleshooting a live application, it would be nice to be able to open Fiddler and (nearly) immediately see HTTP requests flowing through it.

Unfortunately, the current HTTPClient / HttpClient.DefaultProxy is not suitable for this. The first time it is called it seems to capture the value from SystemProxyInfo.ConstructSystemProxy and cache is for the life of the application. This results in needing reflection or other methods in order to retrieve the actualy current system proxy information.

API Proposal

namespace System.Net.Http;

public static class WebProxy
{
    public static IWebProxy GetSystemProxy();
}

API Usage

// Fancy the value
var Proxy = System.Net.Http.WebProxy.GetSystemProxy();

Alternative Designs

Leave as-is.

Risks

I assume that SystemProxyInfo.ConstructSystemProxy is a heavy method. Users of this function might call it too frequently.

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

When troubleshooting a live application, it would be nice to be able to open Fiddler and (nearly) immediately see HTTP requests flowing through it.

Unfortunately, the current HTTPClient / HttpClient.DefaultProxy is not suitable for this. The first time it is called it seems to capture the value from SystemProxyInfo.ConstructSystemProxy and cache is for the life of the application. This results in needing reflection or other methods in order to retrieve the actualy current system proxy information.

API Proposal

namespace System.Net.Http;

public class WebProxy
{
    public IWebProxy GetSystemProxy();
}

API Usage

// Fancy the value
var Proxy = System.Net.Http.WebProxy.GetSystemProxy();

Alternative Designs

Leave as-is.

Risks

I assume that SystemProxyInfo.ConstructSystemProxy is a heavy method. Users of this function might call it too frequently.

Author:	TonyValenti
Assignees:	-
Labels:	api-suggestion, area-System.Net.Http
Milestone:	-

[ManickaP]

Just to be sure, you just want us to expose an existing, private method that creates a proxy based on environmental settings, so that you can manually set it when creating HttpClient? Because the default property is process wide, never refreshed static.
If so, this seems reasonable, we can always document the function as heavy and leave up to the user to decide.

@wfurt any concerns here? BTW, why don't we reconstruct the proxy with HttpClient creation? I assume we don't expect the proxy settings to change?

Also tangentially related #35992

[TonyValenti]

@ManickaP - Yes. This is exactly what I want and the reason I want it.

If this was implemented, it would also be nice if there was an implementation of IWebProxy that would "follow" changes in the system web proxy.

In my own code, I have rolled the following implementations but I imagine these are common asks that many folks could leverage.

Here is my code:

namespace System.Net.Http {

    //Code to call the current SystemProxyInfo method via reflection
    public static class WebProxyFactory {
        static WebProxyFactory() {
            {
                var TypeName = "System.Net.Http.SystemProxyInfo";
                var Type = typeof(HttpClient).Assembly.GetType(TypeName);
                var MethodName = "ConstructSystemProxy";
                var Method = Type?.GetMethod(MethodName);
                var Delegate = Method?.CreateDelegate<Func<IWebProxy>>();

                if (Delegate is { }) {
                    Factory = Delegate;
                } else {
                    throw new MissingMethodException("Unable to find proxy functions");
                }
            }
            SystemProxy = new SystemWebProxy(TimeSpans.OneSecond);
        }


        private static Func<IWebProxy> Factory { get; }
        public static IWebProxy GetCurrentSystemProxy() {
            return Factory();
        }

        public static IWebProxy SystemProxy { get; }

    }

//An IWebProxy that will re-retrieve the proxy from the system every so often.
public class SystemWebProxy : DisplayClass, IWebProxy {
    public TimeSpan Timeout { get; }
    public DateTimeOffset ValidUntil { get; private set; }

    public override DisplayBuilder GetDebuggerDisplayBuilder(DisplayBuilder Builder) {
        return base.GetDebuggerDisplayBuilder(Builder)
            .Data.AddExpression(ValidUntil)
            ;
    }

    public SystemWebProxy(TimeSpan Timeout) {
        this.Timeout = Timeout;

        GetCurrentCache = GetCurrent();
    }

    private readonly object GetCurrentLock = new();
    private volatile IWebProxy GetCurrentCache;
    protected IWebProxy GetCurrent() {
        var ret = GetCurrentCache;

        if (ValidUntil < DateTimeOffset.Now) {
            lock (GetCurrentLock) {
                if (ValidUntil < DateTimeOffset.Now) {
                    ret = WebProxyFactory.GetCurrentSystemProxy();
                    GetCurrentCache = ret;
                    ValidUntil = DateTimeOffset.Now + Timeout;
                }
            }
        }
        return ret;

    }



    public ICredentials? Credentials { get => GetCurrent().Credentials; set => GetCurrent().Credentials = value; }

    public Uri? GetProxy(Uri destination) {
        return GetCurrent().GetProxy(destination);
    }

    public bool IsBypassed(Uri host) {
        return GetCurrent().IsBypassed(host);
    }
}

}

[karelz]

I think it is dupe of #46910

[wfurt]

I think the devil is in the detail @ManickaP. I don't think there is notification about changes. And in broader sense when auto configuration is done via script we don't even have visibility to what server shall be used. That can be completely custom logic and only one way to know is to execute javascript. (Windows only AFAIK)

[ManickaP]

I meant, whether we could re-call SystemProxyInfo.ConstructSystemProxy when we are setting up _proxy on the pool manager here: https://source.dot.net/#System.Net.Http/System/Net/Http/SocketsHttpHandler/HttpConnectionPoolManager.cs,131

What we do now is that we check the system settings once per the whole process, store the result in static and use it for the whole application lifetime.

I'm aware that automatically checking for system changes is nigh impossible.

[wfurt]

There are multiple threads/reasons and I think we should differentiate.

How is this API proposal different from WebRequest.GetSystemWebProxy? Both static methods giving back IWebProxy. If that one does not return fresh proxy value we can make it so IMHO without adding new API surface.

I would be supportive to query the proxy more often @ManickaP . We can do it when HttpClient is created. That is still difficult to use IMHO but I would see it as easy improvement.

I think we should do it on NetworkChange in 9.0. I certainly move my macBook around and I don't want to restart VS when connected to different network with different proxy. Reading it once on app start ignores current mobility IMHO.

I think we should investigate #46910 and have proper fix. The claim there is that Framework worked so we should be able to figure it out. I'm not sure about macOS but that seems less pressing at the moment.

I feel #35992 is different animal. While related, I don't think it is trying to make better experience with system setting.

[ManickaP]

WebRequest.GetSystemWebProxy does not re-fetch, it returns the static, once per-process initialized static field (that was the crux of #46910 - the original complaint).

I'll mark this for further triage and we can talk about the whole topic as a team.