Iron Port TLS 1.0 proxy, pub get fails

[crodier]

I am behind an *_Iron Port firewall *_in the office, on Windows. Could be me, but I found and tried 'https_proxy' and 'http_proxy', and versions of the Dart SDK from 1.12 to 1.15.0 Dev today, including 1.12, 1.13., 1.14, and 1.15.0 Dev, from 1/28/2016 (latest.)

I am trying to run the Dart, Angular2, getting started. Only getting packages from pub.dartlang.org.

On 15.0.0 from Dev today, the error is 407, Proxy Authentication Required..

The pub get trace logs are attached.

pub-get-proxy.zip

If I set https_proxy, get the same issue, I can set both without or with my password to the Iron Port proxy address. like https_proxy=iport:80, https_proxy=crodier:password@iport:80. Also with and without http_proxy, which has no impact on Dev.

The interesting bits:

• npm works from behind the proxy, without any configuration
• Maven and other http fetching tools, are ok behind the proxy, without any configuration (usually)
• git has issues, not with git clone git protocol, but git https protocol
• Between my windows machine and the Iron Port firewall, _TLS 1.0_ is used.
• My guess: TLS 1.0 between me and the proxy is no longer supported by BoringSSL, or needs flags to be allowed.
• In our case, TLS 1.0 is only between my machine and the Iron Port, on a closed network, which is not great, but not in the wild
• The certificate used is the Iron Port certificate, for my organization, which has been given trust by root authorities.
• Windows 7 Professional

I reviewed the alternative, manually downloading the packages, but not being able to run pub get is a major deterrent.

While I could investigate with the networking team, I doubt this leads to any changes in the proxy setup in my organization. It is also difficult to justify that investigation, with other tools working ok, and the proxy generally working, and being a vendor (Iron Port) firewall.

Dart looks to be an incredible platform, and I post in the hopes this is helpful. I also apologize, but I doubt I will be able to learn the code and work on the issue my self in the SDK etc. I can offer to re-test if there are changes to Dev, With guidance, I may be able to work on the sdk issue itself.

Best regards,
Chris

[kevmoo]

@nex3 would you look at this quick? I believe you've dealt w/ firewall issues before...

[nex3]

To be honest, all I know about proxy issues is that they're the domain of dart:io. Pub doesn't do anything explicit with them at all.

[kevmoo]

@whesse is there anything about the new BoringSSL impl that could cause this?

[whesse]

If 1.12 is also failing in the same way, then the problem is not BoringSSL. I don't really know much about proxies. Someone else should look at this, and ask the right questions to the bug reporter.
@sgjesse @mkustermann

This does need to be addressed, so I think the pub team should follow up on this, and find someone who can figure this out.

[mkustermann]

Based on the pub get --trace logs I would assume that dart:ios HttpClient found environment variables which tell which proxy to use and then that proxy requires authentication but no authentication handler was set.

@crodier Could you unset all the environment variables described here to make pub use direct connections and try again? At least that documentation mentions that a username:password@hostname:port value should in theory work.

dart:io's HttpClient does have support for this via the HttpClient.authenticateProxy setter. I assume the pub client doesn't use this functionality so the request fails.

The functionality for finding a proxy or disabling the use of proxies can be configured via HttpClient.findProxy. I also assume that pub doesn't disable proxies but rather let dart:io use it's defaults.

@sgjesse Maybe you have more ideas?

[crodier]

Thanks for looking everyone. I did attach both the with https_proxy
environment variable set and unset, as two zip files. I believe it logs
the value in the one where set, and not in the other. Same result in both,
on the latest 1.15 dev build.

Maven (java) had s similar SSL issue, where adding the iron port
certificate to the trust store resolved the problem. Forcing java to trust
the iron port certificate. Could be because our iron port cert resolves to
a different root authority than java is aware of. That's my next guess,
because I agree that if it doesn't work in 1.12, than it should not be
BoringSSL related, but something that would fail across dart.io versions.

If this theory is not easily testable, I can try to build and debug into
the sdk.

On Tue, Feb 2, 2016, 1:15 PM Martin Kustermann [email protected]
wrote:

Based on the pub get --trace logs I would assume that dart:ios HttpClient
found environment variables which tell which proxy to use and then that
proxy requires authentication but no authentication handler was set.

@crodier https://github.com/crodier Could you unset all the environment
variables described here
https://api.dartlang.org/1.14.0/dart-io/HttpClient/findProxyFromEnvironment.html
to make pub use direct connections and try again?

dart:io's HttpClient does have support for this via the
HttpClient.authenticateProxy
https://api.dartlang.org/1.14.0/dart-io/HttpClient/authenticateProxy.html
setter. I assume the pub client doesn't use this functionality so the
request fails.

The functionality for finding a proxy or disabling the use of proxies can
be configured via HttpClient.findProxy
https://api.dartlang.org/1.14.0/dart-io/HttpClient/findProxy.html. I
also assume that pub doesn't disable proxies but rather let dart:io use
it's defaults.

@sgjesse https://github.com/sgjesse Maybe you have more ideas?

—
Reply to this email directly or view it on GitHub
#25615 (comment).

[sgjesse]

@crodier Have you tried to run a simple dart:io program accessing an https URL? E.g.

import 'dart:io';

main() async {
  HttpClient client = new HttpClient();
  var request = await client.getUrl(Uri.parse("https://www.google.com/"));
  var response = await request.close();
  print(response.headers);
  client.close(force: true);
}

I assume that will give the same error?

Do you know what type of authentication the proxy uses? dart:io support basic and digest.

Regarding the certificate, does the proxy server require an HTTPS connection? If so we are out of luck, as that is not supported. The proxy configuration we support are PROXY and DIRECT as described in https://en.wikipedia.org/wiki/Proxy_auto-config (the string returned from findProxy use that format (we don't handle PAC files - just use the same string format returned from the PAC JavaScript code) and there is no way of specifying HTTPS.

[crodier]

Thanks. Tried it, this uncovers the actual issie!

Connecting to https google.com, as in the example code.

Error: CERTIFICATE_VERIFY_FAILED.

The proxy is https. It issues certificates, proxied, for everything we
access internally, and has been trusted by the root authority.

This puts the remote certificate (pub.dart) in the certificate chain,
beneath ours. It is a transparent proxy, web wise.

Our certificate is valid. The connection is tls 1.0. I am looking now for
the client code to tell dart http:// client to:

• trust our certificate
• trust tls 1.0

If you can update the example I can try it. I expect that fixes it. If it
does, then a command.line flag would be necessary for pub get to do the
same for pub commands.

In maven, this was done by importing our certificate into a trust store,
and then telling java to use that trust store. Similar problem, I believe
need the same for dart pub command. Either npm ignores cer issues or has a
workaround for tls 1.0 amd certificate chains that works for.our Iron Port
vendor proxy. This is a huge vendor and typical in finance, captures all
internal network activity for storage and review.

On Wed, Feb 3, 2016, 11:59 AM Søren Gjesse [email protected] wrote:

@crodier https://github.com/crodier Have you tried to run a simple
dart:io program accessing an https URL? E.g.

import 'dart:io';

main() async {
HttpClient client = new HttpClient();
var request = await client.getUrl(Uri.parse("https://www.google.com/"));
var response = await request.close();
print(response.headers);
client.close(force: true);
}

I assume that will give the same error?

Do you know what type of authentication the proxy uses? dart:io support
basic and digest.

Regarding the certificate, does the proxy server require an HTTPS
connection? If so we are out of luck, as that is not supported. The proxy
configuration we support are PROXY and DIRECT as described in
https://en.wikipedia.org/wiki/Proxy_auto-config (the string returned from
findProxy
https://api.dartlang.org/1.14.0/dart-io/HttpClient/findProxy.html use
that format (we don't handle PAC files - just use the same string
format returned from the PAC JavaScript code) and there is no way of
specifying HTTPS.

—
Reply to this email directly or view it on GitHub
#25615 (comment).

[crodier]

I added our iron port PEM to the SecurityContext, and connected to https
google.com, in the example.

Now question becomes primarily, how to tell pub get to do the same?

Follow up is why it must be added when we are trusted by a major root
authority.

Npm works, as does the web. I don't think the certificate chain is being
honored or similar.

On Wed, Feb 3, 2016, 6:08 PM Chris Rodier [email protected] wrote:

Thanks. Tried it, this uncovers the actual issie!

Connecting to https google.com, as in the example code.

Error: CERTIFICATE_VERIFY_FAILED.

The proxy is https. It issues certificates, proxied, for everything we
access internally, and has been trusted by the root authority.

This puts the remote certificate (pub.dart) in the certificate chain,
beneath ours. It is a transparent proxy, web wise.

Our certificate is valid. The connection is tls 1.0. I am looking now
for the client code to tell dart http:// client to:

• trust our certificate
• trust tls 1.0

If you can update the example I can try it. I expect that fixes it. If
it does, then a command.line flag would be necessary for pub get to do the
same for pub commands.

In maven, this was done by importing our certificate into a trust store,
and then telling java to use that trust store. Similar problem, I believe
need the same for dart pub command. Either npm ignores cer issues or has a
workaround for tls 1.0 amd certificate chains that works for.our Iron Port
vendor proxy. This is a huge vendor and typical in finance, captures all
internal network activity for storage and review.

On Wed, Feb 3, 2016, 11:59 AM Søren Gjesse [email protected]
wrote:

@crodier https://github.com/crodier Have you tried to run a simple
dart:io program accessing an https URL? E.g.

import 'dart:io';

main() async {
HttpClient client = new HttpClient();
var request = await client.getUrl(Uri.parse("https://www.google.com/"));
var response = await request.close();
print(response.headers);
client.close(force: true);
}

I assume that will give the same error?

Do you know what type of authentication the proxy uses? dart:io support
basic and digest.

Regarding the certificate, does the proxy server require an HTTPS
connection? If so we are out of luck, as that is not supported. The proxy
configuration we support are PROXY and DIRECT as described in
https://en.wikipedia.org/wiki/Proxy_auto-config (the string returned
from findProxy
https://api.dartlang.org/1.14.0/dart-io/HttpClient/findProxy.html use
that format (we don't handle PAC files - just use the same string
format returned from the PAC JavaScript code) and there is no way of
specifying HTTPS.

—
Reply to this email directly or view it on GitHub
#25615 (comment).

[sgjesse]

@crodier The default root certificates used by the Dart VM is build-in, and is pulled from a separate repository (https://github.com/dart-lang/root_certificates/blob/master/certdata.pem), and you CA depend on is obviously not in that list.

Regarding pub there are currently no (afaik @munificent or @nex3 correct me if I am wrong) options to specify root certificates or to allow untrusted certificates. So changes will be needed to either pub or dart:io.

There is work in progress on using the root certificates installed on the machine instead of the build-in set, but it will take some time before that lands.

[crodier]

Thanks for helping me!

I agree. I need to say I think both are necessary, and will br prohibitive
to any would be corporate Dart users, or many. Maybe enough to warrant
priority.

I will look into who our root authority is. How do I check that?

On Thu, Feb 4, 2016, 10:28 AM Søren Gjesse [email protected] wrote:

@crodier https://github.com/crodier The default root certificates used
by the Dart VM is build-in, and is pulled from a separate repository (
https://github.com/dart-lang/root_certificates/blob/master/certdata.pem),
and you CA depend on is obviously not in that list.

Regarding pub there are currently no (afaik @munificent
https://github.com/munificent or @nex3 https://github.com/nex3
correct me if I am wrong) options to specify root certificates or to allow
untrusted certificates. So changes will be needed to either pub or dart:io
.

There is work in progress on using the root certificates installed on the
machine instead of the build-in set, but it will take some time before that
lands.

—
Reply to this email directly or view it on GitHub
#25615 (comment).

[sgjesse]

In Chrome you can right-click on the green lock icon for the HTTPS connection, then click on the "Connection" tab and click "Certificate information". The click the "Details" tab which should have the "Certificate Hierarchy" at the top.

[andregs]

On npm we can set strict-ssl false (on git http.sslVerify false) to disable SSL certificate verification. Something like that would be helpful in pub for situations like this one, right?

[mkustermann]

@zanderso / @nex3

This doesn't seem to be a dart:io issue, since dart:io provides mechanisms for the Dart programmer to override which certificates are trusted via the SecurityContext and HttpClient.badCertificateCallback.

This seems to be more of a feature request for Pub, namely to configure pub get to accept untrusted server certificates. Maybe we should move this to the Pub issue tracker?

[dnfield]

I just opened dart-lang/pub#1882 because I'm facing a similar issue with pub (no proxy, but self signed CA).

I'd really like to see the ability to tell Dart to use a self signed CA, but at the least it'd be nice to be able to tell pub to accept one. However, this issue would be an issue with any Dart program I try to run on my machine that wants to do an HTTPS connection and has no way of knowing that I have a self-signed certificate I want it to trust. This will be a major issue for corporate users behind BlueCoat etc.

[natebosch]

@mkustermann - it's dart:io which is honoring https_proxy, it seems like it could also be a VM level flag to handle certificate overrides etc - why do we think it should be the responsibility of every Dart script that wants to make an https request to handle SecurityContext?

[zanderso]

The command line VM has options for overriding the source of trusted root certificates. See: https://github.com/dart-lang/sdk/blob/master/runtime/bin/main_options.cc#L207

[natebosch]

Sounds good. Should be able to use this with DART_VM_OPTIONS=--root-certs-file=some_file pub get

@crodier - if this is still an issue for you please try with DART_VM_OPTIONS and let us know if that doesn't work for you.

Since this issue is so old I'll close for now and we can reopen if we get a report that this doesn't work.

[stephenrooban]

Sounds good. Should be able to use this with DART_VM_OPTIONS=--root-certs-file=some_file pub get

@crodier - if this is still an issue for you please try with DART_VM_OPTIONS and let us know if that doesn't work for you.

Since this issue is so old I'll close for now and we can reopen if we get a report that this doesn't work.

Hi,

can we pass this argument along with flutter doctor ?

[FlareDev0]

Hi Natebosch,
Can you please explain how to use DART_VM_OPTIONS=--root-certs-file=some_file pub get?

[yathit]

Android have option to use user cert store. I have included <certificates src="user" /> to trust user added CAs, but it does not work in Flutter app.

[zanderso]

@yathit Could you file a new issue for that?

[Aaron009]

@crodier Have you tried to run a simple dart:io program accessing an https URL? E.g.

import 'dart:io';

main() async {
  HttpClient client = new HttpClient();
  var request = await client.getUrl(Uri.parse("https://www.google.com/"));
  var response = await request.close();
  print(response.headers);
  client.close(force: true);
}

I assume that will give the same error?

Do you know what type of authentication the proxy uses? dart:io support basic and digest.

Regarding the certificate, does the proxy server require an HTTPS connection? If so we are out of luck, as that is not supported. The proxy configuration we support are PROXY and DIRECT as described in https://en.wikipedia.org/wiki/Proxy_auto-config (the string returned from findProxy use that format (we don't handle PAC files - just use the same string format returned from the PAC JavaScript code) and there is no way of specifying HTTPS.

I tested it and I replaced the link with www.baidu.com without any issues, but replaced the following link and reported a HandshakeException: Handshake error in client (OS Error: CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate(handshake.cc:352))

import 'dart:io';

main() async {
  HttpClient client = new HttpClient();
  var request = await client.getUrl(Uri.parse('https://shop.io.mi-img.com/app/shop/img?id=shop_88f929c5731967cbc8339cfae1f5f0ec.jpeg'));
  var response = await request.close();
  print(response.headers);
  client.close(force: true);
}

I'm sure no proxy is set。