Pub is unstable on CI, hard to download packages

[matanlurey]

Tracking issue for: angulardart/angular#1068

Example, latest with 2.0.0-dev.36.0:
https://api.travis-ci.org/v3/job/353631788/log.txt

Downloading source_map_stack_trace 1.1.4...
Downloading shelf_static 0.2.7...
Downloading intl 0.15.2...
Downloading analyzer 0.31.2-alpha.0...
Downloading plugin 0.2.0+2...
Downloading isolate 1.1.0...
Downloading kernel 0.3.0-alpha.10...
Downloading front_end 0.1.0-alpha.10...
Connection terminated during handshake

I've seen this flake almost daily on Travis.

It would be great to be more resistant and/or retry, at minimum.

[matanlurey]

This for whatever reason happens most often with front_end.

Here I see a different error:
https://api.travis-ci.org/v3/job/353636067/log.txt

Connection closed before full header was received

[matanlurey]

Another one:
https://api.travis-ci.org/v3/job/353636070/log.txt

Resolving dependencies...
Got TLS error trying to find package quiver at https://pub.dartlang.org.

[kevmoo]

We could add this type of error to the set of things we retry - 932e76e

[kevmoo]

Will likely land as part of -dev.41

[matanlurey]

Thanks! I'll keep my eye on it.

[matanlurey]

I'm still hitting this as of -dev.44:

https://api.travis-ci.org/v3/job/362300030/log.txt

+ args 1.4.1
+ async 2.0.6
+ charcode 1.1.1
+ collection 1.14.9
+ dev 0.0.0 from path dev
+ glob 1.1.5
+ meta 1.1.2
+ path 1.5.1
+ pub_semver 1.3.4
+ source_span 1.4.0
+ string_scanner 1.0.2
+ yaml 2.1.13
Downloading yaml 2.1.13...
Downloading path 1.5.1...
Downloading meta 1.1.2...
Downloading glob 1.1.5...
Downloading args 1.4.1...
Downloading charcode 1.1.1...
Downloading pub_semver 1.3.4...
Downloading string_scanner 1.0.2...
Downloading source_span 1.4.0...
Downloading async 2.0.6...
Downloading collection 1.14.9...
Connection closed before full header was received

travis_time:end:0868c784:start=1522869912142984663,finish=1522869913430769239,duration=1287784576
�[0K
�[31;1mThe command "pub get" failed and exited with 69 during .�[0m

Your build has been stopped.

[nex3]

That's very strange... we should be retrying on every IO exception. The only reasons I can imagine for this are that the "Connection closed before full header was received" error isn't actually an IOException, or the error isn't being thrown somewhere that http_retry can catch it (that is, it's not thrown by HttpClient.open() or HttpClientRequest.close()). Either of these seems like a bug in dart:io. I'll spend some time next week seeing if I can get a local reproduction.

[nex3]

Looks like this was a silly error on my part—http wraps errors in a cross-platform ClientException class, which is what we should be checking for.

[sigurdm]

We are planning to improve the http behavior. Hopefully this will also improve this.

Closing in favor of: #3325