Automatically retry on 429 Too Many Requests

[soxofaan]

refs #441, #764

[soxofaan]

TODO:

• changelog entry
• allow user to easily configure retry settings (e.g. number of retries, backoff_factor)
• tests about job.start_and_wait loop?

[soxofaan]

Maybe it's best to not include Internal Server Error by default for now.
Unlike the others (429/502/503/504) this is a very general error situation, and not something where we can assume that just waiting a bit and retrying will resolve the problem.

[soxofaan]

While working on the tests, it also occurred to me that this feature introduces a bit of conflict with poll-loop situations like job.start_and_wait() and the job manager where there is already skipping of "soft errors" or temporary glitches in some way.
By enabling this Retry adapter by default, the user experience (e.g. in terms of responsiveness or throughput of the poll-loops) will get worse because of multiple unnecessary but forced sleeps.

So this needs some more investigation (e.g can the retry config be disabled in particular places or contexts?)