Skip to content

Issue441 automatic backoff retry on http429 #547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

jdries
Copy link
Collaborator

@jdries jdries commented Mar 12, 2024

@soxofaan looks like something we can merge easily as this is mostly about using functionality from requests?

@jdries jdries linked an issue Mar 12, 2024 that may be closed by this pull request
@jdries jdries requested a review from soxofaan March 12, 2024 09:07
Copy link
Member

@soxofaan soxofaan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some quick points to further finetune

@@ -28,6 +28,7 @@
"flake8>=5.0.0",
"time_machine",
"pyproj>=3.2.0", # Pyproj is an optional, best-effort runtime dependency
"re-assert",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have dirty_equals for these kind of things, wee should not add yet another utility here


assert caplog.messages == [
"Starting new HTTPS connection (1): example.test:443",
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are only tests with retries but ending in failure, there are no tests about retries that end up in success

import requests
import requests.adapters

MAX_RETRIES = 3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default of max 3 retries might be a bit low in practice in combination with default backoff_factor: last attempt will be roughly just be 4 seconds after first attempt, which might be too quick to stop trying

@soxofaan
Copy link
Member

soxofaan commented May 9, 2025

Because of conflicts: redoing this PR in #770

@soxofaan soxofaan closed this May 9, 2025
@soxofaan soxofaan deleted the issue441_automatic-backoff-retry-on-HTTP429 branch May 9, 2025 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automatic backoff/retry on HTTP 429 Too Many Requests
3 participants