Skip to content

Transparently re-try image builds if one fails with error code "INTERNAL" #4893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
geropl opened this issue Jul 21, 2021 · 3 comments
Closed
Labels
component: image-builder component: server meta: stale This issue/PR is stale and will be closed soon priority: highest (user impact) Directly user impacting type: bug Something isn't working

Comments

@geropl
Copy link
Member

geropl commented Jul 21, 2021

During image builds we often experience short-lived situations where we starve for resource, e.g. disk space. For this reason we designed the system to be idempotent so we can re-start with confidence.

We don't do so, however. This should be as simple as adding a re-try loop with backoff.

This would catch issues like this: https://community.gitpod.io/t/error-build-failed-when-trying-to-build-a-docker-image/4216

/cc @csweichel suggested this(/sth similar?) yesterday as well

@csweichel
Copy link
Contributor

/schedule

@csweichel
Copy link
Contributor

We can implement this behaviour for image-builder-mk3 easily, but mk2 makes no distinction between system failure and "user input errors".

I'd propose we take on this task once #4812 is done.

(removing from groundwork right now - please re-add once mk3 is merged)

@stale
Copy link

stale bot commented Oct 20, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the meta: stale This issue/PR is stale and will be closed soon label Oct 20, 2021
@stale stale bot closed this as completed Oct 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: image-builder component: server meta: stale This issue/PR is stale and will be closed soon priority: highest (user impact) Directly user impacting type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants