Skip to content

data lost when workspace timeout and pod is in the terminating state  #9544

Closed
@jeffrey-zhang

Description

@jeffrey-zhang

Bug description

Background, we are evaluating the gitpod and deployed it in our self-hosted AKS. The current deployed gitpod version is 2202.03.01.

We encountered the issue several times that the workspace will be timeout after 30 mins by default and it can't be opened again. The serious issue is that all data is lost once this issue occurred. And we try to find the lost data in the storage account, but it seems that it isn't sync-up the data in time.

We try to check this issue in our AKS and find the pod is in the interminating state. The error message is like this from our AKS log.
image

Steps to reproduce

this issue isn't always occurred. But here are the steps that occurred today.

step1: open the gitpod from the github repo.

step2: modify the code this morning about 2h

step3: keep the gitpod opened in the browser for about 2 hours of break time

(left office to have lunch for about 2h)

step4: find gitpod workspace is a timeout
00001

step5: try to open the workspace again and prompt another error
002

step6: click workspace again. it is opened again and all data are lost.

it seems that there is a bug to sync the data to the storage account. it doesn't sync data during the workspace is active by version control so that all data are lost once the final sync-up failed when the workspace is timeout.

Workspace affected

No response

Expected behavior

This issue can be fixed and this function can be worked normally.

in addition, it is better that we can design this feature to sync the data to the storage account by scheduling for version control when the workspace is active at least. this can avoid all data being lost once the final sync-up failed when the workspace is timeout.

Example repository

No response

Anything else?

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions