Prebuilds don't run due to `last backup failed: workspace does not exist.`

[mikenikles]

Bug description

The Prebuild logs display the following console output in the dashboard:

Connecting to workspace logs...

last backup failed: workspace does not exist.

Screenshot

However, the list of Prebuilds for that project lists the above Prebuild as "READY".

Screenshot

Steps to reproduce

1. Create a new repository in GitHub
2. Commit a .gitpod.yml file with the following content

   image: gitpod/workspace-postgres
   
   tasks:
     - command: echo ""

3. Connect the project to Gitpod via https://gitpod.io/new
4. Check the Prebuild that was started automatically

Workspace affected

N/A

Expected behavior

The Prebuild runs, shows the workspace logs and is reported as READY on the relevant screens in the dashboard.

Example repository

https://github.com/mikenikles/gitpod-test-postgresql

Anything else?

No response

[kylos101]

There are actually two problems here:

1. The prebuild failed, hinting last backup failed: workspace does not exist.
2. The prebuild is indicated as Ready, but, no workspace can be started from it. @geropl does this warrant a separate issue, or is it already solved with ongoing prebuild work you have in-flight?

[kylos101]

This appears to heavily impact prebuilds, 753 in the last day.

We'll want to see if we can recreate the problem (i.e. is it related or not to incremental prebuilds, and socialize next steps).

This is a research issue. The output isn't necessarily a code fix - but it could be - it depends what is found.

[geropl]

@geropl does this warrant a separate issue, or is it already solved with ongoing prebuild work you have in-flight?

@kylos101 That's something we just fixed. First part will roll out today, 2nd end of week (I think).

[kylos101]

I am going to close this, in favor of #8326 (which is the same issue), and add that back to the webapp inbox.

@mikenikles please subscribe to 8326 for continued discussion?

[kylos101]

I was a bit hasty there, this is for prebuilds, which won't have a backup like a regular workspace...