shallow clone breaks git rebase, gitlens, timeline and other features

[akosyakov]

Bug description

At the same time Codespaces doing shallow clone as well, but don't have such issues. Some should investigate why it does not work nicely in Gitpod.

Creation log from a Codespace:

2021-11-19T08:00:15: $ git -C "/var/lib/docker/codespacemount/workspace" clone --branch ft/publish-ms-version --depth 1 ******/****** "/var/lib/docker/codespacemount/workspace/publish-extensions"
2021-11-19T08:00:16: Cloning into '/var/lib/docker/codespacemount/workspace/publish-extensions'...
2021-11-19T08:00:16: git process exited with exit code 0
2021-11-19T08:00:17: $ git -C "/var/lib/docker/codespacemount/workspace/publish-extensions" config --local remote.origin.fetch +refs/heads/*:refs/remotes/origin/*
2021-11-19T08:00:17: git process exited with exit code 0
2021-11-19T08:00:17: $ git -C "/var/lib/docker/codespacemount/workspace/publish-extensions" config --local remote..github-pr-remote true
2021-11-19T08:00:17: git process exited with exit code 0
2021-11-19T08:00:17: $ git -C "/var/lib/docker/codespacemount/workspace/publish-extensions" config --local branch..github-pr-owner-number open-vsx#publish-extensions#494
2021-11-19T08:00:17: git process exited with exit code 0

I noticed that Codespaces using git 2.33.0, but Gitpod is using 2.30.1 Maybe new git has some special handling.

Steps to reproduce

Start a workspace on big repo and try to do rebase (PR with more than 20 commits will fail to rebase properly), use blame or see timeline.

Workspace affected

No response

Expected behavior

No response

Example repository

No response

Anything else?

No response

[svenefftinge]

For reference previous analysis with the codespaces behavior.

[csweichel]

From Jan in Slack:

Just got very surprised by the shallow clone again: Wanted to do a rebase of a commit that changed a single file and got overwhelmed by a gazillion of merge conflicts. If I hadn't known that I have to unshallow first, I would have been lost.

[geropl]

Broken usecases for me:

• git blame: gitlens showing odd info, sending me towards debugging Gitlens/Gitpod and finally using GitHub for lookups
• git rebase: trying to start a rebase on main gave weird diffs. This one would have sent me back to local VSCode.

If I had not known about this change because of internal communication I would have not been able to fix both situations. IMO it's mainly a communication thing.

[svenefftinge]

We could consider building a vs code extension that detects shallow cloning and provides UI (e.g. notification) to tell users about it and an action to inflate the shallow clone.

[akosyakov]

We could consider building a vs code extension that detects shallow cloning and provides UI (e.g. notification) to tell users about it and an action to inflate the shallow clone.

@jeanp413 You were already adding something to Gitpod extension right?

[jeanp413]

@akosyakov yes I'll work on this today

[jeanp413]

@akosyakov @svenefftinge I took a look at codespaces logs (you can run Codespaces: export logs) and in the StartEnvironment.log you can see they just run git fetch --unshallow in the background. It takes some time to do the unshallow so if you run git log as soon as the ide shows up you'll just see the last commit.

So, should we fix this in ide extension side as planned or in the gitpod supervisor side? Personally, I think this should be fixed in the supervisor.

[2021-11-22 20:07:24.268 StartEnvironment I] Running unshallow and chown.
[2021-11-22 20:07:24.269 StartEnvironment I] [Cmdline] Set working directory to: /var/lib/docker/codespacemount/workspace/gitpod
[2021-11-22 20:07:24.269 StartEnvironment I] Executing raw command: 'git fetch --unshallow --tags'
[2021-11-22 20:07:24.269 StartEnvironment I] Terminal text: '$ git fetch --unshallow --tags'
.
.
.
[2021-11-22 20:07:29.599 StartEnvironment I] git process exited with exit code 0
[2021-11-22 20:07:29.601 StartEnvironment I] Unshallow succeded.
[2021-11-22 20:07:29.688 StartEnvironment I] Executing raw command: 'docker exec -u root 3dd0e8096aa50c83f6322cbe6a75b5a7d92d5d116a736d19d5b1997f939db41c chown -R codespace /workspaces'
[2021-11-22 20:07:29.688 StartEnvironment I] Terminal text: '$ chown -R codespace /workspaces'
[2021-11-22 20:07:36.832 StartEnvironment I] chown process exited with exit code 0
[2021-11-22 20:07:36.858 StartEnvironment I] Unshallow and chown finished.

[svenefftinge]

Yes, it should only happen on workspace starts not on prebuilds.

[akosyakov]

@jeanp413 can you take it? You can use isHeadless check to filter out workspaces which does not need unshallowing. You need to wait till content is ready.