Add function to checkout the jobs repository #75
Conversation
ColinKinloch
force-pushed
the
git
branch
2 times, most recently
from
587d8dd to
ff98e93
Compare
ColinKinloch
force-pushed
the
git
branch
4 times, most recently
from
082f212 to
7428588
Compare
ColinKinloch
force-pushed
the
git
branch
3 times, most recently
from
dbcc18f to
c0f9c71
Compare
gitlab-runner/src/lib.rs
Outdated
| } | ||
|
|
||
| // TODO: Should clone_git_repository be async | ||
| // gitoxide doesn't currently have async implemented for http backend |
There was a problem hiding this comment.
I don't think this is viable without either async or cloning in a separate thread (which would make cancellation really tricky, though you seem to have some interrupt-related stuff wired up already); blocking an entire I/O thread for what might be a very long clone process is not workable. Should this maybe just be using libgit2 or something else instead?
There was a problem hiding this comment.
fwiw i'd prefer to stay with gitoxide to avoid more C dependences in the core of gitlab-runner. But yes it should be done in a dedicated thread if there is no async api
There was a problem hiding this comment.
I've updated it to use tokio::task::spawn_blocking which runs it on a different thread.
I used a CancellationToken and tokio::select to trigger gitoxide to interrupt, CancellationToken is already used in the gitlab-runner-rs API so hopefully is okay.
gitoxide re-exports Progress from the prodash crate to allow the caller to monitor progress. Is that something we should do?
| * * Clones to BuildDir | ||
| * * Checks out git_info.sha and fetches all in git_info.refspecs | ||
| * * Hardcoded username "gitlab-ci-token" | ||
| * * credHelperCommand? |
There was a problem hiding this comment.
this is a bit tricky to understand out-of-context, might want to add a few more details for some of these points
gitlab-runner/src/lib.rs
Outdated
| } | ||
|
|
||
| let checkout_progress = progress.add_child("checkout".to_string()); | ||
| let (checkout, _outcome) = fetch.fetch_then_checkout(checkout_progress, &should_interrupt)?; |
There was a problem hiding this comment.
thought: if the goal is just to be able to read files from the repo, do we need to do an entire checkout to begin with? couldn't it clone a bare repo and then load files as-needed from there? though the apis might be harder to make for that...but it would likely also be a lot faster.
There was a problem hiding this comment.
Fwiw i wouldn't mind this being a potential future update ; I'm not sure how much faster it would be ("likely be a lot of faster" is always a dangerous statement in the land of optimisation). I guess it primarily depend on what work would be saved, which i guess is primarily unpacking the pack file.
Regardless; the time would be comparable to a normal checking in a normal gitlab runner (or faster as iirc gix is faster at this point then git).
ColinKinloch
force-pushed
the
git
branch
4 times, most recently
from
246508e to
6b04326
Compare
* `clone_git_repository` to clone arbitrary repo * `Job::clone_git_repository` to clone the jobs repo
Also throw an error if the provided HEAD ref cannot be parsed.
This branch adds
crate::clone_git_repositoryto clone git repos generally andJob::clone_git_repositoryto clone specifically defined by the gitlab job.clone_git_repositoryis heavily based onPrepareFetch::fetch_onlySince the gitoxide
PrepareFetchandPrepareCheckoutstructs delete the target path when they are dropped I opted to checkout the repository to a temporary directory, then move it's contents to the path provided to the target directory.This adds a rather unwieldy
GitCheckoutErrorenum. gitoxidesErrortypes triggers theresult_large_errclippy warning.This requires rust version
1.73.0or later.Fixes #21