Deal with concurrent runs due to starting delays

[Bisaloo]

I suspect #86 wasn't the right fix since artifacts will always overwrite the previous ones. If jobs timings make it so that jobs from two different runs pull the same artifact, there will be nothing to commit.

Originally posted by @Bisaloo in #53 (comment)