Skip to content

Tests fail to run locally #960

Closed
Closed
@epage

Description

@epage
Contributor

Duplicates

  • I have searched the existing issues

Current behavior 😯

Tests fail, often in different ways

Expected behavior 🤔

Tests pass

Steps to reproduce 🕹

$ git --version
git version 2.34.1
$ git lfs --version
git-lfs/3.0.2 (GitHub; linux amd64; go 1.18.1)
$ git clone https://github.com/Byron/gitoxide.git
$ git lfs install
$ git lfs pull
$ just test
...
failures:

---- index::file::read::sparse_checkout_non_sparse_index stdout ----
Archive at 'fixtures/generated-archives/v3_skip_worktree.tar.xz' not found, creating fixture using script 'make_index/v3_skip_worktree.sh'
thread 'index::file::read::sparse_checkout_non_sparse_index' panicked at 'assertion failed: `(left == right)`
  left: `Flags(EXTENDED | SKIP_WORKTREE)`,
 right: `Flags(0x0)`', gix-index/tests/index/file/read.rs:245:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    index::file::read::sparse_checkout_non_sparse_index

test result: FAILED. 41 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s

error: test failed, to rerun pass `-p gix-index-tests --test single-threaded`
error: Recipe `unit-tests` failed on line 132 with exit code 101
$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   gix-commitgraph/tests/fixtures/generated-archives/octopus_merges.tar.xz
        modified:   gix-diff/tests/fixtures/generated-archives/make_diff_repo.tar.xz
        modified:   gix-index/tests/fixtures/generated-archives/v3_skip_worktree.tar.xz

no changes added to commit (use "git add" and/or "git commit -a")
$ git reset --hard
$ cargo test --all --exclude gix-index-tests
...
failures:

---- worktree::checkout::accidental_writes_through_symlinks_are_prevented_if_overwriting_is_forbidden stdout ----
Extracted fixture from archive 'tests/fixtures/generated-archives/make_dangerous_symlink.tar.xz' (3715112181, Some("unix"))
thread 'worktree::checkout::accidental_writes_through_symlinks_are_prevented_if_overwriting_is_forbidden' panicked at 'assertion failed: `(left == right)`
  left: `["A-dir/a", "A-file", "fake-dir/b", "fake-file"]`,
 right: `["A-dir/a", "A-file", "FAKE-DIR", "FAKE-FILE", "fake-dir/b", "fake-file"]`', gix-worktree/tests/worktree/checkout.rs:81:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- worktree::cache::attributes::baseline stdout ----
Extracted fixture from archive 'tests/fixtures/generated-archives/make_attributes_baseline.tar.xz' (2874926218, Some("unix"))
thread 'worktree::cache::attributes::baseline' panicked at 'assertion failed: `(left == right)`
  left: `[AssignmentRef { name: NameRef(Borrowed("info")), state: Value(ValueRef(Borrowed("attributes"))) }]`,
 right: `[AssignmentRef { name: NameRef(Borrowed("global-no-wildcard-case-test")), state: Set }, AssignmentRef { name: NameRef(Borrowed("global-wildcard-case-test")), state: Set }, Assignmen
tRef { name: NameRef(Borrowed("info")), state: Value(ValueRef(Borrowed("attributes"))) }]`: we have the same matches: "Z/x/a"', gix-worktree/tests/worktree/cache/attributes.rs:53:9

---- worktree::checkout::writes_through_symlinks_are_prevented_even_if_overwriting_is_allowed stdout ----
thread 'worktree::checkout::writes_through_symlinks_are_prevented_even_if_overwriting_is_allowed' panicked at 'assertion failed: `(left == right)`
  left: `["A-dir/a", "A-file", "fake-dir/b", "fake-file"]`,
 right: `["A-dir/a", "A-file", "FAKE-DIR", "FAKE-FILE", "fake-dir/b", "fake-file"]`', gix-worktree/tests/worktree/checkout.rs:110:9


failures:
    worktree::cache::attributes::baseline
    worktree::checkout::accidental_writes_through_symlinks_are_prevented_if_overwriting_is_forbidden
    worktree::checkout::writes_through_symlinks_are_prevented_even_if_overwriting_is_allowed

test result: FAILED. 19 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.11s

error: test failed, to rerun pass `-p gix-worktree --test worktree-single-threaded`
$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   gix-odb/tests/fixtures/generated-archives/make_repo_multi_index.tar.xz
        modified:   gix-worktree/tests/fixtures/generated-archives/make_ignorecase_collisions.tar.xz

no changes added to commit (use "git add" and/or "git commit -a")
$ git reset --hard
$ cargo test --all --exclude gix-index-tests --exclude gix-worktree
$ git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean

Activity

epage

epage commented on Aug 2, 2023

@epage
ContributorAuthor

My other worktree is seeing:

$ cargo test --all
---- remote::connection::fetch::refs::tests::update::remote_symbolic_refs_can_be_written_locally_and_point_to_tracking_branch stdout ----
thread 'remote::connection::fetch::refs::tests::update::remote_symbolic_refs_can_be_written_locally_and_point_to_tracking_branch' panicked at 'called `Result::unwrap()` on an `Err` value: To
Id(NotFound { oid: Sha1(f99771fe6a1b535783af3163eba95a927aae21d5), name: "refs/heads/main" })', gix/src/remote/connection/fetch/update_refs/tests.rs:585:50
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- remote::connection::fetch::refs::tests::update::non_fast_forward_is_rejected_but_appears_to_be_fast_forward_in_dryrun_mode stdout ----
thread 'remote::connection::fetch::refs::tests::update::non_fast_forward_is_rejected_but_appears_to_be_fast_forward_in_dryrun_mode' panicked at 'called `Result::unwrap()` on an `Err` value:
ToId(NotFound { oid: Sha1(f99771fe6a1b535783af3163eba95a927aae21d5), name: "refs/heads/main" })', gix/src/remote/connection/fetch/update_refs/tests.rs:585:50

---- remote::connection::fetch::refs::tests::update::checked_out_branches_in_worktrees_are_rejected_with_additional_information stdout ----
thread 'remote::connection::fetch::refs::tests::update::checked_out_branches_in_worktrees_are_rejected_with_additional_information' panicked at 'called `Result::unwrap()` on an `Err` value:
ToId(NotFound { oid: Sha1(f99771fe6a1b535783af3163eba95a927aae21d5), name: "refs/remotes/origin/main" })', gix/src/remote/connection/fetch/update_refs/tests.rs:585:50

---- remote::connection::fetch::refs::tests::update::remote_refs_cannot_map_to_local_head stdout ----
thread 'remote::connection::fetch::refs::tests::update::remote_refs_cannot_map_to_local_head' panicked at 'called `Result::unwrap()` on an `Err` value: ToId(NotFound { oid: Sha1(f99771fe6a1b
535783af3163eba95a927aae21d5), name: "refs/heads/main" })', gix/src/remote/connection/fetch/update_refs/tests.rs:585:50

---- remote::connection::fetch::refs::tests::update::various_valid_updates stdout ----
thread 'remote::connection::fetch::refs::tests::update::various_valid_updates' panicked at 'called `Result::unwrap()` on an `Err` value: ToId(NotFound { oid: Sha1(f99771fe6a1b535783af3163eba
95a927aae21d5), name: "refs/heads/main" })', gix/src/remote/connection/fetch/update_refs/tests.rs:585:50

---- remote::connection::fetch::refs::tests::update::local_symbolic_refs_are_never_written stdout ----
thread 'remote::connection::fetch::refs::tests::update::local_symbolic_refs_are_never_written' panicked at 'called `Result::unwrap()` on an `Err` value: ToId(NotFound { oid: Sha1(f99771fe6a1
b535783af3163eba95a927aae21d5), name: "refs/heads/main" })', gix/src/remote/connection/fetch/update_refs/tests.rs:585:50

---- remote::connection::fetch::refs::tests::update::local_direct_refs_are_never_written_with_symbolic_ones_but_see_only_the_destination stdout ----
thread 'remote::connection::fetch::refs::tests::update::local_direct_refs_are_never_written_with_symbolic_ones_but_see_only_the_destination' panicked at 'called `Result::unwrap()` on an `Err
` value: ToId(NotFound { oid: Sha1(f99771fe6a1b535783af3163eba95a927aae21d5), name: "refs/heads/main" })', gix/src/remote/connection/fetch/update_refs/tests.rs:585:50

---- remote::connection::fetch::refs::tests::update::remote_symbolic_refs_can_always_be_set_as_there_is_no_scenario_where_it_could_be_nonexisting_and_rejected stdout ----
thread 'remote::connection::fetch::refs::tests::update::remote_symbolic_refs_can_always_be_set_as_there_is_no_scenario_where_it_could_be_nonexisting_and_rejected' panicked at 'called `Result
::unwrap()` on an `Err` value: ToId(NotFound { oid: Sha1(f99771fe6a1b535783af3163eba95a927aae21d5), name: "refs/heads/main" })', gix/src/remote/connection/fetch/update_refs/tests.rs:585:50

ailures:
    remote::connection::fetch::refs::tests::update::checked_out_branches_in_worktrees_are_rejected_with_additional_information
    remote::connection::fetch::refs::tests::update::local_direct_refs_are_never_written_with_symbolic_ones_but_see_only_the_destination
    remote::connection::fetch::refs::tests::update::local_symbolic_refs_are_never_written
    remote::connection::fetch::refs::tests::update::non_fast_forward_is_rejected_but_appears_to_be_fast_forward_in_dryrun_mode
    remote::connection::fetch::refs::tests::update::remote_refs_cannot_map_to_local_head
    remote::connection::fetch::refs::tests::update::remote_symbolic_refs_can_always_be_set_as_there_is_no_scenario_where_it_could_be_nonexisting_and_rejected
    remote::connection::fetch::refs::tests::update::remote_symbolic_refs_can_be_written_locally_and_point_to_tracking_branch
    remote::connection::fetch::refs::tests::update::various_valid_updates

test result: FAILED. 4 passed; 8 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.30s

error: test failed, to rerun pass `-p gix --lib`
epage

epage commented on Aug 2, 2023

@epage
ContributorAuthor

Likely unrelated but a job for #956 failed but later jobs didn't and related jobs didn't

        FAIL [   0.286s] gix-worktree::worktree-single-threaded worktree::checkout::keep_going_collects_results

--- STDOUT:              gix-worktree::worktree-single-threaded worktree::checkout::keep_going_collects_results ---

running 1 test
test worktree::checkout::keep_going_collects_results ... FAILED

failures:

failures:
    worktree::checkout::keep_going_collects_results

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 21 filtered out; finished in 0.25s


--- STDERR:              gix-worktree::worktree-single-threaded worktree::checkout::keep_going_collects_results ---
thread 'worktree::checkout::keep_going_collects_results' panicked at 'assertion failed: `(left == right)`
  left: `2`,
 right: `3`', gix-worktree/tests/worktree/checkout.rs:293:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

See https://github.com/Byron/gitoxide/actions/runs/5733348381/job/15537902571

self-assigned this
on Aug 3, 2023
Byron

Byron commented on Aug 3, 2023

@Byron
Member

Thanks a lot for the methodical listing of issues executing the test-suite, it's easy to follow what happened and I am trying to reproduce this on a debian system with a case-sensitive file-system.

Just for completeness, regarding the comment right above this one, I have seen this before happen occasionally, but only on CI. My feeling is that it's related to the local state 'somehow', and it only happens when the code is run with multi-threading enabled (via gix-features). I don't know if it always happens for you though, and if so, I hope to see the same issue locally.

Everything else seems state-related, even though it doesn't yet make much sense to me. Maybe there is an issue with some of the git-lfs archives being created on MacOS, and a case-sensitive file-system. That would be strange, as CI runs on linux without git-lfs, so it recreates all state by executing the source script, while also running the archived versions on MacOS and Windows… both of which expect a case-sensitive file-system. CI doesn't actually run the archived version on Linux, coincidentally actually.


The above gave me a lead. It seems that…

  • …CI should run test-fast on linux as it would rely on archives (and probably fail because of it)
  • …archives should include meta-data about whether or not the filesystem that created them was case-sensitive, and the system shouldn't use these if the target file system case handling doesn't match the one in the archive.

Ideally, the above should fix all of the issues except for maybe keep_going_collects_results - I will keep you posted.

added a commit that references this issue on Aug 3, 2023
Byron

Byron commented on Aug 3, 2023

@Byron
Member

It wasn't easy and took quite some time but I think that all of these issues should go away if you rebase onto main and clear out the working tree to get rid of the possibly incorrect caches.
Please let me know if there are any issues remaining, but it works for me now on my linux VM.

niklaswimmer

niklaswimmer commented on Aug 3, 2023

@niklaswimmer
Contributor

Just as an extra data point: on my system (Arch Linux) both just test and just nextest now work. The test suite already worked before the latest changes, the nextest suite not.

epage

epage commented on Aug 4, 2023

@epage
ContributorAuthor

Hmm, I'm still seeing this failure:

$ just test
...
failures:

---- index::file::read::sparse_checkout_non_sparse_index stdout ----
Archive at 'fixtures/generated-archives/v3_skip_worktree.tar.xz' not found, creating fixture using script 'make_index/
v3_skip_worktree.sh'
thread 'index::file::read::sparse_checkout_non_sparse_index' panicked at 'assertion failed: `(left == right)`
  left: `Flags(EXTENDED | SKIP_WORKTREE)`,
 right: `Flags(0x0)`', gix-index/tests/index/file/read.rs:245:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    index::file::read::sparse_checkout_non_sparse_index

test result: FAILED. 41 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s

error: test failed, to rerun pass `-p gix-index-tests --test single-threaded`
error: Recipe `unit-tests` failed on line 132 with exit code 101

However, that is now my only failure. cargo test --exclude gix-index-tests passes

Byron

Byron commented on Aug 5, 2023

@Byron
Member

In this situation, this script executes and it's expected to produce an index with entries that have certain flags.

This doesn't happen, even though the script doesn't fail.
Can I see the git --version you are using?

Lastly, as the correct index is contained in the archive that is not available, and if you clear local caches with something like find gix-index -name generated-do-not-edit -exec rm -Rfv \{\} \; and fetch the archives using git-lfs this test should pass, too.

But overall, I am glad this works so much better now and hope that the additional CI test on linux will help to prevent regressions.

epage

epage commented on Aug 6, 2023

@epage
ContributorAuthor

Can I see the git --version you are using?

$ git --version
git version 2.34.1

Still failing after running that command and doing a git lfs pull and git lfs fetch (just covering all my bases). I had originally cloned a new repo for this anyways, so all of the caches were clean when I did the above testuing.

Byron

Byron commented on Aug 6, 2023

@Byron
Member

Thanks for trying again!

This gets more and more mysterious. The git version used should be able to produce the desired index file, and if not I'd assume it would fail.

If the index is as expected, it would mean the software truly regressed. I thought that it's maybe related to something that happened in the winnow branch, but after retrying there is still worked for me.

So I thought maybe we can do one more test to rule out the data is invalid. If you could do the following:

  • download index.zip (SHA1 of index: 010c0c5fe37833d6aca6d8dca7928e60725b50b4)
  • run sha1 gix-index/tests/fixtures/generated-do-not-edit/v3_skip_worktree/1190435892-unix/.git/index - if it's not the hash above copy the downloaded index file to gix-index/tests/fixtures/generated-do-not-edit/v3_skip_worktree/1190435892-unix/.git/index

Now the test should work as expected as it's definitely the correct file. git does pick up user configuration naturally and maybe that's picking up some configuration that does indeed affect that particular test. Maybe that could be it, let's see.

epage

epage commented on Aug 7, 2023

@epage
ContributorAuthor

My user config: https://github.com/epage/vimfiles/blob/master/extra/config/git/config

btw I was generally doing these tests on master to avoid my changes being involved

Byron

Byron commented on Aug 8, 2023

@Byron
Member

My user config: https://github.com/epage/vimfiles/blob/master/extra/config/git/config

Nothing looks unusual, but nonetheless I (think to) have prevented git from reading system and user configs to isolate the test-suite more.

It seems all that's left is to try to use the provided index.zip file (see my comment above) if you get the chance.

epage

epage commented on Aug 8, 2023

@epage
ContributorAuthor

Ah, accidentally read over those steps.

How do you normally get a sha1 binary? apt isn't suggesting any.

Byron

Byron commented on Aug 8, 2023

@Byron
Member

It appears to be part of coreutils.

~$ dpkg -S sha1sum
coreutils: /usr/share/man/man1/sha1sum.1.gz
coreutils: /usr/bin/sha1sum
epage

epage commented on Aug 8, 2023

@epage
ContributorAuthor

Ah, sum was missing at the end

Before with failure:

$ sha1sum gix-index/tests/fixtures/generated-do-not-edit/v3_skip_worktree/1190435892-unix/.git/index
f8660bca1f34ea1743aa5e096038faff77a3a5e7  gix-index/tests/fixtures/generated-do-not-edit/v3_skip_worktree/1190435892-u
nix/.git/index

Swapping in the provided index got me past that failure and now I'm getting

cd cargo-smart-release && ./tests/journey.sh /home/epage/src/personal/gitoxide/cargo-smart-release/target/debug/cargo-
smart-release
-----------------------------------------------------
changelog
-----------------------------------------------------
[when] interacting with 'a'
  [with] dry-run only
     [it] succeeds
  [with] --write
     [it] succeeds
     [with] .git and target/ directories removed
        [it] managed to write a changelog
-----------------------------------------------------
smart-release
-----------------------------------------------------
[with] 'c'
  [with] -d minor to bump minor dependencies
     [it] succeeds
[when] releasing 'a'
  [with] dry-run only
     [with] conditional version bumping
        [with] explicit bump specification
           [it] succeeds11,12c11,12
< [TRACE] WOULD create tag object a-v0.8.0 with changelog message, first line is: '### Commit Statistics'
< [TRACE] WOULD run "gh" "release" "create" "a-v0.8.0" "--title" "a v0.8.0" "--notes" "### Commit Statistics\n…" [note
 truncated]
\ No newline at end of file
---
> [WARN ] To create github releases, please install the 'gh' program and try again
> [TRACE] WOULD create tag object a-v0.8.0 with changelog message, first line is: '### Commit Statistics'
\ No newline at end of file
 - FAIL
$ /home/epage/src/personal/gitoxide/cargo-smart-release/target/debug/cargo-smart-release smart-release a --no-push --n
o-publish -v --allow-dirty -b minor
Output snapshot did not match snapshot at '/home/epage/src/personal/gitoxide/cargo-smart-release/tests/snapshots/tripl
e-depth-workspace/a-dry-run-success-multi-crate'
[WARN ] Consider running with --update-crates-index to assure bumping on demand uses the latest information
[WARN ] Crates.io index doesn't exist. Consider using --update-crates-index to help determining if release versions ar
e published already
[INFO ] Manifest version of provided package 'a' at 0.8.0 is sufficient, creating a new release 🎉, ignoring computed
version 0.9.0
[INFO ] WOULD adjust version constraints in manifest of 1 package as direct dependencies are changing: b
[INFO ] WOULD modify existing changelog for 'a'.
[TRACE] WOULD persist changes to 2 manifests and 1 changelogs with: "Adjusting changelogs prior to release of a v0.8.0
"
[INFO ] Up to 1 changelog would be previewed if the --execute is set and --no-changelog-preview is unset.
[WARN ] WOULD ask for review after commit as the changelog entry is empty for crate: a
[WARN ] To fix the changelog manually, run: cargo changelog --write a
[TRACE] WOULD run "git" "commit" "-am" "Adjusting changelogs prior to release of a v0.8.0"
[WARN ] To create github releases, please install the 'gh' program and try again
[TRACE] WOULD create tag object a-v0.8.0 with changelog message, first line is: '### Commit Statistics'
error: Recipe `journey-tests-smart-release` failed on line 211 with exit code 1

and

$ git status
On branch main
Your branch is ahead of 'origin/main' by 1052 commits.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   gix-odb/tests/fixtures/generated-archives/make_repo_multi_index.tar.xz

no changes added to commit (use "git add" and/or "git commit -a")
Byron

Byron commented on Aug 9, 2023

@Byron
Member

Swapping in the provided index got me past that failure [..]

That's great, and means that somehow git didn't produce the result it was supposed to. It's unclear to me what happened there. The next time the generated-do-no-edit folder is wiped without the archives being pulled, the error might return as it would re-execute the script. I hope you can rebase against main so that git shouldn't be able anymore to read user and system configuration.

[..] and now I'm getting [..]

It looks like the test implicitly relies on the gh tool to be installed, and if it's not it fails as the output changes.

> [WARN ] To create github releases, please install the 'gh' program and try again
# should be
< [TRACE] WOULD run "gh" "release" "create" "a-v0.8.0" "--title" "a v0.8.0" "--notes" "### Commit Statistics\n…" [note

on main this is fixed as well by skipping these tests if gh isn't present.

Thanks for taking another look, maybe it runs through now :D.

epage

epage commented on Aug 10, 2023

@epage
ContributorAuthor

I guess this can be closed for now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @epage@Byron@niklaswimmer

      Issue actions

        Tests fail to run locally · Issue #960 · GitoxideLabs/gitoxide