Skip to content

testsuite: arduino-cli should never panic during tests #1873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 9, 2022

Conversation

cmaglie
Copy link
Member

@cmaglie cmaglie commented Sep 9, 2022

Please check if the PR fulfills these requirements

  • The PR has no duplicates (please search among the Pull Requests
    before creating one)
  • The PR follows
    our contributing guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • UPGRADING.md has been updated with a migration guide (for breaking changes)
  • What kind of change does this PR introduce?
    The testsuite is now able to detect arduino-cli panics when running integration tests.
  • What is the current behavior?
    The following test
      _, _, err = cli.Run("lib", "upgrade", "Servo")
      require.Error(t, err)
    may pass if arduino-cli panics, but this is wrong, arduino-cli should always return gracefully with a polite error message.
  • What is the new behavior?
    The above test fails:
    arduino-cli.go:165: 
                Error Trace:    /home/megabug/Workspace/arduino-cli/internal/integrationtest/lib/arduino-cli.go:165
                                                        /home/megabug/Workspace/arduino-cli/internal/integrationtest/lib/lib_test.go:40
                Error:          "panic: runtime error: invalid memory address or nil pointer dereference
                                [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xba23e4]
                            
                                goroutine 1 [running]:
                                github.com/arduino/arduino-cli/commands/lib.upgrade(0xc000139440?, {0xc00047fcd8, 0x1, 0xc00047fca0?}, 0xc00047fca8?, 0x40f667?)
                                        /home/megabug/Workspace/arduino-cli/commands/lib/upgrade.go:61 +0x64
                                github.com/arduino/arduino-cli/commands/lib.LibraryUpgrade({0xc0000ec770?, 0x4?}, 0xc005a9acd0, 0x1?, 0x1?)
                                        /home/megabug/Workspace/arduino-cli/commands/lib/upgrade.go:55 +0x11d
                                github.com/arduino/arduino-cli/cli/lib.runUpgradeCommand(0xc000392a00?, {0xc000297990, 0x1, 0x1?})
                                        /home/megabug/Workspace/arduino-cli/cli/lib/upgrade.go:60 +0x2eb
                                github.com/spf13/cobra.(*Command).execute(0xc000392a00, {0xc000297970, 0x1, 0x1})
                                        /home/megabug/.gvm/pkgsets/go1.19/global/pkg/mod/github.com/spf13/[email protected]/command.go:860 +0x663
                                github.com/spf13/cobra.(*Command).ExecuteC(0xc0002bc780)
                                        /home/megabug/.gvm/pkgsets/go1.19/global/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3bd
                                github.com/spf13/cobra.(*Command).Execute(...)
                                        /home/megabug/.gvm/pkgsets/go1.19/global/pkg/mod/github.com/spf13/[email protected]/command.go:902
                                main.main()
                                        /home/megabug/Workspace/arduino-cli/main.go:31 +0x77
                                " should not contain "panic: runtime error:"
                Test:           TestLibUpgradeCommand
                Messages:       arduino-cli panicked
--- FAIL: TestLibUpgradeCommand (22.90s)

@cmaglie cmaglie self-assigned this Sep 9, 2022
@cmaglie cmaglie added priority: high Resolution is a high priority topic: infrastructure Related to project infrastructure type: imperfection Perceived defect in any part of project criticality: low Of low impact labels Sep 9, 2022
@codecov
Copy link

codecov bot commented Sep 9, 2022

Codecov Report

Merging #1873 (d352f05) into master (3cd782d) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1873   +/-   ##
=======================================
  Coverage   36.67%   36.67%           
=======================================
  Files         231      231           
  Lines       19609    19658   +49     
=======================================
+ Hits         7191     7210   +19     
- Misses      11593    11620   +27     
- Partials      825      828    +3     
Flag Coverage Δ
unit 36.67% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
internal/integrationtest/arduino-cli.go 83.93% <100.00%> (+0.74%) ⬆️
cli/output/rpc_progress.go 68.75% <0.00%> (-5.80%) ⬇️
commands/instances.go 39.08% <0.00%> (-1.69%) ⬇️
cli/core/search.go 0.00% <0.00%> (ø)
cli/core/update_index.go 0.00% <0.00%> (ø)
commands/daemon/daemon.go 0.00% <0.00%> (ø)
arduino/httpclient/httpclient.go 67.27% <0.00%> (ø)
cli/instance/instance.go 57.69% <0.00%> (+0.41%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@cmaglie cmaglie requested review from per1234 and umbynos September 9, 2022 13:07
@cmaglie cmaglie merged commit 8b52e80 into arduino:master Sep 9, 2022
@cmaglie cmaglie deleted the no_panic_integration_test branch September 9, 2022 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
criticality: low Of low impact priority: high Resolution is a high priority topic: infrastructure Related to project infrastructure type: imperfection Perceived defect in any part of project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants