Unit tests are not working locally (go 1.17.5)

[camilamacedo86]

What broke? What's expected?

The make test is not working locally
That fails when we run go test -race ./pkg/...

Reproducing this issue

It was reported on the channel that someone was not able to run the tests.
the tests are called in the CI and works see: https://github.com/kubernetes-sigs/kubebuilder/runs/5605172452?check_suite_focus=true#step:5:1

However, I tried to run with;

• Mac os (darwin_amd64)
• Go 1.17.5

And also faced issues:

--- FAIL: TestNewCommand (0.03s)
    --- FAIL: TestNewCommand/testdata/componentconfig (0.02s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4e258ea]

goroutine 23 [running]:
testing.tRunner.func1.2({0x4f3a160, 0x59b1970})
	/usr/local/Cellar/go/1.17.5/libexec/src/testing/testing.go:1209 +0x36c
testing.tRunner.func1()
	/usr/local/Cellar/go/1.17.5/libexec/src/testing/testing.go:1212 +0x3b6
panic({0x4f3a160, 0x59b1970})
	/usr/local/Cellar/go/1.17.5/libexec/src/runtime/panic.go:1047 +0x266
sigs.k8s.io/kubebuilder/v3/pkg/cli/alpha/config-gen.ControllerGenFilter.Filter({0x0}, {0x0, 0x0, 0x4071325})
	/Users/camilamacedo86/go/src/sigs.k8s.io/kubebuilder/pkg/cli/alpha/config-gen/controller-gen-filter.go:68 +0x58a
sigs.k8s.io/kustomize/kyaml/fn/framework.(*TemplateProcessor).doPreProcess(0xc00016cfc0, {0x0, 0x0, 0x0})
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/fn/framework/processors.go:280 +0x17b
sigs.k8s.io/kustomize/kyaml/kio.FilterFunc.Filter(0xc000427ae0, {0x0, 0x0, 0x0})
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/kio/kio.go:60 +0x52
sigs.k8s.io/kustomize/kyaml/kio.Pipeline.ExecuteWithCallback({{0xc0004a5178, 0x1, 0x1}, {0xc0004a51f0, 0x5, 0x5}, {0xc0004a5168, 0x1, 0x1}, 0x1}, ...)
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/kio/kio.go:120 +0x32b
sigs.k8s.io/kustomize/kyaml/kio.Pipeline.Execute(...)
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/kio/kio.go:93
sigs.k8s.io/kustomize/kyaml/fn/framework.TemplateProcessor.Filter({{0x5004bc0, 0xc0001bcc00}, {0xc0003cdcc0, 0x1, 0x1}, {0xc000473710, 0x3, 0x3}, 0x1, {0xc0003cdca0, ...}, ...}, ...)
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/fn/framework/processors.go:245 +0x73a
sigs.k8s.io/kustomize/kyaml/fn/framework.(*ResourceList).Filter(0xc000473b00, {0x5239740, 0xc00016cf30})
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/fn/framework/framework.go:144 +0x74
sigs.k8s.io/kustomize/kyaml/fn/framework.TemplateProcessor.Process({{0x5004bc0, 0xc0001bcc00}, {0xc0003cdcc0, 0x1, 0x1}, {0xc000473710, 0x3, 0x3}, 0x1, {0xc0003cdca0, ...}, ...}, ...)
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/fn/framework/processors.go:259 +0x138
sigs.k8s.io/kustomize/kyaml/fn/framework.Execute({0x5239720, 0xc00016ccf0}, 0xc000195970)
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/fn/framework/framework.go:118 +0x39e
sigs.k8s.io/kustomize/kyaml/fn/framework/command.Build.func1(0xc00047a280, {0xc000427530, 0x1, 0x1})
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/fn/framework/command/command.go:101 +0x40b
github.com/spf13/cobra.(*Command).execute(0xc00047a280, {0xc0004273b0, 0x1, 0x1})
	/Users/camilamacedo86/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:856 +0xa7e
github.com/spf13/cobra.(*Command).ExecuteC(0xc00047a280)
	/Users/camilamacedo86/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:974 +0x5db
github.com/spf13/cobra.(*Command).Execute(...)
	/Users/camilamacedo86/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:902
sigs.k8s.io/kustomize/kyaml/fn/framework/frameworktestutil.(*CommandResultsChecker).Assert.func1()
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/fn/framework/frameworktestutil/frameworktestutil.go:114 +0x505
sigs.k8s.io/kustomize/kyaml/fn/framework/frameworktestutil.(*resultsChecker).runDirectoryTestCase.func2(0xc000423d40)
	/Users/camilamacedo86/go/pkg/mod/sigs.k8s.io/kustomize/kyaml@v0.10.21/fn/framework/frameworktestutil/frameworktestutil.go:303 +0xe2
testing.tRunner(0xc000423d40, 0xc0004800c0)
	/usr/local/Cellar/go/1.17.5/libexec/src/testing/testing.go:1259 +0x230
created by testing.(*T).Run
	/usr/local/Cellar/go/1.17.5/libexec/src/testing/testing.go:1306 +0x727

then, we need to check why to solve it or add the guidance in the Contribution doc

KubeBuilder (CLI) Version

master

PROJECT version

No response

Plugin versions

No response

Other versions

No response

Extra Labels

/kind regression

[Abhra303]

Hey @camilamacedo86 I would like to solve it.

Thanks :)

[Abhra303]

Excuse me, I couldn't reproduce the problem you faced. I followed the same procedure you described, but in my case, all tests are passing. Below is the output of the command go test -race ./pkg/... -

ok  	sigs.k8s.io/kubebuilder/v3/pkg/cli	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/cli/alpha/config-gen	(cached)
?   	sigs.k8s.io/kubebuilder/v3/pkg/cli/alpha/config-gen/examples/advancedextension	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/cli/alpha/config-gen/examples/basicextension	[no test files]
ok  	sigs.k8s.io/kubebuilder/v3/pkg/config	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/config/store	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/config/store/yaml	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/config/v2	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/config/v3	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/internal/validation	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/machinery	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/model/resource	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/model/stage	(cached)
ok  	sigs.k8s.io/kubebuilder/v3/pkg/plugin	(cached)
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugin/external	[no test files]
ok  	sigs.k8s.io/kubebuilder/v3/pkg/plugin/util	(cached)
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/certmanager	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/crd	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/crd/patches	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/kdefault	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/manager	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/prometheus	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/rbac	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/samples	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v1/scaffolds/internal/templates/config/webhook	[no test files]
ok  	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang	(cached)
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/declarative/v1	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/declarative/v1/scaffolds	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/declarative/v1/scaffolds/internal/templates	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/api	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/certmanager	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/crd	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/crd/patches	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/kdefault	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/manager	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/prometheus	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/rbac	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/samples	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/config/webhook	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/controllers	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v2/scaffolds/internal/templates/hack	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3/scaffolds	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3/scaffolds/internal/templates	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3/scaffolds/internal/templates/api	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3/scaffolds/internal/templates/controllers	[no test files]
?   	sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3/scaffolds/internal/templates/hack	[no test files]

My go version is 1.17.6 though.

[Abhra303]

Following the output you provided in the issue description, the problem is occurring at line 68 of ./pkg/cli/alpha/config-gen/controller-gen-filter.go. Most probably because rt is nil for your case. It is trying to assign a value to OutputRules of nil rt.

[camilamacedo86]

@Abhra303,

Yes. I see many issues here:
a) If we add an if there the tests still failing
b) The config-gen is adding deps in kb which are not necessary for the tool
c) its tests do not follows the kb stack/standards
d) that seems it fails only on mac

However, I found the issue here already.
So, I am using the fix with additional changes.

[Abhra303]

d) that seems it fails only on Mac

Not really, I am using Mac and it is passing for me locally.

Here is the info -

abhra303@Abhradeeps-MacBook-Pro ~ % uname -m
arm64
abhra303@Abhradeeps-MacBook-Pro ~ % uname -s
Darwin
abhra303@Abhradeeps-MacBook-Pro ~ % sysctl -n machdep.cpu.brand_string
Apple M1

[camilamacedo86]

HI @Abhra303,

If I remove the race conditional option such as : https://github.com/kubernetes-sigs/kubebuilder/pull/2581/files it works for me locally, would you like to help us investigate why that occurs?

My go env:

h$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/camilamacedo86/Library/Caches/go-build"
GOENV="/Users/camilamacedo86/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/camilamacedo86/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/camilamacedo86/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.17.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.17.5/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.5"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/camilamacedo86/go/src/github.com/operator-framework/audit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/xw/pqcwslld4fzdn81yh9dylz6m0000gn/T/go-build3894865026=/tmp/go-build -gno-record-gcc-switches -fno-common"