Skip to content

Out Of Memory errors when using the unused linter #898

New issue
New issue
Closed
Closed
Out Of Memory errors when using the unused linter#898
Labels
bugSomething isn't workingtopic: memoryHuge memory consumption
@katbyte

Description

@katbyte
katbyte
opened on Dec 26, 2019

Thank you for creating the issue!

  • [*] Yes, I'm using a binary release within 2 latest major releases. Only such installations are supported.
  • [*] Yes, I've searched similar issues on GitHub and didn't find any.
  • [*] Yes, I've included all information below (version, config, etc).

We recently split our project up into many many sub packages (hashicorp/terraform-provider-azurerm#5226) and afterwards golang-ci started to fail on travis with out of memory errors. I managed to track it down to the unused linter and disabling it causes our build to pass again.

Please include the following information:

Version of golangci-lint running in travis - installs latest version via: ``` GO111MODULE=off go get -u github.com/golangci/golangci-lint/cmd/golangci-lint ```
Config file

running unused via cmd line:

lintunused:
	@echo "==> Checking source code against static check linters..."
	(while true; do sleep 300; echo "(I'm still alive and linting!)"; done) & PID=$$!; echo $$PID; \
	golangci-lint run ./... -v --no-config --concurrency 1 --deadline=30m10s --disable-all --enable=unused; ES=$$?; kill -9 $$PID; exit $$ES
Verbose output of running

travis log:

hostname: 45577bf6-a395-4e53-b7a0-9e4b8c8ff744@1.worker-org-85d846cc5-jtjxq.gce-production-1
version: v6.2.6 https://github.com/travis-ci/worker/tree/ba21bd30589fd152126e13df30e0cc78ccdf2837
instance: travis-job-17c3d4c1-ec5a-4057-9c14-e3465b7ce123 travis-ci-sardonyx-xenial-1553530528-f909ac5 (via amqp)
startup: 6.338160567s
0.14s0.01s0.00s0.01s
system_info
Build system information
Build language: go
Build group: stable
Build dist: xenial
Build id: 629500786
Job id: 629500789
Runtime kernel version: 4.15.0-1028-gcp
travis-build version: 2dca56efb
Build image provisioning date and time
Mon Mar 25 16:43:24 UTC 2019
Operating System Details
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.6 LTS
Release:	16.04
Codename:	xenial
Systemd Version
systemd 229
Cookbooks Version
42e42e4 https://github.com/travis-ci/travis-cookbooks/tree/42e42e4
git version
git version 2.21.0
bash version
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)
gcc version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
docker version
Client:
 Version:           18.06.0-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        0ffa825
 Built:             Wed Jul 18 19:11:02 2018
 OS/Arch:           linux/amd64
 Experimental:      false
Server:
 Engine:
  Version:          18.06.0-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       0ffa825
  Built:            Wed Jul 18 19:09:05 2018
  OS/Arch:          linux/amd64
  Experimental:     false
clang version
clang version 7.0.0 (tags/RELEASE_700/final)
jq version
jq-1.5
bats version
Bats 0.4.0
shellcheck version
0.6.0
shfmt version
v2.6.3
ccache version
3.2.4
cmake version
cmake version 3.12.4
heroku version
heroku/7.22.7 linux-x64 node-v11.10.1
imagemagick version
Version: ImageMagick 6.8.9-9 Q16 x86_64 2018-09-28 http://www.imagemagick.org
md5deep version
4.4
mercurial version
version 4.8
mysql version
mysql  Ver 14.14 Distrib 5.7.25, for Linux (x86_64) using  EditLine wrapper
openssl version
OpenSSL 1.0.2g  1 Mar 2016
packer version
1.3.3
postgresql client version
psql (PostgreSQL) 10.7 (Ubuntu 10.7-1.pgdg16.04+1)
ragel version
Ragel State Machine Compiler version 6.8 Feb 2013
sudo version
1.8.16
gzip version
gzip 1.6
zip version
Zip 3.0
vim version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Nov 24 2016 16:44:48)
iptables version
iptables v1.6.0
curl version
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
wget version
GNU Wget 1.17.1 built on linux-gnu.
rsync version
rsync  version 3.1.1  protocol version 31
gimme version
v1.5.3
nvm versionm
0.34.0
perlbrew version
/home/travis/perl5/perlbrew/bin/perlbrew  - App::perlbrew/0.86
phpenv version
rbenv 1.1.2
rvm version
rvm 1.29.7 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]
default ruby version
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]
CouchDB version
couchdb 1.6.1
ElasticSearch version
5.5.0
Installed Firefox version
firefox 63.0.1
MongoDB version
MongoDB 4.0.7
PhantomJS version
2.1.1
Pre-installed PostgreSQL versions
9.4.21
9.5.16
9.6.12
Redis version
redis-server 5.0.4
Pre-installed Go versions
1.11.1
mvn version
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z)
gradle version
Gradle 4.10.2!
lein version
Leiningen 2.9.1 on Java 11.0.2 OpenJDK 64-Bit Server VM
Pre-installed Node.js versions
v10.15.3
v11.0.0
v4.9.1
v6.17.0
v8.12.0
v8.15.1
v8.9
phpenv versions
  system
  5.6
  5.6.40
  7.1
  7.1.27
  7.2
* 7.2.15 (set by /home/travis/.phpenv/version)
  hhvm
  hhvm-stable
composer --version
Composer version 1.8.4 2019-02-11 10:52:10
Pre-installed Ruby versions
ruby-2.3.8
ruby-2.4.5
ruby-2.5.3
0.01s0.30s0.17s0.05s0.00s0.04s0.00s0.01s0.01s0.01s0.01s0.01s0.00s0.00s0.02s0.00s0.01s0.27s0.00s0.00s0.00s0.01s0.00s0.10s0.00s0.72s0.00s0.00s6.03s0.00s2.28s0.00s2.19s
docker_mtu
resolvconf
services
3.01s$ sudo systemctl start docker
git.checkout
5.98s$ git clone --depth=50 https://github.com/terraform-providers/terraform-provider-azurerm.git terraform-providers/terraform-provider-azurerm
$ travis_export_go 1.13.x github.com/terraform-providers/terraform-provider-azurerm
0.01s
Setting environment variables from .travis.yml
$ export GOFLAGS=-mod=vendor
7.62s$ travis_setup_go
go version go1.13.5 linux/amd64
$ export GOPATH="/home/travis/gopath"
$ export PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.13.5.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin"
$ export GO111MODULE="auto"
$ gimme version
v1.5.3
$ go version
go version go1.13.5 linux/amd64
go.env
$ go env
install.1
0.83s$ bash scripts/gogetcookie.sh
install.2
33.92s$ make tools
1014.17s$ GOGC=5 make lintunused
==> Checking source code against static check linters...
(while true; do sleep 300; echo "(I'm still alive and linting!)"; done) & PID=$!; echo $PID; \
golangci-lint run ./... -v --no-config --concurrency 1 --deadline=30m10s --disable-all --enable=unused; ES=$?; kill -9 $PID; exit $ES
5794
INFO [lintersdb] Active 1 linters: [unused]       
INFO [lintersdb] Active 1 linters: [unused]       
(I'm still alive and linting!)
(I'm still alive and linting!)
INFO [loader] Go packages loading at mode 575 (types_sizes|files|imports|name|compiled_files|deps|exports_file) took 12m42.535108738s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 1.043035847s 
(I'm still alive and linting!)
fatal error: runtime: out of memory
runtime stack:
runtime.throw(0xf80127, 0x16)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/panic.go:774 +0x72
runtime.sysMap(0xc1c0000000, 0x4000000, 0x192c158)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/mem_linux.go:169 +0xc5
runtime.(*mheap).sysAlloc(0x1909b40, 0xa000, 0x45a4f0, 0x7f51112fb2a0)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/malloc.go:701 +0x1cd
runtime.(*mheap).grow(0x1909b40, 0x5, 0xffffffff)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/mheap.go:1255 +0xa3
runtime.(*mheap).allocSpanLocked(0x1909b40, 0x5, 0x192c168, 0x41fc35)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/mheap.go:1170 +0x266
runtime.(*mheap).alloc_m(0x1909b40, 0x5, 0x7f510f1b0100, 0x7f510f1b9618)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/mheap.go:1022 +0xc2
runtime.(*mheap).alloc.func1()
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/mheap.go:1093 +0x4c
runtime.(*mheap).alloc(0x1909b40, 0x5, 0x10100, 0x200)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/mheap.go:1092 +0x8a
runtime.largeAlloc(0x9f60, 0x7f512fd60001, 0x45c13a)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/malloc.go:1138 +0x97
runtime.mallocgc.func1()
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/malloc.go:1033 +0x46
runtime.systemstack(0x7f51306cbb80)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/asm_amd64.s:370 +0x66
runtime.mstart()
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/proc.go:1146
goroutine 8286 [running]:
runtime.systemstack_switch()
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/asm_amd64.s:330 fp=0xc002647510 sp=0xc002647508 pc=0x45ded0
runtime.mallocgc(0x9f60, 0xe55e20, 0xdc9501, 0x20)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/malloc.go:1032 +0x895 fp=0xc0026475b0 sp=0xc002647510 pc=0x40e555
runtime.newarray(0xe55e20, 0x96, 0xc1bfff9180)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/malloc.go:1173 +0x63 fp=0xc0026475e0 sp=0xc0026475b0 pc=0x40e993
runtime.makeBucketArray(0xdc5500, 0xc002647707, 0x0, 0xea8cf82cc9ceefe0, 0xee17a0)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/map.go:362 +0x183 fp=0xc002647618 sp=0xc0026475e0 pc=0x40f883
runtime.hashGrow(0xdc5500, 0xc177e7cdb0)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/map.go:1033 +0x89 fp=0xc002647668 sp=0xc002647618 pc=0x411379
runtime.mapassign(0xdc5500, 0xc177e7cdb0, 0xc0026477e8, 0x15)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/map.go:652 +0x140 fp=0xc0026476f0 sp=0xc002647668 pc=0x4101d0
github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/ssa.memberFromObject(0xc170b1eae0, 0x11100c0, 0xc0063e5450, 0x0, 0x0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/ssa/create.go:102 +0x7b7 fp=0xc002647808 sp=0xc0026476f0 pc=0xb5a1a7
github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/ssa.(*Program).CreatePackage(0xc12d2a6dc0, 0xc0062ce280, 0x0, 0x0, 0x0, 0x0, 0xc1bff70f01, 0x17fb080)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/ssa/create.go:205 +0x616 fp=0xc0026478d8 sp=0xc002647808 pc=0xb5b1c6
github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa.run.func1(0xc01d027cd0, 0x2, 0x2)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa/buildssa.go:61 +0x103 fp=0xc002647950 sp=0xc0026478d8 pc=0xb84743
github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa.run.func1(0xc022f5e800, 0x44, 0x80)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa/buildssa.go:62 +0x12f fp=0xc0026479c8 sp=0xc002647950 pc=0xb8476f
github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa.run.func1(0xc0d66a6800, 0x4b, 0x80)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa/buildssa.go:62 +0x12f fp=0xc002647a40 sp=0xc0026479c8 pc=0xb8476f
github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa.run.func1(0xc115ec7b80, 0x10, 0x10)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa/buildssa.go:62 +0x12f fp=0xc002647ab8 sp=0xc002647a40 pc=0xb8476f
github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa.run.func1(0xc1723f5a80, 0xb, 0x10)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa/buildssa.go:62 +0x12f fp=0xc002647b30 sp=0xc002647ab8 pc=0xb8476f
github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa.run(0xc12d2a6d20, 0xec024c4c80, 0x1903320, 0xc0c6171c48, 0x2)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/honnef.co/go/tools/internal/passes/buildssa/buildssa.go:66 +0x21e fp=0xc002647cd8 sp=0xc002647b30 pc=0xb841ee
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*action).analyze(0xc001c335d0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:568 +0xa94 fp=0xc002647e90 sp=0xc002647cd8 pc=0x9d5624
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*action).analyzeSafe.func2()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:490 +0x2a fp=0xc002647ea8 sp=0xc002647e90 pc=0x9de29a
github.com/golangci/golangci-lint/pkg/timeutils.(*Stopwatch).TrackStage(0xc001b93f90, 0xf09139, 0x8, 0xc001c6d760)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/timeutils/stopwatch.go:111 +0x50 fp=0xc002647f08 sp=0xc002647ea8 pc=0x951b30
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*action).analyzeSafe(0xc001c335d0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:489 +0x9e fp=0xc002647f80 sp=0xc002647f08 pc=0x9d4b5e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze.func3(0xc16fcbb4d0, 0xc001c335d0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1034 +0x69 fp=0xc002647fd0 sp=0xc002647f80 pc=0x9de8f9
runtime.goexit()
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc002647fd8 sp=0xc002647fd0 pc=0x45ffa1
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1029 +0x31b
goroutine 1 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc000de82e8)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc000de82e0)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/waitgroup.go:130 +0x64
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze(0xc0000d9d10, 0xc0002e6000, 0x121, 0x200, 0xc0000106a0, 0x1, 0x1, 0xc0002e6000, 0x10f, 0xdc2500)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:323 +0x668
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).run(0xc0000d9d10, 0xc0000106a0, 0x1, 0x1, 0xc0002e6000, 0x121, 0x200, 0x0, 0x0, 0x0, ...)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:112 +0x1d6
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.runAnalyzers(0x11093a0, 0xc00031d9e0, 0xc000691340, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/linter.go:455 +0x5ac
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*Linter).Run(0xc00031d9e0, 0x10fdf20, 0xc00008e360, 0xc000691340, 0xc000306780, 0x6, 0x6, 0x2bd, 0x0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/linter.go:505 +0x97
github.com/golangci/golangci-lint/pkg/lint.(*Runner).runLinterSafe(0xc00106f940, 0x10fdf20, 0xc00008e360, 0xc000127c00, 0xc00031f420, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/lint/runner.go:116 +0x192
github.com/golangci/golangci-lint/pkg/lint.Runner.Run.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/lint/runner.go:186 +0x7c
github.com/golangci/golangci-lint/pkg/timeutils.(*Stopwatch).TrackStage(0xc001b93f40, 0xefc5b4, 0x6, 0xc00106f8f8)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/timeutils/stopwatch.go:111 +0x50
github.com/golangci/golangci-lint/pkg/lint.Runner.Run(0xc0000fc360, 0x11, 0x11, 0x11095e0, 0xc001b90570, 0x10fdf20, 0xc00008e360, 0xc000010ef0, 0x1, 0x1, ...)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/lint/runner.go:185 +0x27b
github.com/golangci/golangci-lint/pkg/commands.(*Executor).runAnalysis(0xc000121100, 0x10fdf20, 0xc00008e360, 0xc00011d500, 0x1, 0x8, 0x5e0220, 0x0, 0x0, 0xc0002e3c98, ...)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/commands/run.go:318 +0x69b
github.com/golangci/golangci-lint/pkg/commands.(*Executor).runAndPrint(0xc000121100, 0x10fdf20, 0xc00008e360, 0xc00011d500, 0x1, 0x8, 0x0, 0x0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/commands/run.go:355 +0x1a5
github.com/golangci/golangci-lint/pkg/commands.(*Executor).executeRun(0xc000121100, 0xc000370000, 0xc00011d500, 0x1, 0x8)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/commands/run.go:418 +0x1b4
github.com/golangci/golangci-lint/vendor/github.com/spf13/cobra.(*Command).execute(0xc000370000, 0xc00011d480, 0x8, 0x8, 0xc000370000, 0xc00011d480)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/github.com/spf13/cobra/command.go:830 +0x2aa
github.com/golangci/golangci-lint/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc000129b80, 0x6, 0xef62a1, 0x1)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/github.com/spf13/cobra/command.go:914 +0x2fb
github.com/golangci/golangci-lint/vendor/github.com/spf13/cobra.(*Command).Execute(...)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/vendor/github.com/spf13/cobra/command.go:864
github.com/golangci/golangci-lint/pkg/commands.(*Executor).Execute(...)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/commands/executor.go:149
main.main()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/cmd/golangci-lint/main.go:20 +0x7e
goroutine 10 [semacquire]:
runtime.ReadMemStats(0xc002642778)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/mstats.go:446 +0x36
github.com/golangci/golangci-lint/pkg/commands.watchResources.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/commands/run.go:479 +0xb5
github.com/golangci/golangci-lint/pkg/commands.watchResources(0x10fdf20, 0xc00008e360, 0xc00009a060, 0x1109460, 0xc0003065d0, 0xfbfea0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/commands/run.go:495 +0x1c5
created by github.com/golangci/golangci-lint/pkg/commands.(*Executor).executeRun
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/commands/run.go:415 +0x39b
goroutine 9 [syscall, 16 minutes]:
os/signal.signal_recv(0x0)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/sigqueue.go:147 +0x9c
os/signal.loop()
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/os/signal/signal_unix.go:29 +0x41
goroutine 437 [chan send, 1 minutes]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bbecc0, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1003 +0x4d
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bbed0c, 0xc001c52768)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bbecc0, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bbecc0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
goroutine 396 [chan send, 1 minutes]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bdcc60, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1003 +0x4d
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bdccac, 0xc001be1f68)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bdcc60, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bdcc60)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
goroutine 465 [chan send, 1 minutes]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bbf1a0, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1003 +0x4d
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bbf1ec, 0xc001c68768)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bbf1a0, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bbf1a0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
goroutine 468 [chan send, 1 minutes]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bdcea0, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1003 +0x4d
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bdceec, 0xc001c69f68)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bdcea0, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bdcea0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
goroutine 471 [chan send, 1 minutes]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bbfb60, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1003 +0x4d
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bbfbac, 0xc001c63768)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bbfb60, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bbfb60)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
goroutine 474 [chan send, 1 minutes]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bdd3e0, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1003 +0x4d
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bdd42c, 0xc001c64f68)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bdd3e0, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bdd3e0)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
goroutine 476 [chan send, 1 minutes]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bdc960, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1003 +0x4d
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bdc9ac, 0xc001c65f68)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bdc960, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bdc960)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
goroutine 8285 [chan receive]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*action).waitUntilDependingAnalyzersWorked(0xc001c33540)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:469 +0x6c
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze.func3(0xc16fcbb4d0, 0xc001c33540)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1032 +0x5b
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1029 +0x31b
goroutine 492 [semacquire]:
sync.runtime_Semacquire(0xc16fcbb4d8)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc16fcbb4d0)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/waitgroup.go:130 +0x64
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bbf140, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1037 +0x347
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bbf18c, 0xc001c6df68)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bbf140, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bbf140)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
goroutine 493 [chan send, 1 minutes]:
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze(0xc000bbfe00, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:1003 +0x4d
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive.func1()
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:998 +0x159
sync.(*Once).doSlow(0xc000bbfe4c, 0xc001c76768)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:66 +0xe3
sync.(*Once).Do(...)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/sync/once.go:57
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyzeRecursive(0xc000bbfe00, 0x3, 0xc0019de420)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:987 +0x7e
github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze.func2(0xc0000d9d10, 0xc0019de420, 0xc000de82e0, 0xc000bbfe00)
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:318 +0x43
created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*runner).analyze
	/home/travis/gopath/src/github.com/golangci/golangci-lint/pkg/golinters/goanalysis/runner.go:317 +0x652
GNUmakefile:51: recipe for target 'lintunused' failed
make: *** [lintunused] Error 2
The command "GOGC=5 make lintunused" exited with 2.
Done. Your build exited with 1.

Activity

katbyte
mentioned this on Dec 26, 2019
tpounds
added
bugSomething isn't working
on Dec 30, 2019
ernado
added
topic: memoryHuge memory consumption
on Jan 7, 2020
jirfag

jirfag commented on Jan 19, 2020

@jirfag
jirfag
on Jan 19, 2020
Contributor

hi,
I'm sorry but I have no idea how to fix it in golangci-lint. I guess it can be optimized only in unused itself.

katbyte

katbyte commented on Jan 26, 2020

@katbyte
katbyte
on Jan 26, 2020
Author

that sounds accurate, do you know where i should open an issue for that?

ernado
mentioned this on Apr 21, 2020
SVilgelm

SVilgelm commented on Jul 12, 2020

@SVilgelm
SVilgelm
on Jul 12, 2020
Member

So I run the golangci-lint on terraform-providers repo.
Here is unused:

20-07-12 18:00 % ~/go/bin/golangci-lint run ./... -v --no-config --concurrency 1 --deadline=30m10s --disable-all --enable=unused
INFO [lintersdb] Active 1 linters: [unused]       
INFO [loader] Go packages loading at mode 575 (deps|imports|name|types_sizes|compiled_files|exports_file|files) took 1m36.639162704s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 1.483863404s 
INFO [linters context/goanalysis] analyzers took 2m7.553156126s with top 10 stages: buildir: 2m4.692773943s, U1000: 2.860382183s 
INFO [runner] Issues before processing: 3, after processing: 0 
INFO [runner] Processors filtering stat (out/in): path_prettifier: 3/3, skip_files: 3/3, nolint: 0/3, filename_unadjuster: 3/3, autogenerated_exclude: 3/3, identifier_marker: 3/3, cgo: 3/3, skip_dirs: 3/3, exclude: 3/3, exclude-rules: 3/3 
INFO [runner] processing took 11.459124ms with stages: nolint: 8.851058ms, identifier_marker: 606.827µs, exclude: 554.16µs, autogenerated_exclude: 462.103µs, path_prettifier: 325.275µs, max_same_issues: 299.629µs, filename_unadjuster: 224.435µs, skip_dirs: 122.682µs, cgo: 8.867µs, max_from_linter: 779ns, uniq_by_line: 625ns, diff: 469ns, skip_files: 431ns, source_code: 353ns, severity-rules: 342ns, exclude-rules: 318ns, path_shortener: 305ns, max_per_file_from_linter: 248ns, path_prefixer: 218ns 
INFO [runner] linters took 2m35.450260507s with stages: unused: 2m35.438323072s 
INFO File cache stats: 0 entries of total size 0B 
INFO Memory: 1850 samples, avg is 555.0MB, max is 1632.8MB 
INFO Execution took 4m13.908521158s

gosec:

20-07-12 18:37 % ~/go/bin/golangci-lint run ./... -v --no-config --concurrency 1 --deadline=30m10s --disable-all --enable=gosec 
INFO [lintersdb] Active 1 linters: [gosec]        
INFO [loader] Go packages loading at mode 575 (deps|exports_file|name|compiled_files|files|imports|types_sizes) took 2.921628313s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 614.832649ms 
INFO [linters context/goanalysis] analyzers took 2.421873466s with top 10 stages: gosec: 2.421873466s 
INFO [runner/max_same_issues] 11/14 issues with text "G201: SQL string formatting" were hidden, use --max-same-issues 
INFO [runner/max_same_issues] 9/12 issues with text "G601: Implicit memory aliasing in for loop." were hidden, use --max-same-issues 
INFO [runner/max_same_issues] 6/9 issues with text "G109: Potential Integer overflow made by strconv.Atoi result conversion to int16/32" were hidden, use --max-same-issues 
INFO [runner] Issues before processing: 4013, after processing: 14 
INFO [runner] Processors filtering stat (out/in): filename_unadjuster: 4013/4013, skip_dirs: 4013/4013, exclude: 40/4013, diff: 40/40, source_code: 14/14, severity-rules: 14/14, path_prefixer: 14/14, autogenerated_exclude: 4013/4013, nolint: 40/40, uniq_by_line: 40/40, skip_files: 4013/4013, identifier_marker: 4013/4013, exclude-rules: 40/40, max_per_file_from_linter: 40/40, max_same_issues: 14/40, path_shortener: 14/14, cgo: 4013/4013, path_prettifier: 4013/4013, max_from_linter: 14/14 
INFO [runner] processing took 115.606232ms with stages: identifier_marker: 27.145669ms, path_prettifier: 27.139709ms, nolint: 24.285469ms, autogenerated_exclude: 24.016409ms, exclude: 9.289677ms, skip_dirs: 2.083048ms, source_code: 986.118µs, cgo: 403.41µs, filename_unadjuster: 163.896µs, max_same_issues: 50.18µs, uniq_by_line: 23.706µs, max_per_file_from_linter: 7.423µs, path_shortener: 6.589µs, max_from_linter: 3.277µs, diff: 691ns, skip_files: 315ns, exclude-rules: 291ns, severity-rules: 225ns, path_prefixer: 130ns 
INFO [runner] linters took 9.274870576s with stages: gosec: 9.15905949s 
INFO File cache stats: 11 entries of total size 208.8KiB 
INFO Memory: 121 samples, avg is 503.5MB, max is 937.1MB 
INFO Execution took 12.857378952s

What I can see, the unused linter takes more memory, but I cannot say that gosec consumes 10 times less than unused. So I thin the unused improved their memory consumption and we can close this issue

SVilgelm
closed this as completedon Jul 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtopic: memoryHuge memory consumption

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @tpounds@SVilgelm@ernado@katbyte@jirfag

        Issue actions
          Out Of Memory errors when using the unused linter · Issue #898 · golangci/golangci-lint