testing: b.Loop prevents range func inlining which drastically changes performance result

[mpx]

Go version

go version devel go1.25-5eaeb7b455 Thu Apr 3 15:36:36 2025 -0700 linux/amd64

Output of go env in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE='on'
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/user/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/user/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1699666494=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/tmp/bench-loop-rangefunc/go.mod'
GOMODCACHE='/home/user/golang/pkg/mod'
GOOS='linux'
GOPATH='/home/user/golang'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org'
GOROOT='/home/user/go-dev'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/home/user/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/home/user/go-dev/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='devel go1.25-5eaeb7b455 Thu Apr 3 15:36:36 2025 -0700'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

Convert an old style benchmark loop containing a range-func loop to the new B.Loop construct.

Example benchmark: https://go.dev/play/p/TMQqkLTVnxB

What did you see happen?

goos: linux
goarch: amd64
pkg: bench
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
BenchmarkIterRangeFunc-8   	307616280	         3.897 ns/op
BenchmarkLoopRangeFunc-8   	13051561	        96.68 ns/op
PASS
ok  	bench	2.861s

What did you expect to see?

The timing result should be roughly similar for both benchmarks, but instead there is a fairly severe performance reduction with B.Loop (25x).

This appears to be a good example of something to fix so B.Loop is more reliable and useful as per #73137 (comment).

[mpx]

Cc @aclements

[prattmic]

cc @JunyangShao @dr2chase @golang/runtime

[aclements]

Thanks. This is a good example of the problem in #73137. Let's track the work in that issue.

[gabyhelp]

Related Issues

• testing: loosen description of how b.Loop works today and ideally avoid forcing heap allocs in future #73137
• testing: clarify that b.N does not change during execution of the benchmark function #67137 (closed)
• cmd/6g: range loops are slower than handmade loops #3909 (closed)
• testing: b.StopTimer breaks b.Loop #72922 (closed)
• testing: add testing.B.Loop for iteration #61515 (closed)
• range based for loop slower than standard for loop on array #16907 (closed)
• testing: improve the text of the b.Loop documentation #70787 (closed)
• testing: early break in a testing.B.Loop benchmark loop should be reported. #72933 (closed)
• cmd/compile: inline is slower #51028 (closed)

Related Discussions

• Go 1.24 BLooper

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}