runtime,runtime/metrics: track on-cpu time per goroutine #51347
Conversation
|
This PR (HEAD: 1d36b93) has been imported to Gerrit for code review. Please visit https://go-review.googlesource.com/c/go/+/387874 to see it. Tip: You can toggle comments from me using the |
irfansharif
force-pushed
the
220224.goroutine-nanos
branch
from
1d36b93 to
50f1cb6
Compare
|
This PR (HEAD: 50f1cb6) has been imported to Gerrit for code review. Please visit https://go-review.googlesource.com/c/go/+/387874 to see it. Tip: You can toggle comments from me using the |
irfansharif
force-pushed
the
220224.goroutine-nanos
branch
from
50f1cb6 to
3b7821c
Compare
|
This PR (HEAD: 3b7821c) has been imported to Gerrit for code review. Please visit https://go-review.googlesource.com/c/go/+/387874 to see it. Tip: You can toggle comments from me using the |
irfansharif
force-pushed
the
220224.goroutine-nanos
branch
from
3b7821c to
231b71c
Compare
|
This PR (HEAD: 231b71c) has been imported to Gerrit for code review. Please visit https://go-review.googlesource.com/c/go/+/387874 to see it. Tip: You can toggle comments from me using the |
|
Message from Michael Pratt: Patch Set 5: Run-TryBot+1 Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Michael Pratt: Patch Set 5: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Gopher Robot: Patch Set 5: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Gopher Robot: Patch Set 5: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Michael Pratt: Patch Set 5: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Gopher Robot: Patch Set 5: TryBot-Result-1 (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
irfansharif
force-pushed
the
220224.goroutine-nanos
branch
from
231b71c to
177a434
Compare
|
This PR (HEAD: 177a434) has been imported to Gerrit for code review. Please visit https://go-review.googlesource.com/c/go/+/387874 to see it. Tip: You can toggle comments from me using the |
irfansharif
force-pushed
the
220224.goroutine-nanos
branch
from
177a434 to
0d55fd1
Compare
|
This PR (HEAD: 0d55fd1) has been imported to Gerrit for code review. Please visit https://go-review.googlesource.com/c/go/+/387874 to see it. Tip: You can toggle comments from me using the |
irfansharif
force-pushed
the
220224.goroutine-nanos
branch
from
0d55fd1 to
75d884d
Compare
|
Message from Irfan Sharif: Patch Set 7: (3 comments) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
This PR (HEAD: 75d884d) has been imported to Gerrit for code review. Please visit https://go-review.googlesource.com/c/go/+/387874 to see it. Tip: You can toggle comments from me using the |
Picks up golang/go#51347 for go1.17.6. Release note: None Release justification: None
Picks up golang/go#51347 for go1.17.6. Release note: None Release justification: None
|
Message from Ian Lance Taylor: Patch Set 8: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Nathan VanBenschoten: Patch Set 8: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Michael Knyszek: Patch Set 8: (4 comments) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Michael Knyszek: Patch Set 8: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
Fixes golang#41554. This commit introduces a /sched/goroutine/running:nanoseconds metric, defined as the total time spent by a goroutine in the running state. This measurement is useful for systems that would benefit from fine-grained CPU attribution. An alternative to scheduler-backed CPU attribution would be the use of profiler labels. Given it's common to spawn multiple goroutines for the same task, goroutine-backed statistics can easily become misleading. Profiler labels instead let you measure CPU performance across a set of cooperating goroutines. That said, it has two downsides: - performance overhead; for high-performance systems that care about fine-grained CPU attribution (databases for e.g. that want to measure total CPU time spent processing each request), profiler labels are too cost-prohibitive, especially given the Go runtime has a much cheaper and more granular view of the data needed - inaccuracy and imprecision, as evaluated in golang#36821 It's worth noting that we already export /sched/latencies:seconds to track scheduling latencies, i.e. time spent by a goroutine in the runnable state (go-review.googlesource.com/c/go/+/308933). This commit does effectively the same except for the running state on the requesting goroutine. Users are free to use this metric to power histograms or tracking on-CPU time across a set of goroutines. Change-Id: Ie78336a3ddeca0521ae29cce57bc7a5ea67da297
irfansharif
force-pushed
the
220224.goroutine-nanos
branch
from
75d884d to
2b9dc3c
Compare
|
Message from Irfan Sharif: Patch Set 8: (5 comments) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
This PR (HEAD: 2b9dc3c) has been imported to Gerrit for code review. Please visit https://go-review.googlesource.com/c/go/+/387874 to see it. Tip: You can toggle comments from me using the |
|
Message from Irfan Sharif: Patch Set 9: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Michael Knyszek: Patch Set 9: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Rhys Hiltner: Patch Set 9: (2 comments) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
Build using golang/go#51347 for go1.17.6 (only for darwin arm64, for now) Release note: None
|
Message from Irfan Sharif: Patch Set 9: (2 comments) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Irfan Sharif: Patch Set 9: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Ian Lance Taylor: Patch Set 9: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
|
Message from Felix Geisendörfer: Patch Set 9: (1 comment) Please don’t reply on this GitHub thread. Visit golang.org/cl/387874. |
Fixes #41554.
This commit introduces a /sched/goroutine/running:nanoseconds metric,
defined as the total time spent by a goroutine in the running state.
This measurement is useful for systems that would benefit from
fine-grained CPU attribution.
An alternative to scheduler-backed CPU attribution would be the use of
profiler labels. Given it's common to spawn multiple goroutines for the
same task, goroutine-backed statistics can easily become misleading.
Profiler labels instead let you measure CPU performance across a set of
cooperating goroutines. That said, it comes with overhead that makes it
unfeasible to always enable. For high-performance systems that care
about fine-grained CPU attribution (databases for e.g. that want to
measure total CPU time spent processing each request), profiler labels
are too cost-prohibitive, especially given the Go runtime has a much
cheaper view of the data needed.
It's worth noting that we already export /sched/latencies:seconds to
track scheduling latencies, i.e. time spent by a goroutine in the
runnable state (go-review.googlesource.com/c/go/+/308933). This commit
does effectively the same, except for the running state. Users are free
to use this metric to power histograms or tracking on-CPU time across a
set of goroutines.