-
Notifications
You must be signed in to change notification settings - Fork 312
Fix flaky out of order comparison for http client test #9158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 9 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.52.0-SNAPSHOT~88a7bbba8b, baseline=1.52.0-SNAPSHOT~ad6d6b9786
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (994.681 ms) : 0, 994681
Total [baseline] (10.645 s) : 0, 10644774
Agent [candidate] (1.001 s) : 0, 1001418
Total [candidate] (10.693 s) : 0, 10693095
section appsec
Agent [baseline] (1.181 s) : 0, 1181009
Total [baseline] (10.767 s) : 0, 10767428
Agent [candidate] (1.186 s) : 0, 1186399
Total [candidate] (10.759 s) : 0, 10758633
section iast
Agent [baseline] (1.135 s) : 0, 1135288
Total [baseline] (10.895 s) : 0, 10895093
Agent [candidate] (1.137 s) : 0, 1136654
Total [candidate] (10.881 s) : 0, 10881130
section profiling
Agent [baseline] (1.27 s) : 0, 1269613
Total [baseline] (11.037 s) : 0, 11037423
Agent [candidate] (1.243 s) : 0, 1242918
Total [candidate] (10.977 s) : 0, 10977351
gantt
title petclinic - break down per module: candidate=1.52.0-SNAPSHOT~88a7bbba8b, baseline=1.52.0-SNAPSHOT~ad6d6b9786
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (686.62 ms) : 0, 686620
BytebuddyAgent [candidate] (691.715 ms) : 0, 691715
GlobalTracer [baseline] (242.271 ms) : 0, 242271
GlobalTracer [candidate] (243.592 ms) : 0, 243592
AppSec [baseline] (30.157 ms) : 0, 30157
AppSec [candidate] (30.317 ms) : 0, 30317
Debugger [baseline] (5.962 ms) : 0, 5962
Debugger [candidate] (6.015 ms) : 0, 6015
Remote Config [baseline] (677.899 µs) : 0, 678
Remote Config [candidate] (674.292 µs) : 0, 674
Telemetry [baseline] (8.175 ms) : 0, 8175
Telemetry [candidate] (8.257 ms) : 0, 8257
section appsec
BytebuddyAgent [baseline] (714.369 ms) : 0, 714369
BytebuddyAgent [candidate] (717.865 ms) : 0, 717865
GlobalTracer [baseline] (236.336 ms) : 0, 236336
GlobalTracer [candidate] (237.025 ms) : 0, 237025
IAST [baseline] (23.763 ms) : 0, 23763
IAST [candidate] (23.941 ms) : 0, 23941
AppSec [baseline] (171.215 ms) : 0, 171215
AppSec [candidate] (172.018 ms) : 0, 172018
Debugger [baseline] (5.709 ms) : 0, 5709
Debugger [candidate] (5.711 ms) : 0, 5711
Remote Config [baseline] (602.942 µs) : 0, 603
Remote Config [candidate] (612.106 µs) : 0, 612
Telemetry [baseline] (8.034 ms) : 0, 8034
Telemetry [candidate] (8.051 ms) : 0, 8051
section iast
BytebuddyAgent [baseline] (809.045 ms) : 0, 809045
BytebuddyAgent [candidate] (810.229 ms) : 0, 810229
GlobalTracer [baseline] (232.95 ms) : 0, 232950
GlobalTracer [candidate] (233.216 ms) : 0, 233216
IAST [baseline] (26.656 ms) : 0, 26656
IAST [candidate] (25.718 ms) : 0, 25718
AppSec [baseline] (31.383 ms) : 0, 31383
AppSec [candidate] (31.381 ms) : 0, 31381
Debugger [baseline] (5.776 ms) : 0, 5776
Debugger [candidate] (6.714 ms) : 0, 6714
Remote Config [baseline] (592.421 µs) : 0, 592
Remote Config [candidate] (591.399 µs) : 0, 591
Telemetry [baseline] (8.016 ms) : 0, 8016
Telemetry [candidate] (7.983 ms) : 0, 7983
section profiling
BytebuddyAgent [baseline] (691.979 ms) : 0, 691979
BytebuddyAgent [candidate] (676.904 ms) : 0, 676904
GlobalTracer [baseline] (367.856 ms) : 0, 367856
GlobalTracer [candidate] (361.08 ms) : 0, 361080
AppSec [baseline] (33.889 ms) : 0, 33889
AppSec [candidate] (30.708 ms) : 0, 30708
Debugger [baseline] (9.829 ms) : 0, 9829
Debugger [candidate] (13.745 ms) : 0, 13745
Remote Config [baseline] (673.736 µs) : 0, 674
Remote Config [candidate] (646.933 µs) : 0, 647
Telemetry [baseline] (9.649 ms) : 0, 9649
Telemetry [candidate] (7.918 ms) : 0, 7918
ProfilingAgent [baseline] (105.891 ms) : 0, 105891
ProfilingAgent [candidate] (103.302 ms) : 0, 103302
Profiling [baseline] (105.916 ms) : 0, 105916
Profiling [candidate] (103.327 ms) : 0, 103327
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.52.0-SNAPSHOT~88a7bbba8b, baseline=1.52.0-SNAPSHOT~ad6d6b9786
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (995.236 ms) : 0, 995236
Total [baseline] (8.573 s) : 0, 8573084
Agent [candidate] (997.527 ms) : 0, 997527
Total [candidate] (8.577 s) : 0, 8577228
section iast
Agent [baseline] (1.139 s) : 0, 1139163
Total [baseline] (9.335 s) : 0, 9335387
Agent [candidate] (1.145 s) : 0, 1145302
Total [candidate] (9.341 s) : 0, 9341139
gantt
title insecure-bank - break down per module: candidate=1.52.0-SNAPSHOT~88a7bbba8b, baseline=1.52.0-SNAPSHOT~ad6d6b9786
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.532 ms) : 0, 687532
BytebuddyAgent [candidate] (688.352 ms) : 0, 688352
GlobalTracer [baseline] (241.937 ms) : 0, 241937
GlobalTracer [candidate] (243.049 ms) : 0, 243049
AppSec [baseline] (30.12 ms) : 0, 30120
AppSec [candidate] (30.366 ms) : 0, 30366
Debugger [baseline] (5.997 ms) : 0, 5997
Debugger [candidate] (5.985 ms) : 0, 5985
Remote Config [baseline] (676.781 µs) : 0, 677
Remote Config [candidate] (691.028 µs) : 0, 691
Telemetry [baseline] (8.184 ms) : 0, 8184
Telemetry [candidate] (8.231 ms) : 0, 8231
section iast
BytebuddyAgent [baseline] (812.755 ms) : 0, 812755
BytebuddyAgent [candidate] (815.304 ms) : 0, 815304
GlobalTracer [baseline] (233.443 ms) : 0, 233443
GlobalTracer [candidate] (235.388 ms) : 0, 235388
IAST [baseline] (27.476 ms) : 0, 27476
IAST [candidate] (27.933 ms) : 0, 27933
AppSec [baseline] (29.537 ms) : 0, 29537
AppSec [candidate] (31.201 ms) : 0, 31201
Debugger [baseline] (5.755 ms) : 0, 5755
Debugger [candidate] (5.874 ms) : 0, 5874
Remote Config [baseline] (582.45 µs) : 0, 582
Remote Config [candidate] (602.953 µs) : 0, 603
Telemetry [baseline] (7.948 ms) : 0, 7948
Telemetry [candidate] (8.091 ms) : 0, 8091
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~88a7bbba8b, baseline=1.52.0-SNAPSHOT~ad6d6b9786
dateFormat X
axisFormat %s
section baseline
no_agent (4.345 ms) : 4297, 4394
. : milestone, 4345,
iast (9.361 ms) : 9200, 9523
. : milestone, 9361,
iast_FULL (14.0 ms) : 13724, 14276
. : milestone, 14000,
iast_GLOBAL (10.05 ms) : 9875, 10226
. : milestone, 10050,
profiling (8.489 ms) : 8355, 8623
. : milestone, 8489,
tracing (7.717 ms) : 7597, 7837
. : milestone, 7717,
section candidate
no_agent (4.456 ms) : 4406, 4505
. : milestone, 4456,
iast (9.715 ms) : 9551, 9878
. : milestone, 9715,
iast_FULL (14.091 ms) : 13813, 14369
. : milestone, 14091,
iast_GLOBAL (10.367 ms) : 10187, 10547
. : milestone, 10367,
profiling (8.432 ms) : 8301, 8564
. : milestone, 8432,
tracing (7.554 ms) : 7438, 7669
. : milestone, 7554,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~88a7bbba8b, baseline=1.52.0-SNAPSHOT~ad6d6b9786
dateFormat X
axisFormat %s
section baseline
no_agent (36.402 ms) : 36103, 36701
. : milestone, 36402,
appsec (46.924 ms) : 46492, 47355
. : milestone, 46924,
code_origins (45.532 ms) : 45142, 45922
. : milestone, 45532,
iast (44.806 ms) : 44424, 45189
. : milestone, 44806,
profiling (47.605 ms) : 47164, 48046
. : milestone, 47605,
tracing (44.651 ms) : 44275, 45028
. : milestone, 44651,
section candidate
no_agent (36.871 ms) : 36585, 37158
. : milestone, 36871,
appsec (45.243 ms) : 44841, 45646
. : milestone, 45243,
code_origins (44.219 ms) : 43845, 44592
. : milestone, 44219,
iast (43.515 ms) : 43142, 43888
. : milestone, 43515,
profiling (47.226 ms) : 46803, 47650
. : milestone, 47226,
tracing (43.621 ms) : 43274, 43969
. : milestone, 43621,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~88a7bbba8b, baseline=1.52.0-SNAPSHOT~ad6d6b9786
dateFormat X
axisFormat %s
section baseline
no_agent (14.955 s) : 14955000, 14955000
. : milestone, 14955000,
appsec (14.756 s) : 14756000, 14756000
. : milestone, 14756000,
iast (18.151 s) : 18151000, 18151000
. : milestone, 18151000,
iast_GLOBAL (18.181 s) : 18181000, 18181000
. : milestone, 18181000,
profiling (15.112 s) : 15112000, 15112000
. : milestone, 15112000,
tracing (14.803 s) : 14803000, 14803000
. : milestone, 14803000,
section candidate
no_agent (15.252 s) : 15252000, 15252000
. : milestone, 15252000,
appsec (14.736 s) : 14736000, 14736000
. : milestone, 14736000,
iast (18.439 s) : 18439000, 18439000
. : milestone, 18439000,
iast_GLOBAL (17.77 s) : 17770000, 17770000
. : milestone, 17770000,
profiling (15.089 s) : 15089000, 15089000
. : milestone, 15089000,
tracing (14.747 s) : 14747000, 14747000
. : milestone, 14747000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~88a7bbba8b, baseline=1.52.0-SNAPSHOT~ad6d6b9786
dateFormat X
axisFormat %s
section baseline
no_agent (1.479 ms) : 1467, 1491
. : milestone, 1479,
appsec (2.424 ms) : 2373, 2474
. : milestone, 2424,
iast (2.207 ms) : 2143, 2270
. : milestone, 2207,
iast_GLOBAL (2.247 ms) : 2184, 2311
. : milestone, 2247,
profiling (2.039 ms) : 1989, 2089
. : milestone, 2039,
tracing (2.032 ms) : 1982, 2081
. : milestone, 2032,
section candidate
no_agent (1.478 ms) : 1467, 1490
. : milestone, 1478,
appsec (2.415 ms) : 2365, 2465
. : milestone, 2415,
iast (2.213 ms) : 2149, 2276
. : milestone, 2213,
iast_GLOBAL (2.255 ms) : 2192, 2319
. : milestone, 2255,
profiling (2.066 ms) : 2014, 2118
. : milestone, 2066,
tracing (2.017 ms) : 1968, 2066
. : milestone, 2017,
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix
What Does This Do
Apache http client tests are quite flaky. Especially those couple with parent and errors.
This PR just try to remove the flakiness by sorting the spans by start in order to make the assertions reliable
Motivation
Additional Notes
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]