Skip to content

Mark tests as flaky #8583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 19, 2025
Merged

Mark tests as flaky #8583

merged 1 commit into from
Mar 19, 2025

Conversation

PerfectSlayer
Copy link
Contributor

@PerfectSlayer PerfectSlayer commented Mar 18, 2025 •
edited
Loading

What Does This Do

This PR is a follow up of #8568 as it started to flake on Semeru 11 and 17 too.

Motivation

3 master builds failed Today due to this tests (2 on Semeru 11 and 1 on Semeru 17)

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@PerfectSlayer PerfectSlayer added tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests tag: flaky test/disabled Disabled flaky tests labels Mar 18, 2025
@PerfectSlayer PerfectSlayer requested a review from a team as a code owner March 18, 2025 14:51
@PerfectSlayer PerfectSlayer enabled auto-merge (squash) March 18, 2025 14:51
@pr-commenter
Copy link

pr-commenter bot commented Mar 18, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/ci
git_commit_date 1742305462 1742309437
git_commit_sha ffa2073 4b79f6d
release_version 1.48.0-SNAPSHOT~ffa2073a5c 1.48.0-SNAPSHOT~4b79f6d301
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1742311958 1742311958
ci_job_id 852995676 852995676
ci_pipeline_id 59190736 59190736
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-sugdksyp-project-304-concurrent-2-n3v818gt 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-sugdksyp-project-304-concurrent-2-n3v818gt 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~4b79f6d301, baseline=1.48.0-SNAPSHOT~ffa2073a5c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039146
Total [baseline] (8.662 s) : 0, 8661573
Agent [candidate] (1.04 s) : 0, 1040411
Total [candidate] (8.654 s) : 0, 8654013
section iast
Agent [baseline] (1.17 s) : 0, 1169633
Total [baseline] (9.21 s) : 0, 9209754
Agent [candidate] (1.181 s) : 0, 1180678
Total [candidate] (9.236 s) : 0, 9236226
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.169 s) : 0, 1169442
Total [baseline] (9.187 s) : 0, 9187403
Agent [candidate] (1.169 s) : 0, 1169449
Total [candidate] (9.18 s) : 0, 9180212
section iast_TELEMETRY_OFF
Agent [baseline] (1.177 s) : 0, 1176522
Total [baseline] (9.208 s) : 0, 9208292
Agent [candidate] (1.172 s) : 0, 1171857
Total [candidate] (9.261 s) : 0, 9260533
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.039 s -
Agent iast 1.17 s 130.486 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.169 s 130.296 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.177 s 137.376 ms (13.2%)
Total tracing 8.662 s -
Total iast 9.21 s 548.181 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.187 s 525.831 ms (6.1%)
Total iast_TELEMETRY_OFF 9.208 s 546.719 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.04 s -
Agent iast 1.181 s 140.267 ms (13.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.169 s 129.038 ms (12.4%)
Agent iast_TELEMETRY_OFF 1.172 s 131.446 ms (12.6%)
Total tracing 8.654 s -
Total iast 9.236 s 582.213 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.18 s 526.199 ms (6.1%)
Total iast_TELEMETRY_OFF 9.261 s 606.521 ms (7.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~4b79f6d301, baseline=1.48.0-SNAPSHOT~ffa2073a5c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.837 ms) : 0, 718837
BytebuddyAgent [candidate] (716.879 ms) : 0, 716879
GlobalTracer [baseline] (239.538 ms) : 0, 239538
GlobalTracer [candidate] (238.702 ms) : 0, 238702
AppSec [baseline] (54.339 ms) : 0, 54339
AppSec [candidate] (54.598 ms) : 0, 54598
Remote Config [baseline] (692.993 µs) : 0, 693
Remote Config [candidate] (689.227 µs) : 0, 689
Telemetry [baseline] (10.582 ms) : 0, 10582
Telemetry [candidate] (14.416 ms) : 0, 14416
section iast
BytebuddyAgent [baseline] (836.624 ms) : 0, 836624
BytebuddyAgent [candidate] (845.713 ms) : 0, 845713
GlobalTracer [baseline] (229.098 ms) : 0, 229098
GlobalTracer [candidate] (231.159 ms) : 0, 231159
IAST [baseline] (22.771 ms) : 0, 22771
IAST [candidate] (23.508 ms) : 0, 23508
AppSec [baseline] (56.513 ms) : 0, 56513
AppSec [candidate] (55.635 ms) : 0, 55635
Remote Config [baseline] (624.086 µs) : 0, 624
Remote Config [candidate] (619.359 µs) : 0, 619
Telemetry [baseline] (8.791 ms) : 0, 8791
Telemetry [candidate] (8.738 ms) : 0, 8738
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (836.435 ms) : 0, 836435
BytebuddyAgent [candidate] (836.499 ms) : 0, 836499
GlobalTracer [baseline] (229.625 ms) : 0, 229625
GlobalTracer [candidate] (229.358 ms) : 0, 229358
IAST [baseline] (22.502 ms) : 0, 22502
IAST [candidate] (22.889 ms) : 0, 22889
AppSec [baseline] (56.302 ms) : 0, 56302
AppSec [candidate] (56.217 ms) : 0, 56217
Remote Config [baseline] (600.084 µs) : 0, 600
Remote Config [candidate] (615.354 µs) : 0, 615
Telemetry [baseline] (8.78 ms) : 0, 8780
Telemetry [candidate] (8.636 ms) : 0, 8636
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (842.285 ms) : 0, 842285
BytebuddyAgent [candidate] (836.818 ms) : 0, 836818
GlobalTracer [baseline] (230.558 ms) : 0, 230558
GlobalTracer [candidate] (231.501 ms) : 0, 231501
IAST [baseline] (22.609 ms) : 0, 22609
IAST [candidate] (22.759 ms) : 0, 22759
AppSec [baseline] (56.466 ms) : 0, 56466
AppSec [candidate] (56.233 ms) : 0, 56233
Remote Config [baseline] (611.649 µs) : 0, 612
Remote Config [candidate] (613.187 µs) : 0, 613
Telemetry [baseline] (8.645 ms) : 0, 8645
Telemetry [candidate] (8.675 ms) : 0, 8675
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~4b79f6d301, baseline=1.48.0-SNAPSHOT~ffa2073a5c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1044866
Total [baseline] (10.483 s) : 0, 10482659
Agent [candidate] (1.055 s) : 0, 1054544
Total [candidate] (10.472 s) : 0, 10471759
section appsec
Agent [baseline] (1.182 s) : 0, 1181864
Total [baseline] (10.715 s) : 0, 10714966
Agent [candidate] (1.182 s) : 0, 1182163
Total [candidate] (10.776 s) : 0, 10776379
section iast
Agent [baseline] (1.169 s) : 0, 1168973
Total [baseline] (10.981 s) : 0, 10981397
Agent [candidate] (1.171 s) : 0, 1170971
Total [candidate] (10.951 s) : 0, 10951102
section profiling
Agent [baseline] (1.26 s) : 0, 1259716
Total [baseline] (10.846 s) : 0, 10846409
Agent [candidate] (1.257 s) : 0, 1256887
Total [candidate] (10.833 s) : 0, 10832717
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.045 s -
Agent appsec 1.182 s 136.999 ms (13.1%)
Agent iast 1.169 s 124.107 ms (11.9%)
Agent profiling 1.26 s 214.851 ms (20.6%)
Total tracing 10.483 s -
Total appsec 10.715 s 232.307 ms (2.2%)
Total iast 10.981 s 498.738 ms (4.8%)
Total profiling 10.846 s 363.75 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent appsec 1.182 s 127.62 ms (12.1%)
Agent iast 1.171 s 116.428 ms (11.0%)
Agent profiling 1.257 s 202.344 ms (19.2%)
Total tracing 10.472 s -
Total appsec 10.776 s 304.619 ms (2.9%)
Total iast 10.951 s 479.343 ms (4.6%)
Total profiling 10.833 s 360.958 ms (3.4%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~4b79f6d301, baseline=1.48.0-SNAPSHOT~ffa2073a5c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (722.32 ms) : 0, 722320
BytebuddyAgent [candidate] (727.085 ms) : 0, 727085
GlobalTracer [baseline] (240.632 ms) : 0, 240632
GlobalTracer [candidate] (243.182 ms) : 0, 243182
AppSec [baseline] (54.952 ms) : 0, 54952
AppSec [candidate] (55.868 ms) : 0, 55868
Remote Config [baseline] (694.54 µs) : 0, 695
Remote Config [candidate] (701.886 µs) : 0, 702
Telemetry [baseline] (11.001 ms) : 0, 11001
Telemetry [candidate] (12.306 ms) : 0, 12306
section appsec
BytebuddyAgent [baseline] (734.846 ms) : 0, 734846
BytebuddyAgent [candidate] (735.469 ms) : 0, 735469
GlobalTracer [baseline] (235.873 ms) : 0, 235873
GlobalTracer [candidate] (236.106 ms) : 0, 236106
IAST [baseline] (21.552 ms) : 0, 21552
IAST [candidate] (21.367 ms) : 0, 21367
AppSec [baseline] (176.301 ms) : 0, 176301
AppSec [candidate] (175.774 ms) : 0, 175774
Remote Config [baseline] (655.94 µs) : 0, 656
Remote Config [candidate] (660.654 µs) : 0, 661
Telemetry [baseline] (8.237 ms) : 0, 8237
Telemetry [candidate] (8.302 ms) : 0, 8302
section iast
BytebuddyAgent [baseline] (836.435 ms) : 0, 836435
BytebuddyAgent [candidate] (838.04 ms) : 0, 838040
GlobalTracer [baseline] (229.683 ms) : 0, 229683
GlobalTracer [candidate] (229.893 ms) : 0, 229893
IAST [baseline] (22.604 ms) : 0, 22604
IAST [candidate] (22.751 ms) : 0, 22751
AppSec [baseline] (55.8 ms) : 0, 55800
AppSec [candidate] (55.861 ms) : 0, 55861
Remote Config [baseline] (610.511 µs) : 0, 611
Remote Config [candidate] (610.491 µs) : 0, 610
Telemetry [baseline] (8.645 ms) : 0, 8645
Telemetry [candidate] (8.601 ms) : 0, 8601
section profiling
BytebuddyAgent [baseline] (709.495 ms) : 0, 709495
BytebuddyAgent [candidate] (707.94 ms) : 0, 707940
GlobalTracer [baseline] (349.382 ms) : 0, 349382
GlobalTracer [candidate] (349.076 ms) : 0, 349076
AppSec [baseline] (54.137 ms) : 0, 54137
AppSec [candidate] (53.348 ms) : 0, 53348
Remote Config [baseline] (668.403 µs) : 0, 668
Remote Config [candidate] (658.78 µs) : 0, 659
Telemetry [baseline] (8.95 ms) : 0, 8950
Telemetry [candidate] (8.867 ms) : 0, 8867
ProfilingAgent [baseline] (96.3 ms) : 0, 96300
ProfilingAgent [candidate] (96.226 ms) : 0, 96226
Profiling [baseline] (96.325 ms) : 0, 96325
Profiling [candidate] (96.25 ms) : 0, 96250
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-18T15:03:22 2025-03-18T15:11:07
git_branch master bbujon/ci
git_commit_date 1742305462 1742309437
git_commit_sha ffa2073 4b79f6d
release_version 1.48.0-SNAPSHOT~ffa2073a5c 1.48.0-SNAPSHOT~4b79f6d301
start_time 2025-03-18T15:03:08 2025-03-18T15:10:53
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1742311064 1742311064
ci_job_id 852995677 852995677
ci_pipeline_id 59190736 59190736
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-bfpa8bu2-project-304-concurrent-2-pu557l6c 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-bfpa8bu2-project-304-concurrent-2-pu557l6c 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~4b79f6d301, baseline=1.48.0-SNAPSHOT~ffa2073a5c
    dateFormat X
    axisFormat %s
section baseline
no_agent (379.518 µs) : 359, 400
.   : milestone, 380,
iast (510.684 µs) : 489, 532
.   : milestone, 511,
iast_FULL (728.09 µs) : 706, 750
.   : milestone, 728,
iast_GLOBAL (552.993 µs) : 531, 575
.   : milestone, 553,
iast_HARDCODED_SECRET_DISABLED (521.726 µs) : 499, 544
.   : milestone, 522,
iast_INACTIVE (466.759 µs) : 445, 489
.   : milestone, 467,
iast_TELEMETRY_OFF (505.179 µs) : 482, 528
.   : milestone, 505,
tracing (458.951 µs) : 437, 480
.   : milestone, 459,
section candidate
no_agent (380.221 µs) : 360, 401
.   : milestone, 380,
iast (513.469 µs) : 492, 535
.   : milestone, 513,
iast_FULL (731.935 µs) : 710, 754
.   : milestone, 732,
iast_GLOBAL (554.472 µs) : 533, 576
.   : milestone, 554,
iast_HARDCODED_SECRET_DISABLED (514.729 µs) : 492, 537
.   : milestone, 515,
iast_INACTIVE (462.072 µs) : 440, 484
.   : milestone, 462,
iast_TELEMETRY_OFF (506.239 µs) : 484, 528
.   : milestone, 506,
tracing (464.504 µs) : 443, 486
.   : milestone, 465,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.518 µs [359.356 µs, 399.68 µs] -
iast 510.684 µs [488.989 µs, 532.379 µs] 131.166 µs (34.6%)
iast_FULL 728.09 µs [705.916 µs, 750.264 µs] 348.572 µs (91.8%)
iast_GLOBAL 552.993 µs [531.279 µs, 574.706 µs] 173.475 µs (45.7%)
iast_HARDCODED_SECRET_DISABLED 521.726 µs [499.414 µs, 544.038 µs] 142.208 µs (37.5%)
iast_INACTIVE 466.759 µs [444.839 µs, 488.679 µs] 87.241 µs (23.0%)
iast_TELEMETRY_OFF 505.179 µs [482.479 µs, 527.879 µs] 125.661 µs (33.1%)
tracing 458.951 µs [437.429 µs, 480.472 µs] 79.433 µs (20.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 380.221 µs [359.52 µs, 400.922 µs] -
iast 513.469 µs [491.789 µs, 535.149 µs] 133.248 µs (35.0%)
iast_FULL 731.935 µs [709.795 µs, 754.076 µs] 351.715 µs (92.5%)
iast_GLOBAL 554.472 µs [532.836 µs, 576.107 µs] 174.251 µs (45.8%)
iast_HARDCODED_SECRET_DISABLED 514.729 µs [492.372 µs, 537.087 µs] 134.509 µs (35.4%)
iast_INACTIVE 462.072 µs [440.393 µs, 483.751 µs] 81.851 µs (21.5%)
iast_TELEMETRY_OFF 506.239 µs [484.131 µs, 528.348 µs] 126.019 µs (33.1%)
tracing 464.504 µs [443.219 µs, 485.789 µs] 84.283 µs (22.2%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~4b79f6d301, baseline=1.48.0-SNAPSHOT~ffa2073a5c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.361 ms) : 1341, 1380
.   : milestone, 1361,
appsec (1.73 ms) : 1707, 1754
.   : milestone, 1730,
appsec_no_iast (1.742 ms) : 1718, 1766
.   : milestone, 1742,
code_origins (1.684 ms) : 1657, 1711
.   : milestone, 1684,
iast (1.52 ms) : 1495, 1545
.   : milestone, 1520,
profiling (1.557 ms) : 1533, 1580
.   : milestone, 1557,
tracing (1.488 ms) : 1463, 1513
.   : milestone, 1488,
section candidate
no_agent (1.358 ms) : 1338, 1377
.   : milestone, 1358,
appsec (1.724 ms) : 1701, 1748
.   : milestone, 1724,
appsec_no_iast (1.725 ms) : 1700, 1750
.   : milestone, 1725,
code_origins (1.658 ms) : 1631, 1686
.   : milestone, 1658,
iast (1.522 ms) : 1497, 1546
.   : milestone, 1522,
profiling (1.517 ms) : 1494, 1540
.   : milestone, 1517,
tracing (1.499 ms) : 1474, 1525
.   : milestone, 1499,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.361 ms [1.341 ms, 1.38 ms] -
appsec 1.73 ms [1.707 ms, 1.754 ms] 369.757 µs (27.2%)
appsec_no_iast 1.742 ms [1.718 ms, 1.766 ms] 381.869 µs (28.1%)
code_origins 1.684 ms [1.657 ms, 1.711 ms] 323.367 µs (23.8%)
iast 1.52 ms [1.495 ms, 1.545 ms] 159.611 µs (11.7%)
profiling 1.557 ms [1.533 ms, 1.58 ms] 196.015 µs (14.4%)
tracing 1.488 ms [1.463 ms, 1.513 ms] 127.17 µs (9.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.358 ms [1.338 ms, 1.377 ms] -
appsec 1.724 ms [1.701 ms, 1.748 ms] 366.635 µs (27.0%)
appsec_no_iast 1.725 ms [1.7 ms, 1.75 ms] 367.469 µs (27.1%)
code_origins 1.658 ms [1.631 ms, 1.686 ms] 300.57 µs (22.1%)
iast 1.522 ms [1.497 ms, 1.546 ms] 163.934 µs (12.1%)
profiling 1.517 ms [1.494 ms, 1.54 ms] 159.309 µs (11.7%)
tracing 1.499 ms [1.474 ms, 1.525 ms] 141.511 µs (10.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/ci
git_commit_date 1742305462 1742309437
git_commit_sha ffa2073 4b79f6d
release_version 1.48.0-SNAPSHOT~ffa2073a5c 1.48.0-SNAPSHOT~4b79f6d301
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1742311622 1742311622
ci_job_id 852995678 852995678
ci_pipeline_id 59190736 59190736
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-oryhnspu-project-304-concurrent-2-vuif31jm 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-oryhnspu-project-304-concurrent-2-vuif31jm 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~4b79f6d301, baseline=1.48.0-SNAPSHOT~ffa2073a5c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.106 s) : 15106000, 15106000
.   : milestone, 15106000,
appsec (15.104 s) : 15104000, 15104000
.   : milestone, 15104000,
iast (18.86 s) : 18860000, 18860000
.   : milestone, 18860000,
iast_GLOBAL (17.984 s) : 17984000, 17984000
.   : milestone, 17984000,
profiling (15.271 s) : 15271000, 15271000
.   : milestone, 15271000,
tracing (14.729 s) : 14729000, 14729000
.   : milestone, 14729000,
section candidate
no_agent (15.131 s) : 15131000, 15131000
.   : milestone, 15131000,
appsec (15.132 s) : 15132000, 15132000
.   : milestone, 15132000,
iast (18.852 s) : 18852000, 18852000
.   : milestone, 18852000,
iast_GLOBAL (17.964 s) : 17964000, 17964000
.   : milestone, 17964000,
profiling (15.6 s) : 15600000, 15600000
.   : milestone, 15600000,
tracing (15.241 s) : 15241000, 15241000
.   : milestone, 15241000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.106 s [15.106 s, 15.106 s] -
appsec 15.104 s [15.104 s, 15.104 s] -2.0 ms (-0.0%)
iast 18.86 s [18.86 s, 18.86 s] 3.754 s (24.9%)
iast_GLOBAL 17.984 s [17.984 s, 17.984 s] 2.878 s (19.1%)
profiling 15.271 s [15.271 s, 15.271 s] 165.0 ms (1.1%)
tracing 14.729 s [14.729 s, 14.729 s] -377.0 ms (-2.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.131 s [15.131 s, 15.131 s] -
appsec 15.132 s [15.132 s, 15.132 s] 1.0 ms (0.0%)
iast 18.852 s [18.852 s, 18.852 s] 3.721 s (24.6%)
iast_GLOBAL 17.964 s [17.964 s, 17.964 s] 2.833 s (18.7%)
profiling 15.6 s [15.6 s, 15.6 s] 469.0 ms (3.1%)
tracing 15.241 s [15.241 s, 15.241 s] 110.0 ms (0.7%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~4b79f6d301, baseline=1.48.0-SNAPSHOT~ffa2073a5c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.347 ms) : 2303, 2391
.   : milestone, 2347,
iast (2.116 ms) : 2060, 2171
.   : milestone, 2116,
iast_GLOBAL (2.157 ms) : 2101, 2213
.   : milestone, 2157,
profiling (1.971 ms) : 1927, 2015
.   : milestone, 1971,
tracing (1.943 ms) : 1900, 1985
.   : milestone, 1943,
section candidate
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (2.354 ms) : 2310, 2399
.   : milestone, 2354,
iast (2.123 ms) : 2067, 2179
.   : milestone, 2123,
iast_GLOBAL (2.17 ms) : 2114, 2227
.   : milestone, 2170,
profiling (1.969 ms) : 1924, 2013
.   : milestone, 1969,
tracing (1.95 ms) : 1908, 1993
.   : milestone, 1950,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 2.347 ms [2.303 ms, 2.391 ms] 875.148 µs (59.5%)
iast 2.116 ms [2.06 ms, 2.171 ms] 643.897 µs (43.7%)
iast_GLOBAL 2.157 ms [2.101 ms, 2.213 ms] 685.362 µs (46.6%)
profiling 1.971 ms [1.927 ms, 2.015 ms] 499.552 µs (33.9%)
tracing 1.943 ms [1.9 ms, 1.985 ms] 471.04 µs (32.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.463 ms, 1.486 ms] -
appsec 2.354 ms [2.31 ms, 2.399 ms] 880.044 µs (59.7%)
iast 2.123 ms [2.067 ms, 2.179 ms] 649.127 µs (44.0%)
iast_GLOBAL 2.17 ms [2.114 ms, 2.227 ms] 696.18 µs (47.2%)
profiling 1.969 ms [1.924 ms, 2.013 ms] 494.662 µs (33.6%)
tracing 1.95 ms [1.908 ms, 1.993 ms] 475.958 µs (32.3%)

@PerfectSlayer PerfectSlayer merged commit b1d7def into master Mar 19, 2025
268 of 271 checks passed
@PerfectSlayer PerfectSlayer deleted the bbujon/ci branch March 19, 2025 08:44
@github-actions github-actions bot added this to the 1.48.0 milestone Mar 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@manuel-alvarez-alvarez manuel-alvarez-alvarez manuel-alvarez-alvarez approved these changes

Assignees
No one assigned
Labels
tag: flaky test/disabled Disabled flaky tests tag: flaky test Flaky tests tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.48.0
Development

Successfully merging this pull request may close these issues.
3 participants
@PerfectSlayer @manuel-alvarez-alvarez @amarziali