Skip to content

Mark tests as flaky #8568

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 17, 2025
Merged

Mark tests as flaky #8568

merged 1 commit into from
Mar 17, 2025

Conversation

PerfectSlayer
Copy link
Contributor

@PerfectSlayer PerfectSlayer commented Mar 14, 2025 •
edited by atlassian bot
Loading

What Does This Do

Mark the top flaky tests of the week as flaky.

Motivation

Keep the CI as stable as possible

Additional Notes

I can't find IBM8 JVM to setup locally so I didn't try to fix it in the first place.

Contributor Checklist

Jira ticket: LANGPLAT-323

@PerfectSlayer PerfectSlayer added tag: flaky test Flaky tests tag: flaky test/disabled Disabled flaky tests labels Mar 14, 2025
@PerfectSlayer PerfectSlayer requested a review from a team as a code owner March 14, 2025 17:40
@PerfectSlayer PerfectSlayer requested a review from mcculls March 14, 2025 17:40
@PerfectSlayer PerfectSlayer enabled auto-merge (squash) March 14, 2025 17:40
@github-actions GitHub Actions
Copy link
Contributor

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@PerfectSlayer PerfectSlayer added the tag: no release notes Changes to exclude from release notes label Mar 14, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 14, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/ci
git_commit_date 1741967085 1741973812
git_commit_sha c06a408 f7ff70a
release_version 1.48.0-SNAPSHOT~c06a40868a 1.48.0-SNAPSHOT~f7ff70a8d7
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741976438 1741976438
ci_job_id 849056471 849056471
ci_pipeline_id 58885009 58885009
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zz5zaysw-project-304-concurrent-0-jykmvova 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zz5zaysw-project-304-concurrent-0-jykmvova 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~f7ff70a8d7, baseline=1.48.0-SNAPSHOT~c06a40868a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.042 s) : 0, 1041637
Total [baseline] (8.718 s) : 0, 8717659
Agent [candidate] (1.042 s) : 0, 1041944
Total [candidate] (8.677 s) : 0, 8677072
section iast
Agent [baseline] (1.182 s) : 0, 1181722
Total [baseline] (9.303 s) : 0, 9302715
Agent [candidate] (1.18 s) : 0, 1179660
Total [candidate] (9.259 s) : 0, 9258600
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.172 s) : 0, 1172329
Total [baseline] (9.224 s) : 0, 9224121
Agent [candidate] (1.194 s) : 0, 1193860
Total [candidate] (9.27 s) : 0, 9269810
section iast_TELEMETRY_OFF
Agent [baseline] (1.168 s) : 0, 1168067
Total [baseline] (9.273 s) : 0, 9273139
Agent [candidate] (1.176 s) : 0, 1175647
Total [candidate] (9.294 s) : 0, 9294048
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.042 s -
Agent iast 1.182 s 140.085 ms (13.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.172 s 130.692 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.168 s 126.43 ms (12.1%)
Total tracing 8.718 s -
Total iast 9.303 s 585.056 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.224 s 506.462 ms (5.8%)
Total iast_TELEMETRY_OFF 9.273 s 555.48 ms (6.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.042 s -
Agent iast 1.18 s 137.715 ms (13.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.194 s 151.915 ms (14.6%)
Agent iast_TELEMETRY_OFF 1.176 s 133.703 ms (12.8%)
Total tracing 8.677 s -
Total iast 9.259 s 581.528 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.27 s 592.739 ms (6.8%)
Total iast_TELEMETRY_OFF 9.294 s 616.977 ms (7.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~f7ff70a8d7, baseline=1.48.0-SNAPSHOT~c06a40868a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (719.867 ms) : 0, 719867
BytebuddyAgent [candidate] (718.549 ms) : 0, 718549
GlobalTracer [baseline] (240.282 ms) : 0, 240282
GlobalTracer [candidate] (240.059 ms) : 0, 240059
AppSec [baseline] (54.799 ms) : 0, 54799
AppSec [candidate] (54.578 ms) : 0, 54578
Remote Config [baseline] (692.788 µs) : 0, 693
Remote Config [candidate] (697.03 µs) : 0, 697
Telemetry [baseline] (10.807 ms) : 0, 10807
Telemetry [candidate] (12.899 ms) : 0, 12899
section iast
BytebuddyAgent [baseline] (844.623 ms) : 0, 844623
BytebuddyAgent [candidate] (844.4 ms) : 0, 844400
GlobalTracer [baseline] (231.925 ms) : 0, 231925
GlobalTracer [candidate] (231.83 ms) : 0, 231830
AppSec [baseline] (56.922 ms) : 0, 56922
AppSec [candidate] (56.054 ms) : 0, 56054
Remote Config [baseline] (623.499 µs) : 0, 623
Remote Config [candidate] (599.468 µs) : 0, 599
Telemetry [baseline] (8.89 ms) : 0, 8890
Telemetry [candidate] (8.641 ms) : 0, 8641
IAST [baseline] (23.423 ms) : 0, 23423
IAST [candidate] (22.839 ms) : 0, 22839
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (837.347 ms) : 0, 837347
BytebuddyAgent [candidate] (855.098 ms) : 0, 855098
GlobalTracer [baseline] (231.238 ms) : 0, 231238
GlobalTracer [candidate] (233.384 ms) : 0, 233384
AppSec [baseline] (56.257 ms) : 0, 56257
AppSec [candidate] (56.928 ms) : 0, 56928
Remote Config [baseline] (606.907 µs) : 0, 607
Remote Config [candidate] (629.347 µs) : 0, 629
Telemetry [baseline] (8.699 ms) : 0, 8699
Telemetry [candidate] (8.929 ms) : 0, 8929
IAST [baseline] (22.964 ms) : 0, 22964
IAST [candidate] (23.401 ms) : 0, 23401
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (834.706 ms) : 0, 834706
BytebuddyAgent [candidate] (839.827 ms) : 0, 839827
GlobalTracer [baseline] (230.956 ms) : 0, 230956
GlobalTracer [candidate] (231.912 ms) : 0, 231912
AppSec [baseline] (55.594 ms) : 0, 55594
AppSec [candidate] (56.639 ms) : 0, 56639
Remote Config [baseline] (595.013 µs) : 0, 595
Remote Config [candidate] (626.949 µs) : 0, 627
Telemetry [baseline] (8.577 ms) : 0, 8577
Telemetry [candidate] (8.659 ms) : 0, 8659
IAST [baseline] (22.465 ms) : 0, 22465
IAST [candidate] (22.703 ms) : 0, 22703
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~f7ff70a8d7, baseline=1.48.0-SNAPSHOT~c06a40868a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039317
Total [baseline] (10.473 s) : 0, 10472855
Agent [candidate] (1.04 s) : 0, 1039772
Total [candidate] (10.48 s) : 0, 10480458
section appsec
Agent [baseline] (1.183 s) : 0, 1183056
Total [baseline] (10.777 s) : 0, 10776667
Agent [candidate] (1.187 s) : 0, 1187339
Total [candidate] (10.75 s) : 0, 10750284
section iast
Agent [baseline] (1.173 s) : 0, 1173433
Total [baseline] (11.005 s) : 0, 11004968
Agent [candidate] (1.171 s) : 0, 1170963
Total [candidate] (10.964 s) : 0, 10964342
section profiling
Agent [baseline] (1.26 s) : 0, 1260000
Total [baseline] (10.807 s) : 0, 10806718
Agent [candidate] (1.26 s) : 0, 1259525
Total [candidate] (10.872 s) : 0, 10872050
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.039 s -
Agent appsec 1.183 s 143.738 ms (13.8%)
Agent iast 1.173 s 134.116 ms (12.9%)
Agent profiling 1.26 s 220.683 ms (21.2%)
Total tracing 10.473 s -
Total appsec 10.777 s 303.812 ms (2.9%)
Total iast 11.005 s 532.114 ms (5.1%)
Total profiling 10.807 s 333.864 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.04 s -
Agent appsec 1.187 s 147.567 ms (14.2%)
Agent iast 1.171 s 131.191 ms (12.6%)
Agent profiling 1.26 s 219.753 ms (21.1%)
Total tracing 10.48 s -
Total appsec 10.75 s 269.826 ms (2.6%)
Total iast 10.964 s 483.884 ms (4.6%)
Total profiling 10.872 s 391.592 ms (3.7%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~f7ff70a8d7, baseline=1.48.0-SNAPSHOT~c06a40868a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.158 ms) : 0, 716158
BytebuddyAgent [candidate] (718.205 ms) : 0, 718205
GlobalTracer [baseline] (239.506 ms) : 0, 239506
GlobalTracer [candidate] (239.579 ms) : 0, 239579
AppSec [baseline] (54.795 ms) : 0, 54795
AppSec [candidate] (54.677 ms) : 0, 54677
Remote Config [baseline] (693.26 µs) : 0, 693
Remote Config [candidate] (684.229 µs) : 0, 684
Telemetry [baseline] (12.989 ms) : 0, 12989
Telemetry [candidate] (11.479 ms) : 0, 11479
section appsec
BytebuddyAgent [baseline] (735.745 ms) : 0, 735745
BytebuddyAgent [candidate] (740.313 ms) : 0, 740313
GlobalTracer [baseline] (236.649 ms) : 0, 236649
GlobalTracer [candidate] (236.501 ms) : 0, 236501
AppSec [baseline] (175.772 ms) : 0, 175772
AppSec [candidate] (175.236 ms) : 0, 175236
Remote Config [baseline] (657.122 µs) : 0, 657
Remote Config [candidate] (660.506 µs) : 0, 661
Telemetry [baseline] (8.276 ms) : 0, 8276
Telemetry [candidate] (8.641 ms) : 0, 8641
IAST [baseline] (21.565 ms) : 0, 21565
IAST [candidate] (21.425 ms) : 0, 21425
section iast
BytebuddyAgent [baseline] (837.856 ms) : 0, 837856
BytebuddyAgent [candidate] (837.314 ms) : 0, 837314
GlobalTracer [baseline] (231.27 ms) : 0, 231270
GlobalTracer [candidate] (230.614 ms) : 0, 230614
AppSec [baseline] (56.63 ms) : 0, 56630
AppSec [candidate] (55.837 ms) : 0, 55837
Remote Config [baseline] (619.762 µs) : 0, 620
Remote Config [candidate] (593.744 µs) : 0, 594
Telemetry [baseline] (8.87 ms) : 0, 8870
Telemetry [candidate] (8.582 ms) : 0, 8582
IAST [baseline] (22.971 ms) : 0, 22971
IAST [candidate] (22.803 ms) : 0, 22803
section profiling
ProfilingAgent [baseline] (95.766 ms) : 0, 95766
ProfilingAgent [candidate] (96.172 ms) : 0, 96172
BytebuddyAgent [baseline] (708.748 ms) : 0, 708748
BytebuddyAgent [candidate] (709.618 ms) : 0, 709618
GlobalTracer [baseline] (350.736 ms) : 0, 350736
GlobalTracer [candidate] (349.968 ms) : 0, 349968
AppSec [baseline] (54.409 ms) : 0, 54409
AppSec [candidate] (53.429 ms) : 0, 53429
Remote Config [baseline] (669.801 µs) : 0, 670
Remote Config [candidate] (653.92 µs) : 0, 654
Telemetry [baseline] (8.954 ms) : 0, 8954
Telemetry [candidate] (9.006 ms) : 0, 9006
Profiling [baseline] (95.791 ms) : 0, 95791
Profiling [candidate] (96.196 ms) : 0, 96196
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-14T17:50:48 2025-03-14T17:58:31
git_branch master bbujon/ci
git_commit_date 1741967085 1741973812
git_commit_sha c06a408 f7ff70a
release_version 1.48.0-SNAPSHOT~c06a40868a 1.48.0-SNAPSHOT~f7ff70a8d7
start_time 2025-03-14T17:50:34 2025-03-14T17:58:17
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741975508 1741975508
ci_job_id 849056473 849056473
ci_pipeline_id 58885009 58885009
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zz5zaysw-project-304-concurrent-1-09qcejxm 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zz5zaysw-project-304-concurrent-1-09qcejxm 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~f7ff70a8d7, baseline=1.48.0-SNAPSHOT~c06a40868a
    dateFormat X
    axisFormat %s
section baseline
no_agent (384.637 µs) : 365, 404
.   : milestone, 385,
iast (516.786 µs) : 495, 539
.   : milestone, 517,
iast_FULL (728.335 µs) : 706, 750
.   : milestone, 728,
iast_GLOBAL (553.66 µs) : 532, 575
.   : milestone, 554,
iast_HARDCODED_SECRET_DISABLED (513.467 µs) : 492, 535
.   : milestone, 513,
iast_INACTIVE (466.912 µs) : 445, 489
.   : milestone, 467,
iast_TELEMETRY_OFF (502.97 µs) : 481, 525
.   : milestone, 503,
tracing (456.962 µs) : 436, 478
.   : milestone, 457,
section candidate
no_agent (386.16 µs) : 365, 407
.   : milestone, 386,
iast (509.374 µs) : 488, 531
.   : milestone, 509,
iast_FULL (729.284 µs) : 707, 751
.   : milestone, 729,
iast_GLOBAL (556.383 µs) : 535, 578
.   : milestone, 556,
iast_HARDCODED_SECRET_DISABLED (519.215 µs) : 497, 541
.   : milestone, 519,
iast_INACTIVE (459.9 µs) : 439, 481
.   : milestone, 460,
iast_TELEMETRY_OFF (500.661 µs) : 478, 523
.   : milestone, 501,
tracing (453.699 µs) : 433, 475
.   : milestone, 454,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 384.637 µs [364.96 µs, 404.314 µs] -
iast 516.786 µs [494.757 µs, 538.815 µs] 132.149 µs (34.4%)
iast_FULL 728.335 µs [706.422 µs, 750.249 µs] 343.698 µs (89.4%)
iast_GLOBAL 553.66 µs [532.03 µs, 575.29 µs] 169.022 µs (43.9%)
iast_HARDCODED_SECRET_DISABLED 513.467 µs [491.802 µs, 535.131 µs] 128.829 µs (33.5%)
iast_INACTIVE 466.912 µs [445.246 µs, 488.578 µs] 82.274 µs (21.4%)
iast_TELEMETRY_OFF 502.97 µs [480.727 µs, 525.214 µs] 118.333 µs (30.8%)
tracing 456.962 µs [435.758 µs, 478.166 µs] 72.324 µs (18.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 386.16 µs [365.42 µs, 406.899 µs] -
iast 509.374 µs [487.554 µs, 531.194 µs] 123.214 µs (31.9%)
iast_FULL 729.284 µs [707.291 µs, 751.278 µs] 343.124 µs (88.9%)
iast_GLOBAL 556.383 µs [534.722 µs, 578.045 µs] 170.224 µs (44.1%)
iast_HARDCODED_SECRET_DISABLED 519.215 µs [497.364 µs, 541.066 µs] 133.055 µs (34.5%)
iast_INACTIVE 459.9 µs [438.874 µs, 480.926 µs] 73.74 µs (19.1%)
iast_TELEMETRY_OFF 500.661 µs [477.949 µs, 523.374 µs] 114.502 µs (29.7%)
tracing 453.699 µs [432.886 µs, 474.512 µs] 67.539 µs (17.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~f7ff70a8d7, baseline=1.48.0-SNAPSHOT~c06a40868a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.376 ms) : 1357, 1396
.   : milestone, 1376,
appsec (1.721 ms) : 1697, 1744
.   : milestone, 1721,
appsec_no_iast (1.737 ms) : 1712, 1761
.   : milestone, 1737,
code_origins (1.665 ms) : 1637, 1693
.   : milestone, 1665,
iast (1.508 ms) : 1484, 1532
.   : milestone, 1508,
profiling (1.51 ms) : 1486, 1534
.   : milestone, 1510,
tracing (1.512 ms) : 1487, 1537
.   : milestone, 1512,
section candidate
no_agent (1.355 ms) : 1335, 1376
.   : milestone, 1355,
appsec (1.712 ms) : 1689, 1735
.   : milestone, 1712,
appsec_no_iast (1.757 ms) : 1733, 1782
.   : milestone, 1757,
code_origins (1.683 ms) : 1656, 1711
.   : milestone, 1683,
iast (1.5 ms) : 1475, 1525
.   : milestone, 1500,
profiling (1.517 ms) : 1494, 1541
.   : milestone, 1517,
tracing (1.5 ms) : 1477, 1524
.   : milestone, 1500,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.376 ms [1.357 ms, 1.396 ms] -
appsec 1.721 ms [1.697 ms, 1.744 ms] 344.324 µs (25.0%)
appsec_no_iast 1.737 ms [1.712 ms, 1.761 ms] 360.439 µs (26.2%)
code_origins 1.665 ms [1.637 ms, 1.693 ms] 288.767 µs (21.0%)
iast 1.508 ms [1.484 ms, 1.532 ms] 131.459 µs (9.6%)
profiling 1.51 ms [1.486 ms, 1.534 ms] 133.447 µs (9.7%)
tracing 1.512 ms [1.487 ms, 1.537 ms] 135.329 µs (9.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.335 ms, 1.376 ms] -
appsec 1.712 ms [1.689 ms, 1.735 ms] 356.534 µs (26.3%)
appsec_no_iast 1.757 ms [1.733 ms, 1.782 ms] 401.904 µs (29.7%)
code_origins 1.683 ms [1.656 ms, 1.711 ms] 328.098 µs (24.2%)
iast 1.5 ms [1.475 ms, 1.525 ms] 144.401 µs (10.7%)
profiling 1.517 ms [1.494 ms, 1.541 ms] 162.122 µs (12.0%)
tracing 1.5 ms [1.477 ms, 1.524 ms] 145.068 µs (10.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/ci
git_commit_date 1741967085 1741973812
git_commit_sha c06a408 f7ff70a
release_version 1.48.0-SNAPSHOT~c06a40868a 1.48.0-SNAPSHOT~f7ff70a8d7
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1741975999 1741975999
ci_job_id 849056475 849056475
ci_pipeline_id 58885009 58885009
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-nmmaosst-project-304-concurrent-0-crwfba6a 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-nmmaosst-project-304-concurrent-0-crwfba6a 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~f7ff70a8d7, baseline=1.48.0-SNAPSHOT~c06a40868a
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.543 s) : 15543000, 15543000
.   : milestone, 15543000,
appsec (14.797 s) : 14797000, 14797000
.   : milestone, 14797000,
iast (18.789 s) : 18789000, 18789000
.   : milestone, 18789000,
iast_GLOBAL (17.461 s) : 17461000, 17461000
.   : milestone, 17461000,
profiling (15.298 s) : 15298000, 15298000
.   : milestone, 15298000,
tracing (14.947 s) : 14947000, 14947000
.   : milestone, 14947000,
section candidate
no_agent (15.229 s) : 15229000, 15229000
.   : milestone, 15229000,
appsec (14.844 s) : 14844000, 14844000
.   : milestone, 14844000,
iast (19.079 s) : 19079000, 19079000
.   : milestone, 19079000,
iast_GLOBAL (18.173 s) : 18173000, 18173000
.   : milestone, 18173000,
profiling (15.783 s) : 15783000, 15783000
.   : milestone, 15783000,
tracing (15.025 s) : 15025000, 15025000
.   : milestone, 15025000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.543 s [15.543 s, 15.543 s] -
appsec 14.797 s [14.797 s, 14.797 s] -746.0 ms (-4.8%)
iast 18.789 s [18.789 s, 18.789 s] 3.246 s (20.9%)
iast_GLOBAL 17.461 s [17.461 s, 17.461 s] 1.918 s (12.3%)
profiling 15.298 s [15.298 s, 15.298 s] -245.0 ms (-1.6%)
tracing 14.947 s [14.947 s, 14.947 s] -596.0 ms (-3.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.229 s [15.229 s, 15.229 s] -
appsec 14.844 s [14.844 s, 14.844 s] -385.0 ms (-2.5%)
iast 19.079 s [19.079 s, 19.079 s] 3.85 s (25.3%)
iast_GLOBAL 18.173 s [18.173 s, 18.173 s] 2.944 s (19.3%)
profiling 15.783 s [15.783 s, 15.783 s] 554.0 ms (3.6%)
tracing 15.025 s [15.025 s, 15.025 s] -204.0 ms (-1.3%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~f7ff70a8d7, baseline=1.48.0-SNAPSHOT~c06a40868a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.342 ms) : 2298, 2386
.   : milestone, 2342,
iast (2.109 ms) : 2053, 2164
.   : milestone, 2109,
iast_GLOBAL (2.154 ms) : 2097, 2210
.   : milestone, 2154,
profiling (1.97 ms) : 1926, 2014
.   : milestone, 1970,
tracing (1.938 ms) : 1896, 1981
.   : milestone, 1938,
section candidate
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.338 ms) : 2294, 2382
.   : milestone, 2338,
iast (2.114 ms) : 2058, 2170
.   : milestone, 2114,
iast_GLOBAL (2.158 ms) : 2102, 2214
.   : milestone, 2158,
profiling (1.968 ms) : 1924, 2012
.   : milestone, 1968,
tracing (1.946 ms) : 1903, 1988
.   : milestone, 1946,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.342 ms [2.298 ms, 2.386 ms] 872.759 µs (59.4%)
iast 2.109 ms [2.053 ms, 2.164 ms] 639.453 µs (43.5%)
iast_GLOBAL 2.154 ms [2.097 ms, 2.21 ms] 684.614 µs (46.6%)
profiling 1.97 ms [1.926 ms, 2.014 ms] 500.854 µs (34.1%)
tracing 1.938 ms [1.896 ms, 1.981 ms] 469.212 µs (31.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 2.338 ms [2.294 ms, 2.382 ms] 868.208 µs (59.1%)
iast 2.114 ms [2.058 ms, 2.17 ms] 643.742 µs (43.8%)
iast_GLOBAL 2.158 ms [2.102 ms, 2.214 ms] 687.931 µs (46.8%)
profiling 1.968 ms [1.924 ms, 2.012 ms] 497.638 µs (33.8%)
tracing 1.946 ms [1.903 ms, 1.988 ms] 475.272 µs (32.3%)

@PerfectSlayer PerfectSlayer merged commit 0bc5b2f into master Mar 17, 2025
273 of 275 checks passed
@PerfectSlayer PerfectSlayer deleted the bbujon/ci branch March 17, 2025 10:12
@github-actions github-actions bot added this to the 1.48.0 milestone Mar 17, 2025
@PerfectSlayer PerfectSlayer mentioned this pull request Mar 18, 2025
Merged
mtoffl01 pushed a commit that referenced this pull request Mar 24, 2025
@PerfectSlayer @mtoffl01
fix(ci): Mark tests as flaky (#8568)
e8eb022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
tag: flaky test/disabled Disabled flaky tests tag: flaky test Flaky tests tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.48.0
Development

Successfully merging this pull request may close these issues.
2 participants
@PerfectSlayer @smola