Skip to content

Add Java 24 tests to CI #8739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

Add Java 24 tests to CI #8739

wants to merge 2 commits into from

Conversation

sarahchen6
Copy link
Contributor

What Does This Do

Add minimum Java 24 tests to CI

Motivation

Test Java 24

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@sarahchen6 sarahchen6 force-pushed the sarahchen6/add-jdk-24-test branch from a107c8b to 9d76aeb Compare April 25, 2025 21:07
@sarahchen6 sarahchen6 added type: enhancement Enhancements and improvements comp: testing Testing labels Apr 25, 2025
@pr-commenter
Copy link

pr-commenter bot commented Apr 25, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/add-jdk-24-test
git_commit_date 1745613844 1745615243
git_commit_sha 4d01d58 9d76aeb
release_version 1.49.0-SNAPSHOT~4d01d583bb 1.49.0-SNAPSHOT~9d76aeb34c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745618288 1745618288
ci_job_id 913756515 913756515
ci_pipeline_id 63401101 63401101
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-px9uuxed-project-304-concurrent-0-vy940zlk 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-px9uuxed-project-304-concurrent-0-vy940zlk 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 66 metrics, 5 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.007 s) : 0, 1007063
Total [baseline] (8.661 s) : 0, 8660791
Agent [candidate] (1.011 s) : 0, 1010595
Total [candidate] (8.647 s) : 0, 8646807
section iast
Agent [baseline] (1.135 s) : 0, 1134509
Total [baseline] (9.205 s) : 0, 9204626
Agent [candidate] (1.142 s) : 0, 1141888
Total [candidate] (9.274 s) : 0, 9274097
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.137 s) : 0, 1137326
Total [baseline] (9.179 s) : 0, 9179493
Agent [candidate] (1.142 s) : 0, 1142484
Total [candidate] (9.216 s) : 0, 9215848
section iast_TELEMETRY_OFF
Agent [baseline] (1.134 s) : 0, 1133965
Total [baseline] (9.18 s) : 0, 9179725
Agent [candidate] (1.135 s) : 0, 1135310
Total [candidate] (9.19 s) : 0, 9189537
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.007 s -
Agent iast 1.135 s 127.446 ms (12.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.137 s 130.263 ms (12.9%)
Agent iast_TELEMETRY_OFF 1.134 s 126.902 ms (12.6%)
Total tracing 8.661 s -
Total iast 9.205 s 543.835 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.179 s 518.702 ms (6.0%)
Total iast_TELEMETRY_OFF 9.18 s 518.934 ms (6.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.011 s -
Agent iast 1.142 s 131.293 ms (13.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.142 s 131.889 ms (13.1%)
Agent iast_TELEMETRY_OFF 1.135 s 124.715 ms (12.3%)
Total tracing 8.647 s -
Total iast 9.274 s 627.29 ms (7.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.216 s 569.041 ms (6.6%)
Total iast_TELEMETRY_OFF 9.19 s 542.729 ms (6.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (670.958 ms) : 0, 670958
BytebuddyAgent [candidate] (673.042 ms) : 0, 673042
GlobalTracer [baseline] (240.416 ms) : 0, 240416
GlobalTracer [candidate] (240.692 ms) : 0, 240692
AppSec [baseline] (54.593 ms) : 0, 54593
AppSec [candidate] (54.863 ms) : 0, 54863
Debugger [baseline] (6.872 ms) : 0, 6872
Debugger [candidate] (6.205 ms) : 0, 6205
Remote Config [baseline] (713.246 µs) : 0, 713
Remote Config [candidate] (725.354 µs) : 0, 725
Telemetry [baseline] (10.045 ms) : 0, 10045
Telemetry [candidate] (11.482 ms) : 0, 11482
section iast
BytebuddyAgent [baseline] (788.529 ms) : 0, 788529
BytebuddyAgent [candidate] (795.229 ms) : 0, 795229
GlobalTracer [baseline] (229.531 ms) : 0, 229531
GlobalTracer [candidate] (229.737 ms) : 0, 229737
IAST [baseline] (22.725 ms) : 0, 22725
IAST [candidate] (22.78 ms) : 0, 22780
AppSec [baseline] (56.091 ms) : 0, 56091
AppSec [candidate] (56.377 ms) : 0, 56377
Debugger [baseline] (5.852 ms) : 0, 5852
Debugger [candidate] (5.839 ms) : 0, 5839
Remote Config [baseline] (593.346 µs) : 0, 593
Remote Config [candidate] (586.118 µs) : 0, 586
Telemetry [baseline] (7.819 ms) : 0, 7819
Telemetry [candidate] (7.843 ms) : 0, 7843
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (789.516 ms) : 0, 789516
BytebuddyAgent [candidate] (792.633 ms) : 0, 792633
GlobalTracer [baseline] (230.305 ms) : 0, 230305
GlobalTracer [candidate] (231.764 ms) : 0, 231764
IAST [baseline] (22.856 ms) : 0, 22856
IAST [candidate] (23.045 ms) : 0, 23045
AppSec [baseline] (56.688 ms) : 0, 56688
AppSec [candidate] (57.069 ms) : 0, 57069
Debugger [baseline] (5.932 ms) : 0, 5932
Debugger [candidate] (5.94 ms) : 0, 5940
Remote Config [baseline] (601.238 µs) : 0, 601
Remote Config [candidate] (614.031 µs) : 0, 614
Telemetry [baseline] (7.967 ms) : 0, 7967
Telemetry [candidate] (8.004 ms) : 0, 8004
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (786.794 ms) : 0, 786794
BytebuddyAgent [candidate] (787.958 ms) : 0, 787958
GlobalTracer [baseline] (230.094 ms) : 0, 230094
GlobalTracer [candidate] (230.839 ms) : 0, 230839
IAST [baseline] (22.386 ms) : 0, 22386
IAST [candidate] (22.353 ms) : 0, 22353
AppSec [baseline] (56.884 ms) : 0, 56884
AppSec [candidate] (56.456 ms) : 0, 56456
Debugger [baseline] (5.974 ms) : 0, 5974
Debugger [candidate] (5.935 ms) : 0, 5935
Remote Config [baseline] (596.156 µs) : 0, 596
Remote Config [candidate] (614.423 µs) : 0, 614
Telemetry [baseline] (7.807 ms) : 0, 7807
Telemetry [candidate] (7.791 ms) : 0, 7791
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.008 s) : 0, 1008470
Total [baseline] (10.485 s) : 0, 10484825
Agent [candidate] (1.01 s) : 0, 1009968
Total [candidate] (10.518 s) : 0, 10517696
section appsec
Agent [baseline] (1.156 s) : 0, 1156323
Total [baseline] (10.672 s) : 0, 10672352
Agent [candidate] (1.158 s) : 0, 1158488
Total [candidate] (10.702 s) : 0, 10702382
section iast
Agent [baseline] (1.153 s) : 0, 1152544
Total [baseline] (10.862 s) : 0, 10862095
Agent [candidate] (1.153 s) : 0, 1153342
Total [candidate] (10.866 s) : 0, 10866446
section profiling
Agent [baseline] (1.262 s) : 0, 1261834
Total [baseline] (10.853 s) : 0, 10853009
Agent [candidate] (1.274 s) : 0, 1273757
Total [candidate] (10.936 s) : 0, 10935623
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.008 s -
Agent appsec 1.156 s 147.854 ms (14.7%)
Agent iast 1.153 s 144.074 ms (14.3%)
Agent profiling 1.262 s 253.365 ms (25.1%)
Total tracing 10.485 s -
Total appsec 10.672 s 187.527 ms (1.8%)
Total iast 10.862 s 377.27 ms (3.6%)
Total profiling 10.853 s 368.184 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.01 s -
Agent appsec 1.158 s 148.52 ms (14.7%)
Agent iast 1.153 s 143.374 ms (14.2%)
Agent profiling 1.274 s 263.789 ms (26.1%)
Total tracing 10.518 s -
Total appsec 10.702 s 184.686 ms (1.8%)
Total iast 10.866 s 348.75 ms (3.3%)
Total profiling 10.936 s 417.927 ms (4.0%) 
gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (672.515 ms) : 0, 672515
BytebuddyAgent [candidate] (672.785 ms) : 0, 672785
GlobalTracer [baseline] (241.094 ms) : 0, 241094
GlobalTracer [candidate] (241.248 ms) : 0, 241248
AppSec [baseline] (54.555 ms) : 0, 54555
AppSec [candidate] (54.795 ms) : 0, 54795
Debugger [baseline] (6.147 ms) : 0, 6147
Debugger [candidate] (6.142 ms) : 0, 6142
Remote Config [baseline] (721.334 µs) : 0, 721
Remote Config [candidate] (714.486 µs) : 0, 714
Telemetry [baseline] (9.929 ms) : 0, 9929
Telemetry [candidate] (10.726 ms) : 0, 10726
section appsec
BytebuddyAgent [baseline] (693.031 ms) : 0, 693031
BytebuddyAgent [candidate] (694.26 ms) : 0, 694260
GlobalTracer [baseline] (237.99 ms) : 0, 237990
GlobalTracer [candidate] (238.357 ms) : 0, 238357
AppSec [baseline] (176.262 ms) : 0, 176262
AppSec [candidate] (176.697 ms) : 0, 176697
Debugger [baseline] (5.881 ms) : 0, 5881
Debugger [candidate] (5.972 ms) : 0, 5972
Remote Config [baseline] (632.786 µs) : 0, 633
Remote Config [candidate] (659.687 µs) : 0, 660
Telemetry [baseline] (7.752 ms) : 0, 7752
Telemetry [candidate] (7.864 ms) : 0, 7864
IAST [baseline] (22.331 ms) : 0, 22331
IAST [candidate] (22.214 ms) : 0, 22214
section iast
BytebuddyAgent [baseline] (801.029 ms) : 0, 801029
BytebuddyAgent [candidate] (802.481 ms) : 0, 802481
GlobalTracer [baseline] (232.571 ms) : 0, 232571
GlobalTracer [candidate] (232.624 ms) : 0, 232624
AppSec [baseline] (57.125 ms) : 0, 57125
AppSec [candidate] (56.656 ms) : 0, 56656
Debugger [baseline] (5.982 ms) : 0, 5982
Debugger [candidate] (5.961 ms) : 0, 5961
Remote Config [baseline] (611.757 µs) : 0, 612
Remote Config [candidate] (604.418 µs) : 0, 604
Telemetry [baseline] (8.059 ms) : 0, 8059
Telemetry [candidate] (8.014 ms) : 0, 8014
IAST [baseline] (23.426 ms) : 0, 23426
IAST [candidate] (23.155 ms) : 0, 23155
section profiling
BytebuddyAgent [baseline] (666.744 ms) : 0, 666744
BytebuddyAgent [candidate] (673.478 ms) : 0, 673478
GlobalTracer [baseline] (376.963 ms) : 0, 376963
GlobalTracer [candidate] (380.234 ms) : 0, 380234
AppSec [baseline] (54.615 ms) : 0, 54615
AppSec [candidate] (54.131 ms) : 0, 54131
Debugger [baseline] (6.143 ms) : 0, 6143
Debugger [candidate] (6.233 ms) : 0, 6233
Remote Config [baseline] (663.064 µs) : 0, 663
Remote Config [candidate] (671.362 µs) : 0, 671
Telemetry [baseline] (8.151 ms) : 0, 8151
Telemetry [candidate] (8.367 ms) : 0, 8367
ProfilingAgent [baseline] (97.976 ms) : 0, 97976
ProfilingAgent [candidate] (99.724 ms) : 0, 99724
Profiling [baseline] (98.0 ms) : 0, 98000
Profiling [candidate] (99.749 ms) : 0, 99749
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-25T21:28:30 2025-04-25T21:36:15
git_branch master sarahchen6/add-jdk-24-test
git_commit_date 1745613844 1745615243
git_commit_sha 4d01d58 9d76aeb
release_version 1.49.0-SNAPSHOT~4d01d583bb 1.49.0-SNAPSHOT~9d76aeb34c
start_time 2025-04-25T21:28:16 2025-04-25T21:36:00
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745617372 1745617372
ci_job_id 913756516 913756516
ci_pipeline_id 63401101 63401101
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-syfpehly-project-304-concurrent-0-ayj9p1ia 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-syfpehly-project-304-concurrent-0-ayj9p1ia 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (386.362 µs) : 367, 406
.   : milestone, 386,
iast (527.965 µs) : 505, 551
.   : milestone, 528,
iast_FULL (736.601 µs) : 714, 760
.   : milestone, 737,
iast_GLOBAL (576.13 µs) : 552, 600
.   : milestone, 576,
iast_HARDCODED_SECRET_DISABLED (513.262 µs) : 491, 536
.   : milestone, 513,
iast_INACTIVE (466.749 µs) : 445, 489
.   : milestone, 467,
iast_TELEMETRY_OFF (520.31 µs) : 498, 543
.   : milestone, 520,
tracing (464.516 µs) : 442, 487
.   : milestone, 465,
section candidate
no_agent (385.965 µs) : 366, 406
.   : milestone, 386,
iast (522.769 µs) : 499, 546
.   : milestone, 523,
iast_FULL (732.808 µs) : 709, 756
.   : milestone, 733,
iast_GLOBAL (570.471 µs) : 546, 595
.   : milestone, 570,
iast_HARDCODED_SECRET_DISABLED (531.416 µs) : 509, 554
.   : milestone, 531,
iast_INACTIVE (471.595 µs) : 449, 494
.   : milestone, 472,
iast_TELEMETRY_OFF (505.718 µs) : 483, 529
.   : milestone, 506,
tracing (467.024 µs) : 445, 489
.   : milestone, 467,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 386.362 µs [366.501 µs, 406.222 µs] -
iast 527.965 µs [505.056 µs, 550.875 µs] 141.604 µs (36.7%)
iast_FULL 736.601 µs [713.629 µs, 759.572 µs] 350.239 µs (90.7%)
iast_GLOBAL 576.13 µs [552.399 µs, 599.862 µs] 189.769 µs (49.1%)
iast_HARDCODED_SECRET_DISABLED 513.262 µs [490.522 µs, 536.003 µs] 126.901 µs (32.8%)
iast_INACTIVE 466.749 µs [444.828 µs, 488.669 µs] 80.387 µs (20.8%)
iast_TELEMETRY_OFF 520.31 µs [497.787 µs, 542.833 µs] 133.948 µs (34.7%)
tracing 464.516 µs [442.325 µs, 486.707 µs] 78.155 µs (20.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.965 µs [366.271 µs, 405.66 µs] -
iast 522.769 µs [499.324 µs, 546.213 µs] 136.803 µs (35.4%)
iast_FULL 732.808 µs [709.493 µs, 756.123 µs] 346.843 µs (89.9%)
iast_GLOBAL 570.471 µs [546.384 µs, 594.558 µs] 184.506 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED 531.416 µs [508.894 µs, 553.939 µs] 145.451 µs (37.7%)
iast_INACTIVE 471.595 µs [448.848 µs, 494.342 µs] 85.63 µs (22.2%)
iast_TELEMETRY_OFF 505.718 µs [482.917 µs, 528.519 µs] 119.753 µs (31.0%)
tracing 467.024 µs [444.694 µs, 489.354 µs] 81.059 µs (21.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.36 ms) : 1341, 1379
.   : milestone, 1360,
appsec (1.725 ms) : 1701, 1749
.   : milestone, 1725,
appsec_no_iast (1.735 ms) : 1712, 1758
.   : milestone, 1735,
code_origins (1.681 ms) : 1654, 1708
.   : milestone, 1681,
iast (1.518 ms) : 1494, 1541
.   : milestone, 1518,
profiling (1.531 ms) : 1508, 1554
.   : milestone, 1531,
tracing (1.495 ms) : 1470, 1520
.   : milestone, 1495,
section candidate
no_agent (1.376 ms) : 1357, 1396
.   : milestone, 1376,
appsec (1.724 ms) : 1699, 1748
.   : milestone, 1724,
appsec_no_iast (1.741 ms) : 1718, 1764
.   : milestone, 1741,
code_origins (1.676 ms) : 1649, 1704
.   : milestone, 1676,
iast (1.524 ms) : 1500, 1548
.   : milestone, 1524,
profiling (1.506 ms) : 1481, 1530
.   : milestone, 1506,
tracing (1.494 ms) : 1469, 1519
.   : milestone, 1494,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.36 ms [1.341 ms, 1.379 ms] -
appsec 1.725 ms [1.701 ms, 1.749 ms] 365.295 µs (26.9%)
appsec_no_iast 1.735 ms [1.712 ms, 1.758 ms] 375.164 µs (27.6%)
code_origins 1.681 ms [1.654 ms, 1.708 ms] 321.269 µs (23.6%)
iast 1.518 ms [1.494 ms, 1.541 ms] 157.791 µs (11.6%)
profiling 1.531 ms [1.508 ms, 1.554 ms] 171.481 µs (12.6%)
tracing 1.495 ms [1.47 ms, 1.52 ms] 134.929 µs (9.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.376 ms [1.357 ms, 1.396 ms] -
appsec 1.724 ms [1.699 ms, 1.748 ms] 347.282 µs (25.2%)
appsec_no_iast 1.741 ms [1.718 ms, 1.764 ms] 364.241 µs (26.5%)
code_origins 1.676 ms [1.649 ms, 1.704 ms] 300.067 µs (21.8%)
iast 1.524 ms [1.5 ms, 1.548 ms] 147.619 µs (10.7%)
profiling 1.506 ms [1.481 ms, 1.53 ms] 129.456 µs (9.4%)
tracing 1.494 ms [1.469 ms, 1.519 ms] 117.604 µs (8.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/add-jdk-24-test
git_commit_date 1745613844 1745615243
git_commit_sha 4d01d58 9d76aeb
release_version 1.49.0-SNAPSHOT~4d01d583bb 1.49.0-SNAPSHOT~9d76aeb34c
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1745617876 1745617876
ci_job_id 913756517 913756517
ci_pipeline_id 63401101 63401101
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-px9uuxed-project-304-concurrent-1-p44dvvql 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-px9uuxed-project-304-concurrent-1-p44dvvql 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.263 s) : 15263000, 15263000
.   : milestone, 15263000,
appsec (14.873 s) : 14873000, 14873000
.   : milestone, 14873000,
iast (18.907 s) : 18907000, 18907000
.   : milestone, 18907000,
iast_GLOBAL (18.258 s) : 18258000, 18258000
.   : milestone, 18258000,
profiling (15.029 s) : 15029000, 15029000
.   : milestone, 15029000,
tracing (14.816 s) : 14816000, 14816000
.   : milestone, 14816000,
section candidate
no_agent (15.489 s) : 15489000, 15489000
.   : milestone, 15489000,
appsec (14.947 s) : 14947000, 14947000
.   : milestone, 14947000,
iast (18.48 s) : 18480000, 18480000
.   : milestone, 18480000,
iast_GLOBAL (17.858 s) : 17858000, 17858000
.   : milestone, 17858000,
profiling (15.572 s) : 15572000, 15572000
.   : milestone, 15572000,
tracing (14.949 s) : 14949000, 14949000
.   : milestone, 14949000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.263 s [15.263 s, 15.263 s] -
appsec 14.873 s [14.873 s, 14.873 s] -390.0 ms (-2.6%)
iast 18.907 s [18.907 s, 18.907 s] 3.644 s (23.9%)
iast_GLOBAL 18.258 s [18.258 s, 18.258 s] 2.995 s (19.6%)
profiling 15.029 s [15.029 s, 15.029 s] -234.0 ms (-1.5%)
tracing 14.816 s [14.816 s, 14.816 s] -447.0 ms (-2.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.489 s [15.489 s, 15.489 s] -
appsec 14.947 s [14.947 s, 14.947 s] -542.0 ms (-3.5%)
iast 18.48 s [18.48 s, 18.48 s] 2.991 s (19.3%)
iast_GLOBAL 17.858 s [17.858 s, 17.858 s] 2.369 s (15.3%)
profiling 15.572 s [15.572 s, 15.572 s] 83.0 ms (0.5%)
tracing 14.949 s [14.949 s, 14.949 s] -540.0 ms (-3.5%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.377 ms) : 2330, 2424
.   : milestone, 2377,
iast (2.15 ms) : 2091, 2208
.   : milestone, 2150,
iast_GLOBAL (2.188 ms) : 2129, 2247
.   : milestone, 2188,
profiling (2.452 ms) : 2283, 2620
.   : milestone, 2452,
tracing (1.977 ms) : 1931, 2022
.   : milestone, 1977,
section candidate
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (2.369 ms) : 2323, 2416
.   : milestone, 2369,
iast (2.15 ms) : 2091, 2209
.   : milestone, 2150,
iast_GLOBAL (2.187 ms) : 2129, 2246
.   : milestone, 2187,
profiling (2.459 ms) : 2279, 2638
.   : milestone, 2459,
tracing (1.985 ms) : 1939, 2030
.   : milestone, 1985,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 2.377 ms [2.33 ms, 2.424 ms] 906.88 µs (61.7%)
iast 2.15 ms [2.091 ms, 2.208 ms] 679.122 µs (46.2%)
iast_GLOBAL 2.188 ms [2.129 ms, 2.247 ms] 717.54 µs (48.8%)
profiling 2.452 ms [2.283 ms, 2.62 ms] 981.218 µs (66.7%)
tracing 1.977 ms [1.931 ms, 2.022 ms] 506.145 µs (34.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.461 ms, 1.484 ms] -
appsec 2.369 ms [2.323 ms, 2.416 ms] 896.558 µs (60.9%)
iast 2.15 ms [2.091 ms, 2.209 ms] 677.298 µs (46.0%)
iast_GLOBAL 2.187 ms [2.129 ms, 2.246 ms] 714.666 µs (48.5%)
profiling 2.459 ms [2.279 ms, 2.638 ms] 985.945 µs (66.9%)
tracing 1.985 ms [1.939 ms, 2.03 ms] 511.88 µs (34.8%)

PerfectSlayer
PerfectSlayer reviewed Apr 28, 2025
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sarahchen6

Where does this requirement come from?
I plan to address the question about JVM testing at next the guild meeting, and with the CI migration actors too, as it will need to be coordinated in the 3 CI platforms.
Can we sync about it internally first?

In addition, the value 24 won’t work as it is not supported yet in our builder image.

@PerfectSlayer PerfectSlayer added the tag: no release notes Changes to exclude from release notes label Apr 28, 2025
@sarahchen6
Copy link
Contributor Author

Hi @PerfectSlayer yes, sounds great - let's sync internally first! I was trying to get started with adding Java 24 tests here: https://datadoghq.atlassian.net/browse/LANGPLAT-421, but missing a lot.. we should definitely sync first 😅 Thanks!

@sarahchen6 sarahchen6 closed this Apr 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer left review comments

Assignees
No one assigned
Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sarahchen6 @PerfectSlayer