Migrate agent integration tests to gitlab #8448

randomanderson · 2025-02-25T23:58:04Z

What Does This Do

This PR moves agent integration tests to Gitlab from CircleCI. Additionally, I extracted the testcontainer and setup logic to a base test class, AbstractTraceAgentTest, so that tests can be run locally on dev machines.

Motivation

Eventually, all jobs will be moved from CircleCI to Gitlab

Additional Notes

Test results upload to Datadog correctly works. Visualizing results in Gitlab itself doesn't work currently.

I did not remove the CircleCI jobs because the last removal was reverted in #8429

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

# Conflicts: # .gitlab-ci.yml

pr-commenter · 2025-02-25T23:58:27Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	landerson/gitlab-migration-agent-tests
git_commit_date	1740507766	1740553688
git_commit_sha	`5a2c959`	`ee55e63`
release_version	1.47.0-SNAPSHOT~5a2c959566	1.47.0-SNAPSHOT~ee55e639a9

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1740556100	1740556100
ci_job_id	823266596	823266596
ci_pipeline_id	56984197	56984197
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-2rtlzjgm-project-304-concurrent-0-exnrbwi3 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-2rtlzjgm-project-304-concurrent-0-exnrbwi3 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~ee55e639a9, baseline=1.47.0-SNAPSHOT~5a2c959566

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1046470
Total [baseline] (10.477 s) : 0, 10476930
Agent [candidate] (1.04 s) : 0, 1039615
Total [candidate] (10.483 s) : 0, 10483431
section appsec
Agent [baseline] (1.185 s) : 0, 1184955
Total [baseline] (10.749 s) : 0, 10749286
Agent [candidate] (1.187 s) : 0, 1187286
Total [candidate] (10.761 s) : 0, 10760535
section iast
Agent [baseline] (1.17 s) : 0, 1169619
Total [baseline] (11.028 s) : 0, 11027579
Agent [candidate] (1.173 s) : 0, 1172840
Total [candidate] (11.106 s) : 0, 11105789
section profiling
Agent [baseline] (1.26 s) : 0, 1260358
Total [baseline] (10.854 s) : 0, 10853701
Agent [candidate] (1.261 s) : 0, 1261177
Total [candidate] (10.849 s) : 0, 10848962

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.185 s	138.484 ms (13.2%)
Agent	iast	1.17 s	123.149 ms (11.8%)
Agent	profiling	1.26 s	213.887 ms (20.4%)
Total	tracing	10.477 s	-
Total	appsec	10.749 s	272.356 ms (2.6%)
Total	iast	11.028 s	550.649 ms (5.3%)
Total	profiling	10.854 s	376.771 ms (3.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	appsec	1.187 s	147.671 ms (14.2%)
Agent	iast	1.173 s	133.225 ms (12.8%)
Agent	profiling	1.261 s	221.562 ms (21.3%)
Total	tracing	10.483 s	-
Total	appsec	10.761 s	277.104 ms (2.6%)
Total	iast	11.106 s	622.358 ms (5.9%)
Total	profiling	10.849 s	365.531 ms (3.5%)

gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~ee55e639a9, baseline=1.47.0-SNAPSHOT~5a2c959566

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.94 ms) : 0, 720940
BytebuddyAgent [candidate] (717.447 ms) : 0, 717447
GlobalTracer [baseline] (239.8 ms) : 0, 239800
GlobalTracer [candidate] (239.429 ms) : 0, 239429
AppSec [baseline] (56.079 ms) : 0, 56079
AppSec [candidate] (55.769 ms) : 0, 55769
Remote Config [baseline] (718.076 µs) : 0, 718
Remote Config [candidate] (694.171 µs) : 0, 694
Telemetry [baseline] (13.96 ms) : 0, 13960
Telemetry [candidate] (11.397 ms) : 0, 11397
section appsec
BytebuddyAgent [baseline] (736.528 ms) : 0, 736528
BytebuddyAgent [candidate] (737.633 ms) : 0, 737633
GlobalTracer [baseline] (236.983 ms) : 0, 236983
GlobalTracer [candidate] (237.305 ms) : 0, 237305
IAST [baseline] (21.447 ms) : 0, 21447
IAST [candidate] (21.772 ms) : 0, 21772
AppSec [baseline] (176.85 ms) : 0, 176850
AppSec [candidate] (177.525 ms) : 0, 177525
Remote Config [baseline] (658.817 µs) : 0, 659
Remote Config [candidate] (661.247 µs) : 0, 661
Telemetry [baseline] (8.293 ms) : 0, 8293
Telemetry [candidate] (8.355 ms) : 0, 8355
section iast
BytebuddyAgent [baseline] (835.83 ms) : 0, 835830
BytebuddyAgent [candidate] (837.929 ms) : 0, 837929
GlobalTracer [baseline] (229.43 ms) : 0, 229430
GlobalTracer [candidate] (230.57 ms) : 0, 230570
IAST [baseline] (22.785 ms) : 0, 22785
IAST [candidate] (22.911 ms) : 0, 22911
AppSec [baseline] (57.324 ms) : 0, 57324
AppSec [candidate] (57.127 ms) : 0, 57127
Remote Config [baseline] (616.933 µs) : 0, 617
Remote Config [candidate] (614.239 µs) : 0, 614
Telemetry [baseline] (8.723 ms) : 0, 8723
Telemetry [candidate] (8.737 ms) : 0, 8737
section profiling
BytebuddyAgent [baseline] (709.129 ms) : 0, 709129
BytebuddyAgent [candidate] (709.778 ms) : 0, 709778
GlobalTracer [baseline] (349.542 ms) : 0, 349542
GlobalTracer [candidate] (350.605 ms) : 0, 350605
AppSec [baseline] (55.345 ms) : 0, 55345
AppSec [candidate] (54.618 ms) : 0, 54618
Remote Config [baseline] (664.025 µs) : 0, 664
Remote Config [candidate] (672.695 µs) : 0, 673
Telemetry [baseline] (8.957 ms) : 0, 8957
Telemetry [candidate] (8.937 ms) : 0, 8937
ProfilingAgent [baseline] (96.346 ms) : 0, 96346
ProfilingAgent [candidate] (96.28 ms) : 0, 96280
Profiling [baseline] (96.37 ms) : 0, 96370
Profiling [candidate] (96.304 ms) : 0, 96304

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~ee55e639a9, baseline=1.47.0-SNAPSHOT~5a2c959566

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1043234
Total [baseline] (8.669 s) : 0, 8669474
Agent [candidate] (1.041 s) : 0, 1041167
Total [candidate] (8.652 s) : 0, 8651590
section iast
Agent [baseline] (1.17 s) : 0, 1169940
Total [baseline] (9.221 s) : 0, 9220561
Agent [candidate] (1.172 s) : 0, 1172306
Total [candidate] (9.241 s) : 0, 9240976
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.177 s) : 0, 1176537
Total [baseline] (9.183 s) : 0, 9182884
Agent [candidate] (1.17 s) : 0, 1169918
Total [candidate] (9.18 s) : 0, 9179987
section iast_TELEMETRY_OFF
Agent [baseline] (1.176 s) : 0, 1175658
Total [baseline] (9.228 s) : 0, 9227889
Agent [candidate] (1.175 s) : 0, 1174638
Total [candidate] (9.256 s) : 0, 9255661

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.043 s	-
Agent	iast	1.17 s	126.706 ms (12.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.177 s	133.303 ms (12.8%)
Agent	iast_TELEMETRY_OFF	1.176 s	132.424 ms (12.7%)
Total	tracing	8.669 s	-
Total	iast	9.221 s	551.087 ms (6.4%)
Total	iast_HARDCODED_SECRET_DISABLED	9.183 s	513.411 ms (5.9%)
Total	iast_TELEMETRY_OFF	9.228 s	558.415 ms (6.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.041 s	-
Agent	iast	1.172 s	131.139 ms (12.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.17 s	128.751 ms (12.4%)
Agent	iast_TELEMETRY_OFF	1.175 s	133.47 ms (12.8%)
Total	tracing	8.652 s	-
Total	iast	9.241 s	589.386 ms (6.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.18 s	528.397 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.256 s	604.071 ms (7.0%)

gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~ee55e639a9, baseline=1.47.0-SNAPSHOT~5a2c959566

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.878 ms) : 0, 720878
BytebuddyAgent [candidate] (719.314 ms) : 0, 719314
GlobalTracer [baseline] (239.338 ms) : 0, 239338
GlobalTracer [candidate] (239.91 ms) : 0, 239910
AppSec [baseline] (55.786 ms) : 0, 55786
AppSec [candidate] (55.576 ms) : 0, 55576
Remote Config [baseline] (687.502 µs) : 0, 688
Remote Config [candidate] (693.481 µs) : 0, 693
Telemetry [baseline] (11.565 ms) : 0, 11565
Telemetry [candidate] (10.697 ms) : 0, 10697
section iast
BytebuddyAgent [baseline] (835.396 ms) : 0, 835396
BytebuddyAgent [candidate] (838.106 ms) : 0, 838106
GlobalTracer [baseline] (229.968 ms) : 0, 229968
GlobalTracer [candidate] (230.18 ms) : 0, 230180
IAST [baseline] (22.749 ms) : 0, 22749
IAST [candidate] (22.805 ms) : 0, 22805
AppSec [baseline] (57.517 ms) : 0, 57517
AppSec [candidate] (56.88 ms) : 0, 56880
Remote Config [baseline] (607.343 µs) : 0, 607
Remote Config [candidate] (621.338 µs) : 0, 621
Telemetry [baseline] (8.732 ms) : 0, 8732
Telemetry [candidate] (8.737 ms) : 0, 8737
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (841.569 ms) : 0, 841569
BytebuddyAgent [candidate] (835.601 ms) : 0, 835601
GlobalTracer [baseline] (230.781 ms) : 0, 230781
GlobalTracer [candidate] (230.43 ms) : 0, 230430
IAST [baseline] (22.913 ms) : 0, 22913
IAST [candidate] (22.779 ms) : 0, 22779
AppSec [baseline] (56.92 ms) : 0, 56920
AppSec [candidate] (56.908 ms) : 0, 56908
Remote Config [baseline] (602.249 µs) : 0, 602
Remote Config [candidate] (603.092 µs) : 0, 603
Telemetry [baseline] (8.662 ms) : 0, 8662
Telemetry [candidate] (8.676 ms) : 0, 8676
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (840.796 ms) : 0, 840796
BytebuddyAgent [candidate] (840.263 ms) : 0, 840263
GlobalTracer [baseline] (231.07 ms) : 0, 231070
GlobalTracer [candidate] (230.854 ms) : 0, 230854
IAST [baseline] (26.298 ms) : 0, 26298
IAST [candidate] (24.829 ms) : 0, 24829
AppSec [baseline] (53.319 ms) : 0, 53319
AppSec [candidate] (54.494 ms) : 0, 54494
Remote Config [baseline] (623.296 µs) : 0, 623
Remote Config [candidate] (606.88 µs) : 0, 607
Telemetry [baseline] (8.556 ms) : 0, 8556
Telemetry [candidate] (8.5 ms) : 0, 8500

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-02-26T07:19:31	2025-02-26T07:27:16
git_branch	master	landerson/gitlab-migration-agent-tests
git_commit_date	1740507766	1740553688
git_commit_sha	`5a2c959`	`ee55e63`
release_version	1.47.0-SNAPSHOT~5a2c959566	1.47.0-SNAPSHOT~ee55e639a9
start_time	2025-02-26T07:19:17	2025-02-26T07:27:02

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1740555235	1740555235
ci_job_id	823266597	823266597
ci_pipeline_id	56984197	56984197
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-2rtlzjgm-project-304-concurrent-1-uqe1ycwh 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-2rtlzjgm-project-304-concurrent-1-uqe1ycwh 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~ee55e639a9, baseline=1.47.0-SNAPSHOT~5a2c959566
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.364 ms) : 1345, 1384
.   : milestone, 1364,
appsec (1.761 ms) : 1737, 1786
.   : milestone, 1761,
appsec_no_iast (1.755 ms) : 1729, 1781
.   : milestone, 1755,
code_origins (1.683 ms) : 1646, 1720
.   : milestone, 1683,
iast (1.518 ms) : 1493, 1542
.   : milestone, 1518,
profiling (1.535 ms) : 1510, 1561
.   : milestone, 1535,
tracing (1.492 ms) : 1467, 1516
.   : milestone, 1492,
section candidate
no_agent (1.36 ms) : 1341, 1380
.   : milestone, 1360,
appsec (1.739 ms) : 1715, 1763
.   : milestone, 1739,
appsec_no_iast (1.759 ms) : 1735, 1783
.   : milestone, 1759,
code_origins (1.686 ms) : 1653, 1719
.   : milestone, 1686,
iast (1.522 ms) : 1499, 1546
.   : milestone, 1522,
profiling (1.53 ms) : 1506, 1554
.   : milestone, 1530,
tracing (1.499 ms) : 1474, 1524
.   : milestone, 1499,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.364 ms [1.345 ms, 1.384 ms]	-
appsec	1.761 ms [1.737 ms, 1.786 ms]	397.005 µs (29.1%)
appsec_no_iast	1.755 ms [1.729 ms, 1.781 ms]	390.589 µs (28.6%)
code_origins	1.683 ms [1.646 ms, 1.72 ms]	318.795 µs (23.4%)
iast	1.518 ms [1.493 ms, 1.542 ms]	153.502 µs (11.3%)
profiling	1.535 ms [1.51 ms, 1.561 ms]	170.787 µs (12.5%)
tracing	1.492 ms [1.467 ms, 1.516 ms]	127.172 µs (9.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.36 ms [1.341 ms, 1.38 ms]	-
appsec	1.739 ms [1.715 ms, 1.763 ms]	378.426 µs (27.8%)
appsec_no_iast	1.759 ms [1.735 ms, 1.783 ms]	398.781 µs (29.3%)
code_origins	1.686 ms [1.653 ms, 1.719 ms]	325.279 µs (23.9%)
iast	1.522 ms [1.499 ms, 1.546 ms]	161.943 µs (11.9%)
profiling	1.53 ms [1.506 ms, 1.554 ms]	169.624 µs (12.5%)
tracing	1.499 ms [1.474 ms, 1.524 ms]	138.502 µs (10.2%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~ee55e639a9, baseline=1.47.0-SNAPSHOT~5a2c959566
    dateFormat X
    axisFormat %s
section baseline
no_agent (385.114 µs) : 365, 405
.   : milestone, 385,
iast (515.037 µs) : 493, 537
.   : milestone, 515,
iast_FULL (728.91 µs) : 707, 751
.   : milestone, 729,
iast_GLOBAL (552.284 µs) : 531, 574
.   : milestone, 552,
iast_HARDCODED_SECRET_DISABLED (509.315 µs) : 487, 532
.   : milestone, 509,
iast_INACTIVE (472.542 µs) : 451, 494
.   : milestone, 473,
iast_TELEMETRY_OFF (494.656 µs) : 472, 517
.   : milestone, 495,
tracing (457.053 µs) : 436, 479
.   : milestone, 457,
section candidate
no_agent (381.428 µs) : 361, 401
.   : milestone, 381,
iast (516.581 µs) : 494, 540
.   : milestone, 517,
iast_FULL (729.167 µs) : 707, 751
.   : milestone, 729,
iast_GLOBAL (562.444 µs) : 540, 585
.   : milestone, 562,
iast_HARDCODED_SECRET_DISABLED (516.432 µs) : 493, 539
.   : milestone, 516,
iast_INACTIVE (460.492 µs) : 439, 482
.   : milestone, 460,
iast_TELEMETRY_OFF (497.388 µs) : 474, 521
.   : milestone, 497,
tracing (458.855 µs) : 438, 479
.   : milestone, 459,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	385.114 µs [365.499 µs, 404.73 µs]	-
iast	515.037 µs [492.905 µs, 537.168 µs]	129.922 µs (33.7%)
iast_FULL	728.91 µs [706.907 µs, 750.914 µs]	343.796 µs (89.3%)
iast_GLOBAL	552.284 µs [530.515 µs, 574.053 µs]	167.17 µs (43.4%)
iast_HARDCODED_SECRET_DISABLED	509.315 µs [486.525 µs, 532.105 µs]	124.201 µs (32.3%)
iast_INACTIVE	472.542 µs [451.494 µs, 493.589 µs]	87.427 µs (22.7%)
iast_TELEMETRY_OFF	494.656 µs [471.932 µs, 517.381 µs]	109.542 µs (28.4%)
tracing	457.053 µs [435.559 µs, 478.547 µs]	71.938 µs (18.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.428 µs [361.361 µs, 401.495 µs]	-
iast	516.581 µs [493.617 µs, 539.545 µs]	135.153 µs (35.4%)
iast_FULL	729.167 µs [707.273 µs, 751.062 µs]	347.74 µs (91.2%)
iast_GLOBAL	562.444 µs [539.686 µs, 585.202 µs]	181.017 µs (47.5%)
iast_HARDCODED_SECRET_DISABLED	516.432 µs [493.4 µs, 539.463 µs]	135.004 µs (35.4%)
iast_INACTIVE	460.492 µs [438.882 µs, 482.102 µs]	79.064 µs (20.7%)
iast_TELEMETRY_OFF	497.388 µs [474.237 µs, 520.539 µs]	115.96 µs (30.4%)
tracing	458.855 µs [438.427 µs, 479.283 µs]	77.427 µs (20.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	landerson/gitlab-migration-agent-tests
git_commit_date	1740507766	1740553688
git_commit_sha	`5a2c959`	`ee55e63`
release_version	1.47.0-SNAPSHOT~5a2c959566	1.47.0-SNAPSHOT~ee55e639a9

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1740555640	1740555640
ci_job_id	823266598	823266598
ci_pipeline_id	56984197	56984197
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-6zkoxsgo-project-304-concurrent-0-jgpwotnh 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-6zkoxsgo-project-304-concurrent-0-jgpwotnh 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~ee55e639a9, baseline=1.47.0-SNAPSHOT~5a2c959566
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.367 ms) : 2323, 2411
.   : milestone, 2367,
iast (2.115 ms) : 2060, 2170
.   : milestone, 2115,
iast_GLOBAL (2.16 ms) : 2105, 2216
.   : milestone, 2160,
profiling (1.967 ms) : 1923, 2010
.   : milestone, 1967,
tracing (1.961 ms) : 1918, 2003
.   : milestone, 1961,
section candidate
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (2.378 ms) : 2334, 2421
.   : milestone, 2378,
iast (2.121 ms) : 2066, 2176
.   : milestone, 2121,
iast_GLOBAL (2.167 ms) : 2112, 2223
.   : milestone, 2167,
profiling (2.475 ms) : 2292, 2658
.   : milestone, 2475,
tracing (1.949 ms) : 1907, 1991
.   : milestone, 1949,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms, 1.484 ms]	-
appsec	2.367 ms [2.323 ms, 2.411 ms]	894.868 µs (60.8%)
iast	2.115 ms [2.06 ms, 2.17 ms]	642.72 µs (43.7%)
iast_GLOBAL	2.16 ms [2.105 ms, 2.216 ms]	687.896 µs (46.7%)
profiling	1.967 ms [1.923 ms, 2.01 ms]	494.572 µs (33.6%)
tracing	1.961 ms [1.918 ms, 2.003 ms]	488.604 µs (33.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	2.378 ms [2.334 ms, 2.421 ms]	906.678 µs (61.6%)
iast	2.121 ms [2.066 ms, 2.176 ms]	649.479 µs (44.1%)
iast_GLOBAL	2.167 ms [2.112 ms, 2.223 ms]	696.006 µs (47.3%)
profiling	2.475 ms [2.292 ms, 2.658 ms]	1.004 ms (68.2%)
tracing	1.949 ms [1.907 ms, 1.991 ms]	477.624 µs (32.5%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~ee55e639a9, baseline=1.47.0-SNAPSHOT~5a2c959566
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.547 s) : 15547000, 15547000
.   : milestone, 15547000,
appsec (14.919 s) : 14919000, 14919000
.   : milestone, 14919000,
iast (18.399 s) : 18399000, 18399000
.   : milestone, 18399000,
iast_GLOBAL (17.83 s) : 17830000, 17830000
.   : milestone, 17830000,
profiling (15.688 s) : 15688000, 15688000
.   : milestone, 15688000,
tracing (14.818 s) : 14818000, 14818000
.   : milestone, 14818000,
section candidate
no_agent (15.59 s) : 15590000, 15590000
.   : milestone, 15590000,
appsec (15.109 s) : 15109000, 15109000
.   : milestone, 15109000,
iast (18.665 s) : 18665000, 18665000
.   : milestone, 18665000,
iast_GLOBAL (17.955 s) : 17955000, 17955000
.   : milestone, 17955000,
profiling (15.478 s) : 15478000, 15478000
.   : milestone, 15478000,
tracing (14.844 s) : 14844000, 14844000
.   : milestone, 14844000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.547 s [15.547 s, 15.547 s]	-
appsec	14.919 s [14.919 s, 14.919 s]	-628.0 ms (-4.0%)
iast	18.399 s [18.399 s, 18.399 s]	2.852 s (18.3%)
iast_GLOBAL	17.83 s [17.83 s, 17.83 s]	2.283 s (14.7%)
profiling	15.688 s [15.688 s, 15.688 s]	141.0 ms (0.9%)
tracing	14.818 s [14.818 s, 14.818 s]	-729.0 ms (-4.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.59 s [15.59 s, 15.59 s]	-
appsec	15.109 s [15.109 s, 15.109 s]	-481.0 ms (-3.1%)
iast	18.665 s [18.665 s, 18.665 s]	3.075 s (19.7%)
iast_GLOBAL	17.955 s [17.955 s, 17.955 s]	2.365 s (15.2%)
profiling	15.478 s [15.478 s, 15.478 s]	-112.0 ms (-0.7%)
tracing	14.844 s [14.844 s, 14.844 s]	-746.0 ms (-4.8%)

This reverts commit 90602a2.

PerfectSlayer

It's looking good. Thanks for the refactoring and enabling us to run tests locally!

randomanderson added 30 commits January 29, 2025 15:52

Add checks, spotless, and test_published_artifact

fdac48b

small fixes

317d9fa

WIP

c8524dd

syntax errors

4a4ba1b

add gradle daemon logs everywhere

b350dbf

fix c/p error

57f4c7b

Merge branch 'master' into landerson/gitlab-migration

4da532a

# Conflicts: # .gitlab-ci.yml

fix system tests dir

0145135

add some ls

9372118

always save artifacts

d5f41d3

fix workspace path for system tests

5113dbd

move more jobs to the tests stage

acba4a9

split muzzle into 6 jobs

54b5391

remove system-tests debug output

1f25bce

fix path and index for muzzle

29ac8e3

add signing args

507258c

8 parallism

2477dfa

no needs for spotless

89005db

java 7 for test published artifacts

b0d0b76

use the image for the test jvm

38b4fae

upload ci_app results

70d8a27

when always -> on_success

528e725

Cleanup TraceAgentTests to always run

a043cdb

try agent integration test

8c8f20e

add the actual target

847cd9f

clean up branch

a31d1bf

set keys and use explicit agent value in test

53d9a04

remove DD_* variables from environment before tests

42c24df

save/restore must be toplevel

4390c73

some debug logic for keys

fc66b89

randomanderson added 6 commits February 25, 2025 15:33

yaml

926e618

set api keys in after_script

facd78f

fix DDApiIntegrationTest

6200c3c

remove agent integration tests from circleci

90602a2

Merge branch 'master' into landerson/gitlab-migration-agent-tests

a2475e8

# Conflicts: # .gitlab-ci.yml

cleanup merge issues

d69f2b4

randomanderson added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Feb 25, 2025

randomanderson requested a review from a team as a code owner February 25, 2025 23:58

randomanderson requested a review from smola February 25, 2025 23:58

randomanderson added 4 commits February 25, 2025 19:04

Revert "remove agent integration tests from circleci"

e64f4ba

This reverts commit 90602a2.

assign CI_AGENT_HOST to localhost

ed827ff

don't skip tests when populating cache

ac582ed

codenarc

ee55e63

randomanderson requested a review from PerfectSlayer February 27, 2025 16:04

PerfectSlayer approved these changes Feb 28, 2025

View reviewed changes

PerfectSlayer added the type: enhancement Enhancements and improvements label Feb 28, 2025

randomanderson merged commit 93bbda1 into master Feb 28, 2025
218 checks passed

randomanderson deleted the landerson/gitlab-migration-agent-tests branch February 28, 2025 14:45

github-actions bot added this to the 1.47.0 milestone Feb 28, 2025

PerfectSlayer mentioned this pull request May 5, 2025

Update Docker build image #8754

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Migrate agent integration tests to gitlab #8448

Migrate agent integration tests to gitlab #8448

Uh oh!

randomanderson commented Feb 25, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Feb 25, 2025 •

edited

Loading

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Migrate agent integration tests to gitlab #8448

Migrate agent integration tests to gitlab #8448

Uh oh!

Conversation

randomanderson commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

randomanderson commented Feb 25, 2025 •

edited

Loading

pr-commenter bot commented Feb 25, 2025 •

edited

Loading