Test http server requests in parallel #8222

amarziali · 2025-01-16T09:48:08Z

What Does This Do

Use parallelism to execute http server request for the scenario test success with #count requests

Motivation

Improves issue detection of multithreaded cases. I'll need also to test a fix for vertx worker context propagation in case of multiple parallel requests

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

github-actions · 2025-01-16T09:48:21Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

pr-commenter · 2025-01-16T10:28:46Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/test-http-parallel
git_commit_date	1737040366	1737041004
git_commit_sha	`679f9e2`	`adf899d`
release_version	1.46.0-SNAPSHOT~679f9e2ece	1.46.0-SNAPSHOT~adf899d34f

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737043332	1737043332
ci_job_id	769163456	769163456
ci_pipeline_id	53197386	53197386
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~adf899d34f, baseline=1.46.0-SNAPSHOT~679f9e2ece

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060464
Total [baseline] (8.655 s) : 0, 8654645
Agent [candidate] (1.069 s) : 0, 1068560
Total [candidate] (8.672 s) : 0, 8672396
section iast
Agent [baseline] (1.184 s) : 0, 1183811
Total [baseline] (9.206 s) : 0, 9205688
Agent [candidate] (1.182 s) : 0, 1182308
Total [candidate] (9.243 s) : 0, 9242715
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.185 s) : 0, 1184715
Total [baseline] (9.191 s) : 0, 9191110
Agent [candidate] (1.183 s) : 0, 1183026
Total [candidate] (9.226 s) : 0, 9226427
section iast_TELEMETRY_OFF
Agent [baseline] (1.177 s) : 0, 1177373
Total [baseline] (9.174 s) : 0, 9174147
Agent [candidate] (1.183 s) : 0, 1183211
Total [candidate] (9.212 s) : 0, 9212108

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	iast	1.184 s	123.347 ms (11.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.185 s	124.251 ms (11.7%)
Agent	iast_TELEMETRY_OFF	1.177 s	116.909 ms (11.0%)
Total	tracing	8.655 s	-
Total	iast	9.206 s	551.043 ms (6.4%)
Total	iast_HARDCODED_SECRET_DISABLED	9.191 s	536.465 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.174 s	519.502 ms (6.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.069 s	-
Agent	iast	1.182 s	113.748 ms (10.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.183 s	114.465 ms (10.7%)
Agent	iast_TELEMETRY_OFF	1.183 s	114.651 ms (10.7%)
Total	tracing	8.672 s	-
Total	iast	9.243 s	570.319 ms (6.6%)
Total	iast_HARDCODED_SECRET_DISABLED	9.226 s	554.031 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.212 s	539.711 ms (6.2%)

gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~adf899d34f, baseline=1.46.0-SNAPSHOT~679f9e2ece

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.492 ms) : 0, 717492
BytebuddyAgent [candidate] (724.313 ms) : 0, 724313
GlobalTracer [baseline] (257.207 ms) : 0, 257207
GlobalTracer [candidate] (259.188 ms) : 0, 259188
AppSec [baseline] (55.446 ms) : 0, 55446
AppSec [candidate] (57.342 ms) : 0, 57342
Remote Config [baseline] (733.223 µs) : 0, 733
Remote Config [candidate] (735.876 µs) : 0, 736
Telemetry [baseline] (14.426 ms) : 0, 14426
Telemetry [candidate] (11.726 ms) : 0, 11726
section iast
BytebuddyAgent [baseline] (832.805 ms) : 0, 832805
BytebuddyAgent [candidate] (831.978 ms) : 0, 831978
GlobalTracer [baseline] (246.812 ms) : 0, 246812
GlobalTracer [candidate] (246.538 ms) : 0, 246538
AppSec [baseline] (58.173 ms) : 0, 58173
AppSec [candidate] (57.728 ms) : 0, 57728
IAST [baseline] (21.518 ms) : 0, 21518
IAST [candidate] (21.521 ms) : 0, 21521
Remote Config [baseline] (668.778 µs) : 0, 669
Remote Config [candidate] (662.821 µs) : 0, 663
Telemetry [baseline] (8.808 ms) : 0, 8808
Telemetry [candidate] (8.817 ms) : 0, 8817
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (833.177 ms) : 0, 833177
BytebuddyAgent [candidate] (832.136 ms) : 0, 832136
GlobalTracer [baseline] (247.28 ms) : 0, 247280
GlobalTracer [candidate] (246.778 ms) : 0, 246778
AppSec [baseline] (58.062 ms) : 0, 58062
AppSec [candidate] (57.979 ms) : 0, 57979
IAST [baseline] (21.545 ms) : 0, 21545
IAST [candidate] (21.563 ms) : 0, 21563
Remote Config [baseline] (674.726 µs) : 0, 675
Remote Config [candidate] (670.361 µs) : 0, 670
Telemetry [baseline] (8.918 ms) : 0, 8918
Telemetry [candidate] (8.844 ms) : 0, 8844
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (827.951 ms) : 0, 827951
BytebuddyAgent [candidate] (833.323 ms) : 0, 833323
GlobalTracer [baseline] (246.297 ms) : 0, 246297
GlobalTracer [candidate] (246.599 ms) : 0, 246599
AppSec [baseline] (57.883 ms) : 0, 57883
AppSec [candidate] (57.776 ms) : 0, 57776
IAST [baseline] (20.948 ms) : 0, 20948
IAST [candidate] (21.066 ms) : 0, 21066
Remote Config [baseline] (676.984 µs) : 0, 677
Remote Config [candidate] (650.848 µs) : 0, 651
Telemetry [baseline] (8.667 ms) : 0, 8667
Telemetry [candidate] (8.647 ms) : 0, 8647

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~adf899d34f, baseline=1.46.0-SNAPSHOT~679f9e2ece

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1054772
Total [baseline] (10.481 s) : 0, 10481006
Agent [candidate] (1.056 s) : 0, 1056494
Total [candidate] (10.54 s) : 0, 10539837
section appsec
Agent [baseline] (1.188 s) : 0, 1187509
Total [baseline] (10.709 s) : 0, 10708673
Agent [candidate] (1.191 s) : 0, 1191280
Total [candidate] (10.714 s) : 0, 10714420
section iast
Agent [baseline] (1.184 s) : 0, 1183681
Total [baseline] (10.962 s) : 0, 10961644
Agent [candidate] (1.182 s) : 0, 1182167
Total [candidate] (10.969 s) : 0, 10969150
section profiling
Agent [baseline] (1.255 s) : 0, 1255264
Total [baseline] (10.88 s) : 0, 10880213
Agent [candidate] (1.253 s) : 0, 1253335
Total [candidate] (10.823 s) : 0, 10822753

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.055 s	-
Agent	appsec	1.188 s	132.737 ms (12.6%)
Agent	iast	1.184 s	128.909 ms (12.2%)
Agent	profiling	1.255 s	200.493 ms (19.0%)
Total	tracing	10.481 s	-
Total	appsec	10.709 s	227.667 ms (2.2%)
Total	iast	10.962 s	480.639 ms (4.6%)
Total	profiling	10.88 s	399.207 ms (3.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	appsec	1.191 s	134.786 ms (12.8%)
Agent	iast	1.182 s	125.672 ms (11.9%)
Agent	profiling	1.253 s	196.841 ms (18.6%)
Total	tracing	10.54 s	-
Total	appsec	10.714 s	174.584 ms (1.7%)
Total	iast	10.969 s	429.314 ms (4.1%)
Total	profiling	10.823 s	282.916 ms (2.7%)

gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~adf899d34f, baseline=1.46.0-SNAPSHOT~679f9e2ece

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.141 ms) : 0, 714141
BytebuddyAgent [candidate] (714.383 ms) : 0, 714383
GlobalTracer [baseline] (255.671 ms) : 0, 255671
GlobalTracer [candidate] (255.737 ms) : 0, 255737
AppSec [baseline] (55.187 ms) : 0, 55187
AppSec [candidate] (57.444 ms) : 0, 57444
Remote Config [baseline] (725.657 µs) : 0, 726
Remote Config [candidate] (722.241 µs) : 0, 722
Telemetry [baseline] (14.017 ms) : 0, 14017
Telemetry [candidate] (13.131 ms) : 0, 13131
section appsec
BytebuddyAgent [baseline] (730.187 ms) : 0, 730187
BytebuddyAgent [candidate] (732.553 ms) : 0, 732553
GlobalTracer [baseline] (252.836 ms) : 0, 252836
GlobalTracer [candidate] (253.701 ms) : 0, 253701
AppSec [baseline] (170.997 ms) : 0, 170997
AppSec [candidate] (170.719 ms) : 0, 170719
Remote Config [baseline] (664.391 µs) : 0, 664
Remote Config [candidate] (668.617 µs) : 0, 669
Telemetry [baseline] (8.19 ms) : 0, 8190
Telemetry [candidate] (8.559 ms) : 0, 8559
IAST [baseline] (19.364 ms) : 0, 19364
IAST [candidate] (20.051 ms) : 0, 20051
section iast
BytebuddyAgent [baseline] (832.555 ms) : 0, 832555
BytebuddyAgent [candidate] (831.558 ms) : 0, 831558
GlobalTracer [baseline] (246.792 ms) : 0, 246792
GlobalTracer [candidate] (246.573 ms) : 0, 246573
AppSec [baseline] (58.123 ms) : 0, 58123
AppSec [candidate] (58.007 ms) : 0, 58007
Remote Config [baseline] (668.203 µs) : 0, 668
Remote Config [candidate] (678.065 µs) : 0, 678
Telemetry [baseline] (8.821 ms) : 0, 8821
Telemetry [candidate] (8.772 ms) : 0, 8772
IAST [baseline] (21.761 ms) : 0, 21761
IAST [candidate] (21.517 ms) : 0, 21517
section profiling
ProfilingAgent [baseline] (95.805 ms) : 0, 95805
ProfilingAgent [candidate] (94.782 ms) : 0, 94782
BytebuddyAgent [baseline] (703.222 ms) : 0, 703222
BytebuddyAgent [candidate] (703.496 ms) : 0, 703496
GlobalTracer [baseline] (350.043 ms) : 0, 350043
GlobalTracer [candidate] (349.186 ms) : 0, 349186
AppSec [baseline] (54.701 ms) : 0, 54701
AppSec [candidate] (54.311 ms) : 0, 54311
Remote Config [baseline] (663.478 µs) : 0, 663
Remote Config [candidate] (676.964 µs) : 0, 677
Telemetry [baseline] (8.85 ms) : 0, 8850
Telemetry [candidate] (8.905 ms) : 0, 8905
Profiling [baseline] (95.829 ms) : 0, 95829
Profiling [candidate] (94.807 ms) : 0, 94807

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-16T15:32:10	2025-01-16T15:39:14
git_branch	master	andrea.marziali/test-http-parallel
git_commit_date	1737040366	1737041004
git_commit_sha	`679f9e2`	`adf899d`
release_version	1.46.0-SNAPSHOT~679f9e2ece	1.46.0-SNAPSHOT~adf899d34f
start_time	2025-01-16T15:31:56	2025-01-16T15:39:00

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737042312	1737042312
ci_job_id	769163457	769163457
ci_pipeline_id	53197386	53197386
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~adf899d34f, baseline=1.46.0-SNAPSHOT~679f9e2ece
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.354 ms) : 1333, 1374
.   : milestone, 1354,
appsec (1.754 ms) : 1730, 1777
.   : milestone, 1754,
appsec_no_iast (1.759 ms) : 1734, 1784
.   : milestone, 1759,
iast (1.53 ms) : 1506, 1554
.   : milestone, 1530,
profiling (1.527 ms) : 1503, 1551
.   : milestone, 1527,
tracing (1.483 ms) : 1458, 1508
.   : milestone, 1483,
section candidate
no_agent (1.356 ms) : 1336, 1376
.   : milestone, 1356,
appsec (1.738 ms) : 1714, 1762
.   : milestone, 1738,
appsec_no_iast (1.768 ms) : 1745, 1790
.   : milestone, 1768,
iast (1.541 ms) : 1516, 1565
.   : milestone, 1541,
profiling (1.508 ms) : 1484, 1532
.   : milestone, 1508,
tracing (1.471 ms) : 1446, 1496
.   : milestone, 1471,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.354 ms [1.333 ms, 1.374 ms]	-
appsec	1.754 ms [1.73 ms, 1.777 ms]	400.191 µs (29.6%)
appsec_no_iast	1.759 ms [1.734 ms, 1.784 ms]	405.683 µs (30.0%)
iast	1.53 ms [1.506 ms, 1.554 ms]	176.3 µs (13.0%)
profiling	1.527 ms [1.503 ms, 1.551 ms]	173.517 µs (12.8%)
tracing	1.483 ms [1.458 ms, 1.508 ms]	129.365 µs (9.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.356 ms [1.336 ms, 1.376 ms]	-
appsec	1.738 ms [1.714 ms, 1.762 ms]	382.169 µs (28.2%)
appsec_no_iast	1.768 ms [1.745 ms, 1.79 ms]	411.813 µs (30.4%)
iast	1.541 ms [1.516 ms, 1.565 ms]	184.877 µs (13.6%)
profiling	1.508 ms [1.484 ms, 1.532 ms]	152.051 µs (11.2%)
tracing	1.471 ms [1.446 ms, 1.496 ms]	115.308 µs (8.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~adf899d34f, baseline=1.46.0-SNAPSHOT~679f9e2ece
    dateFormat X
    axisFormat %s
section baseline
no_agent (379.999 µs) : 358, 402
.   : milestone, 380,
iast (509.568 µs) : 488, 532
.   : milestone, 510,
iast_FULL (741.821 µs) : 719, 764
.   : milestone, 742,
iast_GLOBAL (552.072 µs) : 530, 574
.   : milestone, 552,
iast_HARDCODED_SECRET_DISABLED (506.574 µs) : 485, 528
.   : milestone, 507,
iast_INACTIVE (459.586 µs) : 438, 481
.   : milestone, 460,
iast_TELEMETRY_OFF (491.462 µs) : 470, 513
.   : milestone, 491,
tracing (457.431 µs) : 436, 478
.   : milestone, 457,
section candidate
no_agent (373.616 µs) : 354, 394
.   : milestone, 374,
iast (506.387 µs) : 485, 528
.   : milestone, 506,
iast_FULL (746.707 µs) : 725, 769
.   : milestone, 747,
iast_GLOBAL (558.226 µs) : 535, 581
.   : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (503.636 µs) : 482, 525
.   : milestone, 504,
iast_INACTIVE (461.445 µs) : 440, 483
.   : milestone, 461,
iast_TELEMETRY_OFF (495.163 µs) : 473, 517
.   : milestone, 495,
tracing (451.571 µs) : 431, 472
.   : milestone, 452,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	379.999 µs [358.216 µs, 401.782 µs]	-
iast	509.568 µs [487.581 µs, 531.555 µs]	129.569 µs (34.1%)
iast_FULL	741.821 µs [719.422 µs, 764.22 µs]	361.822 µs (95.2%)
iast_GLOBAL	552.072 µs [529.729 µs, 574.415 µs]	172.073 µs (45.3%)
iast_HARDCODED_SECRET_DISABLED	506.574 µs [484.86 µs, 528.289 µs]	126.575 µs (33.3%)
iast_INACTIVE	459.586 µs [438.158 µs, 481.014 µs]	79.587 µs (20.9%)
iast_TELEMETRY_OFF	491.462 µs [469.928 µs, 512.996 µs]	111.463 µs (29.3%)
tracing	457.431 µs [436.442 µs, 478.419 µs]	77.431 µs (20.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	373.616 µs [353.625 µs, 393.607 µs]	-
iast	506.387 µs [484.507 µs, 528.267 µs]	132.771 µs (35.5%)
iast_FULL	746.707 µs [724.65 µs, 768.764 µs]	373.091 µs (99.9%)
iast_GLOBAL	558.226 µs [535.197 µs, 581.255 µs]	184.61 µs (49.4%)
iast_HARDCODED_SECRET_DISABLED	503.636 µs [482.234 µs, 525.037 µs]	130.019 µs (34.8%)
iast_INACTIVE	461.445 µs [439.668 µs, 483.223 µs]	87.829 µs (23.5%)
iast_TELEMETRY_OFF	495.163 µs [473.044 µs, 517.282 µs]	121.547 µs (32.5%)
tracing	451.571 µs [430.799 µs, 472.343 µs]	77.955 µs (20.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/test-http-parallel
git_commit_date	1737040366	1737041004
git_commit_sha	`679f9e2`	`adf899d`
release_version	1.46.0-SNAPSHOT~679f9e2ece	1.46.0-SNAPSHOT~adf899d34f

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1737043018	1737043018
ci_job_id	769163459	769163459
ci_pipeline_id	53197386	53197386
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~adf899d34f, baseline=1.46.0-SNAPSHOT~679f9e2ece
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.356 ms) : 2313, 2398
.   : milestone, 2356,
iast (2.114 ms) : 2060, 2169
.   : milestone, 2114,
iast_GLOBAL (2.162 ms) : 2106, 2217
.   : milestone, 2162,
profiling (1.962 ms) : 1919, 2005
.   : milestone, 1962,
tracing (1.955 ms) : 1913, 1997
.   : milestone, 1955,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.372 ms) : 2329, 2415
.   : milestone, 2372,
iast (2.12 ms) : 2065, 2174
.   : milestone, 2120,
iast_GLOBAL (2.154 ms) : 2099, 2208
.   : milestone, 2154,
profiling (1.977 ms) : 1933, 2021
.   : milestone, 1977,
tracing (1.945 ms) : 1904, 1987
.   : milestone, 1945,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.487 ms]	-
appsec	2.356 ms [2.313 ms, 2.398 ms]	879.569 µs (59.6%)
iast	2.114 ms [2.06 ms, 2.169 ms]	638.429 µs (43.3%)
iast_GLOBAL	2.162 ms [2.106 ms, 2.217 ms]	685.644 µs (46.5%)
profiling	1.962 ms [1.919 ms, 2.005 ms]	485.826 µs (32.9%)
tracing	1.955 ms [1.913 ms, 1.997 ms]	478.846 µs (32.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.464 ms, 1.487 ms]	-
appsec	2.372 ms [2.329 ms, 2.415 ms]	896.268 µs (60.7%)
iast	2.12 ms [2.065 ms, 2.174 ms]	644.314 µs (43.7%)
iast_GLOBAL	2.154 ms [2.099 ms, 2.208 ms]	678.082 µs (46.0%)
profiling	1.977 ms [1.933 ms, 2.021 ms]	501.742 µs (34.0%)
tracing	1.945 ms [1.904 ms, 1.987 ms]	470.018 µs (31.9%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~adf899d34f, baseline=1.46.0-SNAPSHOT~679f9e2ece
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.022 s) : 15022000, 15022000
.   : milestone, 15022000,
appsec (15.201 s) : 15201000, 15201000
.   : milestone, 15201000,
iast (18.748 s) : 18748000, 18748000
.   : milestone, 18748000,
iast_GLOBAL (18.091 s) : 18091000, 18091000
.   : milestone, 18091000,
profiling (15.727 s) : 15727000, 15727000
.   : milestone, 15727000,
tracing (15.001 s) : 15001000, 15001000
.   : milestone, 15001000,
section candidate
no_agent (14.627 s) : 14627000, 14627000
.   : milestone, 14627000,
appsec (14.821 s) : 14821000, 14821000
.   : milestone, 14821000,
iast (18.561 s) : 18561000, 18561000
.   : milestone, 18561000,
iast_GLOBAL (18.111 s) : 18111000, 18111000
.   : milestone, 18111000,
profiling (14.981 s) : 14981000, 14981000
.   : milestone, 14981000,
tracing (15.138 s) : 15138000, 15138000
.   : milestone, 15138000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.022 s [15.022 s, 15.022 s]	-
appsec	15.201 s [15.201 s, 15.201 s]	179.0 ms (1.2%)
iast	18.748 s [18.748 s, 18.748 s]	3.726 s (24.8%)
iast_GLOBAL	18.091 s [18.091 s, 18.091 s]	3.069 s (20.4%)
profiling	15.727 s [15.727 s, 15.727 s]	705.0 ms (4.7%)
tracing	15.001 s [15.001 s, 15.001 s]	-21.0 ms (-0.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.627 s [14.627 s, 14.627 s]	-
appsec	14.821 s [14.821 s, 14.821 s]	194.0 ms (1.3%)
iast	18.561 s [18.561 s, 18.561 s]	3.934 s (26.9%)
iast_GLOBAL	18.111 s [18.111 s, 18.111 s]	3.484 s (23.8%)
profiling	14.981 s [14.981 s, 14.981 s]	354.0 ms (2.4%)
tracing	15.138 s [15.138 s, 15.138 s]	511.0 ms (3.5%)

smola

Good idea. Hopefully it doesn't lead to increased flakiness, worth a try.

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-datastore](https://github.com/googleapis/java-datastore) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.25.4` -> `2.26.0` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | --- ### Release Notes <details> <summary>googleapis/java-datastore (com.google.cloud:google-cloud-datastore)</summary> ### [`v2.26.0`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2260-2025-01-29) ##### Features - Add firestoreInDatastoreMode for datastore emulator ([#1698](googleapis/java-datastore#1698)) ([50f106d](googleapis/java-datastore@50f106d)) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.42.0 ([#1725](googleapis/java-datastore#1725)) ([1cbaf22](googleapis/java-datastore@1cbaf22)) </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.46.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.46.0): 1.46.0 ##### Breaking Changes > \[!WARNING] > jnr-unixsocket is now an external dependency of dd-trace-ot and must be included when deploying dd-trace-ot. > \[!NOTE] > The API `TracerScope.setAsync(boolean)`, used to manually control asynchronous span propagation, does no more apply to the scope instance but to the active span scope. ##### Components ##### Application Security Management (IAST) - 🐛 Fix String.replace instrumentation for IAST ([#8281](DataDog/dd-trace-java#8281) - [@Mariovido](https://github.com/Mariovido)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Exclude false positive weak randomness ([#8232](DataDog/dd-trace-java#8232) - [@jandro996](https://github.com/jandro996)) - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Add security control metrics ([#8175](DataDog/dd-trace-java#8175) - [@jandro996](https://github.com/jandro996)) - ✨ Increase IAST propagation to StringBuffer setLength ([#8128](DataDog/dd-trace-java#8128) - [@Mariovido](https://github.com/Mariovido)) - ✨ Add IAST taint tracking for DB values ([#8072](DataDog/dd-trace-java#8072) - [@Mariovido](https://github.com/Mariovido)) ##### Application Security Management (WAF) - 🐛 Prevents a NPE when there is no subscriber for user events ([#8258](DataDog/dd-trace-java#8258) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Ensure cached subscriptions are cleared on reconfiguration via RC ([#8229](DataDog/dd-trace-java#8229) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Create span tag: \_dd.appsec.rasp.timeout ([#8269](DataDog/dd-trace-java#8269) - [@Mariovido](https://github.com/Mariovido)) ##### Build & Tooling - 🐛 Ensure shaded helpers have unique names when injected into class-loaders ([#8192](DataDog/dd-trace-java#8192) - [@mcculls](https://github.com/mcculls)) ##### Configuration at Runtime - 🐛 Remove filtering of `DD_SERVICE` and `DD_ENV` from the tracer ([#8176](DataDog/dd-trace-java#8176) - [@mhlidd](https://github.com/mhlidd)) ##### Continuous Integration Visibility - 🧹 Generalize TestRetryPolicy to TestExecutionPolicy ([#8302](DataDog/dd-trace-java#8302) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Parallelize CI Visibility settings requests ([#8299](DataDog/dd-trace-java#8299) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize test retry logic ([#8289](DataDog/dd-trace-java#8289) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize tests skipping logic ([#8288](DataDog/dd-trace-java#8288) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Remove skip and shouldBeSkipped methods from TestEventsHandler in favor of isSkippable ([#8286](DataDog/dd-trace-java#8286) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨⚡ Optimize Git repository information computation ([#8270](DataDog/dd-trace-java#8270) - [@dougqh](https://github.com/dougqh)) - ✨ Always request known tests from the backend ([#8268](DataDog/dd-trace-java#8268) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Fix NPE when trying to get retry analyzer in Test NG ([#8253](DataDog/dd-trace-java#8253) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Set test framework and test framework version tags atomically ([#8252](DataDog/dd-trace-java#8252) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add debug logging to Android Gradle module layout logic ([#8251](DataDog/dd-trace-java#8251) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Fix source and destination folders computation for Android Gradle projects ([#8190](DataDog/dd-trace-java#8190) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add basic Scala Weaver sbt support ([#8189](DataDog/dd-trace-java#8189) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Implement impacted tests detection ([#8188](DataDog/dd-trace-java#8188) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) ##### Data Streams Monitoring - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Database Monitoring - Add peer service tag in dbm sql commenter ([#7913](DataDog/dd-trace-java#7913) - [@jordan-wong](https://github.com/jordan-wong)) ##### Dynamic Instrumentation - ✨ Add support for SymDB to scan directories ([#8306](DataDog/dd-trace-java#8306) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add SymDB report for any jar scanning failures ([#8300](DataDog/dd-trace-java#8300) - [@jpbempel](https://github.com/jpbempel)) - ✨ Use two budgets depending on type ([#8283](DataDog/dd-trace-java#8283) - [@evanchooly](https://github.com/evanchooly)) - ✨ Institute a 10 snapshot per probe per trace budget ([#8277](DataDog/dd-trace-java#8277) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Avoid double snapshots for Exception Replay ([#8273](DataDog/dd-trace-java#8273) - [@jpbempel](https://github.com/jpbempel)) - ✨ Simplify code origins. Separate out snapshot generation. ([#8263](DataDog/dd-trace-java#8263) - [@evanchooly](https://github.com/evanchooly)) - ✨ Add Exception probe custom instrumentation ([#8230](DataDog/dd-trace-java#8230) - [@jpbempel](https://github.com/jpbempel)) - ✨ Enhance log probes to honor debug session tags ([#8215](DataDog/dd-trace-java#8215) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Don't redact env tokens from debugger probe snapshots ([#8211](DataDog/dd-trace-java#8211) - [@watson](https://github.com/watson)) - ✨⚡ Move Trace/SpanId capture at commit time ([#8184](DataDog/dd-trace-java#8184) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Capture values at entry for method probe ([#8169](DataDog/dd-trace-java#8169) - [@jpbempel](https://github.com/jpbempel)) ##### JMX fetch - 🐛 Mute JMXFetch Shutdown in progress error ([#8068](DataDog/dd-trace-java#8068) - [@ygree](https://github.com/ygree)) ##### OpenTracing - ⚠️🧹 Make jnr-unixsocket an explicit dependency of dd-trace-ot ([#8307](DataDog/dd-trace-java#8307) - [@mcculls](https://github.com/mcculls)) ##### Profiling - 🐛 Avoid unsupported API call for creating folders on windows ([#8304](DataDog/dd-trace-java#8304) - [@jbachorik](https://github.com/jbachorik)) - ✨ Tag profiles for serverless ([#8279](DataDog/dd-trace-java#8279) - [@jbachorik](https://github.com/jbachorik)) - ✨ add queue type and length to queue events ([#8242](DataDog/dd-trace-java#8242) - [@richardstartin](https://github.com/richardstartin)) - 🐛 TempLocationManager Fixes and Improvements ([#8191](DataDog/dd-trace-java#8191) - [@jbachorik](https://github.com/jbachorik)) - ✨ Bump ddprof to 1.18.0 ([#8173](DataDog/dd-trace-java#8173) - [@jbachorik](https://github.com/jbachorik)) - ✨ Report profiler initialization and configuration errors to telemetry ([#8171](DataDog/dd-trace-java#8171) - [@jbachorik](https://github.com/jbachorik)) ##### Telemetry - ✨ Add pending traces report in tracer flares ([#8053](DataDog/dd-trace-java#8053) - [@mhlidd](https://github.com/mhlidd)) ##### Testing - ✨ Test http server requests in parallel ([#8222](DataDog/dd-trace-java#8222) - [@amarziali](https://github.com/amarziali)) ##### Trace context propagation - ✨ Add non default propagator registration ([#8310](DataDog/dd-trace-java#8310) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Tracer core - ✨ Probe for existence of IBMSASL or ACCP security providers ([#8276](DataDog/dd-trace-java#8276) - [@mcculls](https://github.com/mcculls)) - ✨⚡ Overhead improvement to agent feedback based sampling ([#8265](DataDog/dd-trace-java#8265) - [@dougqh](https://github.com/dougqh)) - 🧹 Move async propagation API from scope to tracer ([#8231](DataDog/dd-trace-java#8231) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Introduce context propagation API ([#8161](DataDog/dd-trace-java#8161) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨🧪 Use env-entry to add tags per webapp deployment ([#8138](DataDog/dd-trace-java#8138) - [@amarziali](https://github.com/amarziali)) - ✨ Introduce context helpers API ([#8134](DataDog/dd-trace-java#8134) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Support IPv6 values for `DD_AGENT_HOST` and `DD_TRACE_AGENT_URL` ([#7984](DataDog/dd-trace-java#7984) - [@mhlidd](https://github.com/mhlidd)) ##### Instrumentations ##### Apache HttpComponents - 🐛 Properly finish spans and support latest apache httpclient5 ([#8272](DataDog/dd-trace-java#8272) - [@amarziali](https://github.com/amarziali)) ##### AWS Lambda instrumentation - 🐛 Properly capture lambda payloads for all handler types. ([#8264](DataDog/dd-trace-java#8264) - [@purple4reina](https://github.com/purple4reina)) ##### AWS S3 instrumentation - 💡 Create S3 instrumentation + add span pointers ([#8075](DataDog/dd-trace-java#8075) - [@nhulston](https://github.com/nhulston)) ##### AWS SDK instrumentation - 🐛 Revert "Add avoid double instrumenting lambda non-streaming handlers." ([#8247](DataDog/dd-trace-java#8247) - [@nhulston](https://github.com/nhulston)) ##### Cassandra - ✨ Allow extracting keyspace from statement result ([#8239](DataDog/dd-trace-java#8239) - [@amarziali](https://github.com/amarziali)) ##### Core Java language instrumentation - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) ##### Eclipse Vert.x instrumentation - 🐛 Fix vertx worker propagation and error handling ([#8237](DataDog/dd-trace-java#8237) - [@amarziali](https://github.com/amarziali)) - ✨ Support vertx 5 ([#8220](DataDog/dd-trace-java#8220) - [@amarziali](https://github.com/amarziali)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) ##### Kafka instrumentation - 🐛 Prevent possible NPE calculating Kafka record header size ([#8292](DataDog/dd-trace-java#8292) - [@ygree](https://github.com/ygree)) ##### Mule instrumentation - 🐛 Fix crash using Mule with JPMS ([#8187](DataDog/dd-trace-java#8187) - [@amarziali](https://github.com/amarziali)) ##### Protocol Buffer instrumentation - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Spring instrumentation - 🐛 Preserve getQualifier from spring scheduling runnables ([#8293](DataDog/dd-trace-java#8293) - [@amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: bb09d47e4eed77a003f630273b4d0a84003eb899

Test http server requests in parallel

035bfdf

amarziali requested a review from a team as a code owner January 16, 2025 09:48

amarziali requested a review from mcculls January 16, 2025 09:48

amarziali added type: enhancement Enhancements and improvements comp: testing Testing labels Jan 16, 2025

use executor service

ac67ac1

mcculls approved these changes Jan 16, 2025

View reviewed changes

smola approved these changes Jan 16, 2025

View reviewed changes

amarziali requested a review from a team as a code owner January 16, 2025 14:44

exclude some parallel

adf899d

amarziali force-pushed the andrea.marziali/test-http-parallel branch from 92e8ad6 to adf899d Compare January 16, 2025 15:23

amarziali enabled auto-merge (squash) January 16, 2025 15:34

amarziali added comp: api Tracer public API and removed comp: api Tracer public API labels Jan 16, 2025

amarziali merged commit f1d359b into master Jan 16, 2025
173 of 174 checks passed

amarziali deleted the andrea.marziali/test-http-parallel branch January 16, 2025 16:17

github-actions bot added this to the 1.46.0 milestone Jan 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Test http server requests in parallel #8222

Test http server requests in parallel #8222

Uh oh!

amarziali commented Jan 16, 2025

Uh oh!

github-actions bot commented Jan 16, 2025

Uh oh!

pr-commenter bot commented Jan 16, 2025 •

edited

Loading

Uh oh!

smola left a comment

Uh oh!

Uh oh!

Uh oh!

Test http server requests in parallel #8222

Test http server requests in parallel #8222

Uh oh!

Conversation

amarziali commented Jan 16, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Jan 16, 2025

Uh oh!

pr-commenter bot commented Jan 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

smola left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pr-commenter bot commented Jan 16, 2025 •

edited

Loading