Gary/llmobs java sdk integration #8519

gary-huang · 2025-03-07T00:08:01Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

github-actions · 2025-03-07T00:08:11Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

pr-commenter · 2025-03-07T00:51:42Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	gary/llmobs-java-sdk-integration
git_commit_date	1747229243	1747239289
git_commit_sha	`7c80dbe`	`a8f7f7e`
release_version	1.50.0-SNAPSHOT~7c80dbede3	1.49.0-SNAPSHOT~a8f7f7e7e9

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1747242248	1747242248
ci_job_id	938361435	938361435
ci_pipeline_id	65096924	65096924
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-lrzlpyic-project-304-concurrent-0-zbwadjud 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-lrzlpyic-project-304-concurrent-0-zbwadjud 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 7 performance improvements and 19 performance regressions! Performance is the same for 29 metrics, 16 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Debugger	better [-1.918ms; -1.650ms] or [-31.690%; -27.263%]	4.268ms	6.052ms
scenario:startup:insecure-bank:iast:Telemetry	worse [+517.734µs; +1019.319µs] or [+6.447%; +12.693%]	8.799ms	8.031ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start	worse [+33.615ms; +39.402ms] or [+2.927%; +3.431%]	1.185s	1.148s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:BytebuddyAgent	worse [+35.085ms; +39.703ms] or [+4.378%; +4.955%]	838.704ms	801.310ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Debugger	better [-1.709ms; -1.542ms] or [-29.027%; -26.196%]	4.261ms	5.887ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Telemetry	worse [+742.839µs; +1005.865µs] or [+9.349%; +12.659%]	8.820ms	7.946ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start	worse [+29.750ms; +63.812ms] or [+2.601%; +5.578%]	1.191s	1.144s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:BytebuddyAgent	worse [+32.505ms; +59.167ms] or [+4.073%; +7.414%]	843.865ms	798.029ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Debugger	better [-1.798ms; -1.598ms] or [-30.551%; -27.144%]	4.188ms	5.886ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config	worse [+26.757µs; +75.905µs] or [+4.553%; +12.917%]	638.971µs	587.640µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry	worse [+836.190µs; +1154.804µs] or [+10.835%; +14.963%]	8.713ms	7.718ms
scenario:startup:insecure-bank:tracing:Agent.start	worse [+34.451ms; +44.189ms] or [+3.381%; +4.337%]	1.058s	1.019s
scenario:startup:insecure-bank:tracing:BytebuddyAgent	worse [+36.105ms; +41.958ms] or [+5.291%; +6.149%]	721.391ms	682.359ms
scenario:startup:petclinic:appsec:Agent.start	worse [+32.187ms; +53.265ms] or [+2.771%; +4.586%]	1.204s	1.162s
scenario:startup:petclinic:appsec:BytebuddyAgent	worse [+35.879ms; +47.419ms] or [+5.122%; +6.769%]	742.205ms	700.556ms
scenario:startup:petclinic:appsec:Telemetry	worse [+667.059µs; +916.826µs] or [+8.938%; +12.284%]	8.255ms	7.463ms
scenario:startup:petclinic:iast:BytebuddyAgent	worse [+21.494ms; +55.633ms] or [+2.657%; +6.877%]	847.528ms	808.964ms
scenario:startup:petclinic:iast:Debugger	better [-1.819ms; -1.535ms] or [-30.797%; -25.989%]	4.230ms	5.907ms
scenario:startup:petclinic:iast:Telemetry	worse [+577.040µs; +1052.718µs] or [+7.264%; +13.252%]	8.759ms	7.944ms
scenario:startup:petclinic:profiling:BytebuddyAgent	worse [+27.709ms; +46.607ms] or [+4.088%; +6.875%]	715.051ms	677.893ms
scenario:startup:petclinic:profiling:GlobalTracer	better [-17.394ms; -9.482ms] or [-4.608%; -2.512%]	364.011ms	377.449ms
scenario:startup:petclinic:profiling:AppSec	better [-9.268ms; -7.815ms] or [-14.990%; -12.640%]	53.287ms	61.828ms
scenario:startup:petclinic:profiling:Debugger	better [-1.989ms; -1.722ms] or [-31.539%; -27.305%]	4.450ms	6.305ms
scenario:startup:petclinic:profiling:Telemetry	worse [+545.797µs; +904.732µs] or [+6.568%; +10.887%]	9.035ms	8.310ms
scenario:startup:petclinic:tracing:Agent.start	worse [+32.872ms; +41.871ms] or [+3.226%; +4.110%]	1.056s	1.019s
scenario:startup:petclinic:tracing:BytebuddyAgent	worse [+35.208ms; +39.256ms] or [+5.158%; +5.751%]	719.865ms	682.632ms

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~a8f7f7e7e9, baseline=1.50.0-SNAPSHOT~7c80dbede3

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.019 s) : 0, 1018929
Total [baseline] (8.721 s) : 0, 8721090
Agent [candidate] (1.058 s) : 0, 1058249
Total [candidate] (8.729 s) : 0, 8728741
section iast
Agent [baseline] (1.164 s) : 0, 1164195
Total [baseline] (9.265 s) : 0, 9265212
Agent [candidate] (1.186 s) : 0, 1185775
Total [candidate] (9.25 s) : 0, 9250141
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.148 s) : 0, 1148321
Total [baseline] (9.214 s) : 0, 9214179
Agent [candidate] (1.185 s) : 0, 1184829
Total [candidate] (9.248 s) : 0, 9248348
section iast_TELEMETRY_OFF
Agent [baseline] (1.144 s) : 0, 1143910
Total [baseline] (9.195 s) : 0, 9194626
Agent [candidate] (1.191 s) : 0, 1190691
Total [candidate] (9.245 s) : 0, 9245355

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.019 s	-
Agent	iast	1.164 s	145.266 ms (14.3%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.148 s	129.392 ms (12.7%)
Agent	iast_TELEMETRY_OFF	1.144 s	124.981 ms (12.3%)
Total	tracing	8.721 s	-
Total	iast	9.265 s	544.122 ms (6.2%)
Total	iast_HARDCODED_SECRET_DISABLED	9.214 s	493.089 ms (5.7%)
Total	iast_TELEMETRY_OFF	9.195 s	473.535 ms (5.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	iast	1.186 s	127.525 ms (12.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.185 s	126.58 ms (12.0%)
Agent	iast_TELEMETRY_OFF	1.191 s	132.442 ms (12.5%)
Total	tracing	8.729 s	-
Total	iast	9.25 s	521.4 ms (6.0%)
Total	iast_HARDCODED_SECRET_DISABLED	9.248 s	519.607 ms (6.0%)
Total	iast_TELEMETRY_OFF	9.245 s	516.615 ms (5.9%)

gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~a8f7f7e7e9, baseline=1.50.0-SNAPSHOT~7c80dbede3

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.359 ms) : 0, 682359
BytebuddyAgent [candidate] (721.391 ms) : 0, 721391
GlobalTracer [baseline] (239.827 ms) : 0, 239827
GlobalTracer [candidate] (239.096 ms) : 0, 239096
AppSec [baseline] (54.439 ms) : 0, 54439
AppSec [candidate] (55.747 ms) : 0, 55747
Debugger [baseline] (8.266 ms) : 0, 8266
Debugger [candidate] (4.351 ms) : 0, 4351
Remote Config [baseline] (705.542 µs) : 0, 706
Remote Config [candidate] (2.839 ms) : 0, 2839
Telemetry [baseline] (9.824 ms) : 0, 9824
Telemetry [candidate] (11.351 ms) : 0, 11351
section iast
BytebuddyAgent [baseline] (812.813 ms) : 0, 812813
BytebuddyAgent [candidate] (839.42 ms) : 0, 839420
GlobalTracer [baseline] (232.846 ms) : 0, 232846
GlobalTracer [candidate] (230.083 ms) : 0, 230083
AppSec [baseline] (52.049 ms) : 0, 52049
AppSec [candidate] (56.442 ms) : 0, 56442
Debugger [baseline] (6.052 ms) : 0, 6052
Debugger [candidate] (4.268 ms) : 0, 4268
Remote Config [baseline] (622.376 µs) : 0, 622
Remote Config [candidate] (607.666 µs) : 0, 608
Telemetry [baseline] (8.031 ms) : 0, 8031
Telemetry [candidate] (8.799 ms) : 0, 8799
IAST [baseline] (28.019 ms) : 0, 28019
IAST [candidate] (22.783 ms) : 0, 22783
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.31 ms) : 0, 801310
BytebuddyAgent [candidate] (838.704 ms) : 0, 838704
GlobalTracer [baseline] (230.507 ms) : 0, 230507
GlobalTracer [candidate] (229.475 ms) : 0, 229475
AppSec [baseline] (49.387 ms) : 0, 49387
AppSec [candidate] (55.775 ms) : 0, 55775
Debugger [baseline] (5.887 ms) : 0, 5887
Debugger [candidate] (4.261 ms) : 0, 4261
Remote Config [baseline] (592.973 µs) : 0, 593
Remote Config [candidate] (614.077 µs) : 0, 614
Telemetry [baseline] (7.946 ms) : 0, 7946
Telemetry [candidate] (8.82 ms) : 0, 8820
IAST [baseline] (29.13 ms) : 0, 29130
IAST [candidate] (23.828 ms) : 0, 23828
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (798.029 ms) : 0, 798029
BytebuddyAgent [candidate] (843.865 ms) : 0, 843865
GlobalTracer [baseline] (230.238 ms) : 0, 230238
GlobalTracer [candidate] (230.679 ms) : 0, 230679
AppSec [baseline] (54.955 ms) : 0, 54955
AppSec [candidate] (56.409 ms) : 0, 56409
Debugger [baseline] (5.886 ms) : 0, 5886
Debugger [candidate] (4.188 ms) : 0, 4188
Remote Config [baseline] (587.64 µs) : 0, 588
Remote Config [candidate] (638.971 µs) : 0, 639
Telemetry [baseline] (7.718 ms) : 0, 7718
Telemetry [candidate] (8.713 ms) : 0, 8713
IAST [baseline] (22.953 ms) : 0, 22953
IAST [candidate] (22.634 ms) : 0, 22634

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~a8f7f7e7e9, baseline=1.50.0-SNAPSHOT~7c80dbede3

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.019 s) : 0, 1018888
Total [baseline] (10.509 s) : 0, 10508884
Agent [candidate] (1.056 s) : 0, 1056260
Total [candidate] (10.45 s) : 0, 10449815
section appsec
Agent [baseline] (1.162 s) : 0, 1161539
Total [baseline] (10.682 s) : 0, 10681594
Agent [candidate] (1.204 s) : 0, 1204265
Total [candidate] (10.795 s) : 0, 10794919
section iast
Agent [baseline] (1.158 s) : 0, 1157617
Total [baseline] (10.956 s) : 0, 10956234
Agent [candidate] (1.195 s) : 0, 1195068
Total [candidate] (11.012 s) : 0, 11011770
section profiling
Agent [baseline] (1.287 s) : 0, 1287313
Total [baseline] (10.825 s) : 0, 10825231
Agent [candidate] (1.299 s) : 0, 1298976
Total [candidate] (10.918 s) : 0, 10917557

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.019 s	-
Agent	appsec	1.162 s	142.651 ms (14.0%)
Agent	iast	1.158 s	138.729 ms (13.6%)
Agent	profiling	1.287 s	268.425 ms (26.3%)
Total	tracing	10.509 s	-
Total	appsec	10.682 s	172.709 ms (1.6%)
Total	iast	10.956 s	447.35 ms (4.3%)
Total	profiling	10.825 s	316.347 ms (3.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	appsec	1.204 s	148.006 ms (14.0%)
Agent	iast	1.195 s	138.809 ms (13.1%)
Agent	profiling	1.299 s	242.716 ms (23.0%)
Total	tracing	10.45 s	-
Total	appsec	10.795 s	345.104 ms (3.3%)
Total	iast	11.012 s	561.956 ms (5.4%)
Total	profiling	10.918 s	467.743 ms (4.5%)

gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~a8f7f7e7e9, baseline=1.50.0-SNAPSHOT~7c80dbede3

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.632 ms) : 0, 682632
BytebuddyAgent [candidate] (719.865 ms) : 0, 719865
GlobalTracer [baseline] (240.672 ms) : 0, 240672
GlobalTracer [candidate] (239.37 ms) : 0, 239370
AppSec [baseline] (54.512 ms) : 0, 54512
AppSec [candidate] (54.99 ms) : 0, 54990
Debugger [baseline] (7.65 ms) : 0, 7650
Debugger [candidate] (4.385 ms) : 0, 4385
Remote Config [baseline] (710.07 µs) : 0, 710
Remote Config [candidate] (705.526 µs) : 0, 706
Telemetry [baseline] (9.062 ms) : 0, 9062
Telemetry [candidate] (13.468 ms) : 0, 13468
section appsec
BytebuddyAgent [baseline] (700.556 ms) : 0, 700556
BytebuddyAgent [candidate] (742.205 ms) : 0, 742205
GlobalTracer [baseline] (236.681 ms) : 0, 236681
GlobalTracer [candidate] (237.414 ms) : 0, 237414
AppSec [baseline] (175.818 ms) : 0, 175818
AppSec [candidate] (176.528 ms) : 0, 176528
Debugger [baseline] (5.969 ms) : 0, 5969
Debugger [candidate] (4.584 ms) : 0, 4584
Remote Config [baseline] (635.213 µs) : 0, 635
Remote Config [candidate] (759.152 µs) : 0, 759
Telemetry [baseline] (7.463 ms) : 0, 7463
Telemetry [candidate] (8.255 ms) : 0, 8255
IAST [baseline] (21.701 ms) : 0, 21701
IAST [candidate] (21.717 ms) : 0, 21717
section iast
BytebuddyAgent [baseline] (808.964 ms) : 0, 808964
BytebuddyAgent [candidate] (847.528 ms) : 0, 847528
GlobalTracer [baseline] (231.937 ms) : 0, 231937
GlobalTracer [candidate] (231.148 ms) : 0, 231148
AppSec [baseline] (50.102 ms) : 0, 50102
AppSec [candidate] (55.401 ms) : 0, 55401
Debugger [baseline] (5.907 ms) : 0, 5907
Debugger [candidate] (4.23 ms) : 0, 4230
Remote Config [baseline] (605.035 µs) : 0, 605
Remote Config [candidate] (617.755 µs) : 0, 618
Telemetry [baseline] (7.944 ms) : 0, 7944
Telemetry [candidate] (8.759 ms) : 0, 8759
IAST [baseline] (28.45 ms) : 0, 28450
IAST [candidate] (23.658 ms) : 0, 23658
section profiling
BytebuddyAgent [baseline] (677.893 ms) : 0, 677893
BytebuddyAgent [candidate] (715.051 ms) : 0, 715051
GlobalTracer [baseline] (377.449 ms) : 0, 377449
GlobalTracer [candidate] (364.011 ms) : 0, 364011
AppSec [baseline] (61.828 ms) : 0, 61828
AppSec [candidate] (53.287 ms) : 0, 53287
Debugger [baseline] (6.305 ms) : 0, 6305
Debugger [candidate] (4.45 ms) : 0, 4450
Remote Config [baseline] (662.032 µs) : 0, 662
Remote Config [candidate] (676.453 µs) : 0, 676
Telemetry [baseline] (8.31 ms) : 0, 8310
Telemetry [candidate] (9.035 ms) : 0, 9035
ProfilingAgent [baseline] (103.561 ms) : 0, 103561
ProfilingAgent [candidate] (101.937 ms) : 0, 101937
Profiling [baseline] (103.585 ms) : 0, 103585
Profiling [candidate] (101.964 ms) : 0, 101964

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-05-14T16:32:30	2025-05-14T16:40:17
git_branch	master	gary/llmobs-java-sdk-integration
git_commit_date	1747229243	1747239289
git_commit_sha	`7c80dbe`	`a8f7f7e`
release_version	1.50.0-SNAPSHOT~7c80dbede3	1.49.0-SNAPSHOT~a8f7f7e7e9
start_time	2025-05-14T16:32:16	2025-05-14T16:40:03

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1747241215	1747241215
ci_job_id	938361436	938361436
ci_pipeline_id	65096924	65096924
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-6ddtk55q-project-304-concurrent-0-zaidkp6i 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-6ddtk55q-project-304-concurrent-0-zaidkp6i 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 18 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:profiling	better [-93.766µs; -42.206µs] or [-5.946%; -2.676%]	unstable [-449.448op/s; +677.369op/s] or [-15.169%; +22.861%]	1.509ms	3076.923op/s	1.577ms	2962.963op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~a8f7f7e7e9, baseline=1.50.0-SNAPSHOT~7c80dbede3
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.362 ms) : 1342, 1383
.   : milestone, 1362,
appsec (1.746 ms) : 1722, 1770
.   : milestone, 1746,
appsec_no_iast (1.727 ms) : 1704, 1750
.   : milestone, 1727,
code_origins (1.684 ms) : 1657, 1710
.   : milestone, 1684,
iast (1.516 ms) : 1492, 1540
.   : milestone, 1516,
profiling (1.577 ms) : 1552, 1602
.   : milestone, 1577,
tracing (1.491 ms) : 1466, 1516
.   : milestone, 1491,
section candidate
no_agent (1.355 ms) : 1336, 1375
.   : milestone, 1355,
appsec (1.736 ms) : 1713, 1759
.   : milestone, 1736,
appsec_no_iast (1.735 ms) : 1712, 1759
.   : milestone, 1735,
code_origins (1.697 ms) : 1671, 1723
.   : milestone, 1697,
iast (1.519 ms) : 1494, 1544
.   : milestone, 1519,
profiling (1.509 ms) : 1486, 1532
.   : milestone, 1509,
tracing (1.501 ms) : 1476, 1526
.   : milestone, 1501,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.362 ms [1.342 ms, 1.383 ms]	-
appsec	1.746 ms [1.722 ms, 1.77 ms]	383.864 µs (28.2%)
appsec_no_iast	1.727 ms [1.704 ms, 1.75 ms]	364.938 µs (26.8%)
code_origins	1.684 ms [1.657 ms, 1.71 ms]	321.369 µs (23.6%)
iast	1.516 ms [1.492 ms, 1.54 ms]	154.046 µs (11.3%)
profiling	1.577 ms [1.552 ms, 1.602 ms]	214.754 µs (15.8%)
tracing	1.491 ms [1.466 ms, 1.516 ms]	128.864 µs (9.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.355 ms [1.336 ms, 1.375 ms]	-
appsec	1.736 ms [1.713 ms, 1.759 ms]	380.902 µs (28.1%)
appsec_no_iast	1.735 ms [1.712 ms, 1.759 ms]	379.911 µs (28.0%)
code_origins	1.697 ms [1.671 ms, 1.723 ms]	341.438 µs (25.2%)
iast	1.519 ms [1.494 ms, 1.544 ms]	163.85 µs (12.1%)
profiling	1.509 ms [1.486 ms, 1.532 ms]	153.556 µs (11.3%)
tracing	1.501 ms [1.476 ms, 1.526 ms]	145.731 µs (10.8%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~a8f7f7e7e9, baseline=1.50.0-SNAPSHOT~7c80dbede3
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.904 µs) : 359, 399
.   : milestone, 379,
iast (521.324 µs) : 498, 545
.   : milestone, 521,
iast_FULL (736.186 µs) : 714, 758
.   : milestone, 736,
iast_GLOBAL (559.121 µs) : 537, 581
.   : milestone, 559,
iast_HARDCODED_SECRET_DISABLED (524.663 µs) : 502, 547
.   : milestone, 525,
iast_INACTIVE (469.456 µs) : 447, 492
.   : milestone, 469,
iast_TELEMETRY_OFF (514.528 µs) : 491, 538
.   : milestone, 515,
tracing (463.855 µs) : 441, 486
.   : milestone, 464,
section candidate
no_agent (386.429 µs) : 366, 407
.   : milestone, 386,
iast (515.02 µs) : 493, 537
.   : milestone, 515,
iast_FULL (739.772 µs) : 718, 762
.   : milestone, 740,
iast_GLOBAL (558.213 µs) : 536, 580
.   : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (518.009 µs) : 496, 540
.   : milestone, 518,
iast_INACTIVE (469.761 µs) : 448, 491
.   : milestone, 470,
iast_TELEMETRY_OFF (497.914 µs) : 476, 520
.   : milestone, 498,
tracing (461.062 µs) : 440, 482
.   : milestone, 461,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	378.904 µs [358.611 µs, 399.197 µs]	-
iast	521.324 µs [497.664 µs, 544.985 µs]	142.42 µs (37.6%)
iast_FULL	736.186 µs [714.12 µs, 758.252 µs]	357.282 µs (94.3%)
iast_GLOBAL	559.121 µs [537.469 µs, 580.774 µs]	180.217 µs (47.6%)
iast_HARDCODED_SECRET_DISABLED	524.663 µs [502.281 µs, 547.044 µs]	145.758 µs (38.5%)
iast_INACTIVE	469.456 µs [446.648 µs, 492.264 µs]	90.552 µs (23.9%)
iast_TELEMETRY_OFF	514.528 µs [491.435 µs, 537.62 µs]	135.623 µs (35.8%)
tracing	463.855 µs [441.325 µs, 486.386 µs]	84.951 µs (22.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	386.429 µs [365.792 µs, 407.065 µs]	-
iast	515.02 µs [493.027 µs, 537.013 µs]	128.592 µs (33.3%)
iast_FULL	739.772 µs [717.546 µs, 761.998 µs]	353.344 µs (91.4%)
iast_GLOBAL	558.213 µs [536.238 µs, 580.188 µs]	171.784 µs (44.5%)
iast_HARDCODED_SECRET_DISABLED	518.009 µs [495.806 µs, 540.211 µs]	131.58 µs (34.1%)
iast_INACTIVE	469.761 µs [448.156 µs, 491.367 µs]	83.333 µs (21.6%)
iast_TELEMETRY_OFF	497.914 µs [476.174 µs, 519.654 µs]	111.485 µs (28.9%)
tracing	461.062 µs [439.858 µs, 482.266 µs]	74.633 µs (19.3%)

Dacapo

github-actions · 2025-03-07T03:37:40Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-03-07T04:49:20Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-03-07T12:33:38Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-03-07T12:36:22Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-03-07T14:09:24Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-04-21T14:01:01Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-04-21T14:19:05Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

…g provided in the override

github-actions · 2025-04-21T14:38:41Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-05-14T16:15:07Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from 058de60 to 612fd71 Compare March 7, 2025 03:37

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from e320771 to b14ebed Compare March 7, 2025 12:36

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from b14ebed to 48b1a07 Compare March 7, 2025 14:09

gary-huang added 15 commits April 10, 2025 15:09

add APIs for llm obs

5134311

add llm message class to support llm spans

b5c7a0a

add llm message class to support llm spans

7f8f586

impl llmobs agent and llmobs apis

9206dbe

support llm messages with tool calls

f5efd26

handle default model name and provider

76740e9

rm unneeded file

5e66a2f

impl llmobs agent and llmobs apis

75290af

impl llmobs agent

7b2db7c

working writer

d76c94d

add support for llm message and tool calls

e0b4510

impl llmobs agent and llmobs apis

87cdaf1

use new ctx api to track parent span

f78b2c5

add api for evals

5fc9069

working impl supporting both agentless and agent

bb6d246

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from 48b1a07 to bb6d246 Compare April 21, 2025 14:00

working impl supporting both agentless and agent

a74e456

handle null tags and default to default ml app if null or empty strin…

19d95d7

…g provided in the override

gary-huang added 2 commits April 21, 2025 10:36

working impl supporting both agentless and agent

84f6c94

Merge branch 'gary/submit-evals-2' into gary/llmobs-java-sdk-integration

80671d1

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from 0735fae to 80671d1 Compare April 21, 2025 14:38

trigger ci

a8f7f7e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gary/llmobs java sdk integration #8519

Gary/llmobs java sdk integration #8519

gary-huang commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

pr-commenter bot commented Mar 7, 2025 •

edited

Loading

github-actions bot commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

github-actions bot commented Apr 21, 2025

github-actions bot commented Apr 21, 2025

github-actions bot commented Apr 21, 2025

github-actions bot commented May 14, 2025

Gary/llmobs java sdk integration #8519

Are you sure you want to change the base?

Gary/llmobs java sdk integration #8519

Conversation

gary-huang commented Mar 7, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

github-actions bot commented Mar 7, 2025

pr-commenter bot commented Mar 7, 2025 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

github-actions bot commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

github-actions bot commented Mar 7, 2025

github-actions bot commented Apr 21, 2025

github-actions bot commented Apr 21, 2025

github-actions bot commented Apr 21, 2025

github-actions bot commented May 14, 2025

pr-commenter bot commented Mar 7, 2025 •

edited

Loading