Skip to content

[wip] LLM Observability SDK merge branch #8781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

[wip] LLM Observability SDK merge branch #8781

wants to merge 1 commit into from
+392 −0

Conversation

gary-huang
Copy link
Contributor

  • add APIs for llm obs

  • add llm message class to support llm spans

  • follow java convention of naming Id instead of ID

  • add codeowners

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented May 7, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master gary/llmobs-sdk-merge
git_commit_date 1747229243 1747246977
git_commit_sha 7c80dbe 2be1b73
release_version 1.50.0-SNAPSHOT~7c80dbede3 1.50.0-SNAPSHOT~2be1b73b0e
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1747249383 1747249383
ci_job_id 938665298 938665298
ci_pipeline_id 65116884 65116884
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-xzg4hwwu-project-304-concurrent-1-4yzdttc1 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-xzg4hwwu-project-304-concurrent-1-4yzdttc1 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 14 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~2be1b73b0e, baseline=1.50.0-SNAPSHOT~7c80dbede3

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.019 s) : 0, 1018603
Total [baseline] (10.47 s) : 0, 10469676
Agent [candidate] (1.02 s) : 0, 1020498
Total [candidate] (10.474 s) : 0, 10474459
section appsec
Agent [baseline] (1.173 s) : 0, 1173190
Total [baseline] (10.68 s) : 0, 10679684
Agent [candidate] (1.161 s) : 0, 1161228
Total [candidate] (10.658 s) : 0, 10658239
section iast
Agent [baseline] (1.161 s) : 0, 1160726
Total [baseline] (10.931 s) : 0, 10931331
Agent [candidate] (1.148 s) : 0, 1147677
Total [candidate] (10.87 s) : 0, 10870422
section profiling
Agent [baseline] (1.282 s) : 0, 1282287
Total [baseline] (10.931 s) : 0, 10931448
Agent [candidate] (1.277 s) : 0, 1277363
Total [candidate] (10.939 s) : 0, 10939181
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.019 s -
Agent appsec 1.173 s 154.587 ms (15.2%)
Agent iast 1.161 s 142.122 ms (14.0%)
Agent profiling 1.282 s 263.684 ms (25.9%)
Total tracing 10.47 s -
Total appsec 10.68 s 210.009 ms (2.0%)
Total iast 10.931 s 461.656 ms (4.4%)
Total profiling 10.931 s 461.773 ms (4.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.02 s -
Agent appsec 1.161 s 140.73 ms (13.8%)
Agent iast 1.148 s 127.179 ms (12.5%)
Agent profiling 1.277 s 256.866 ms (25.2%)
Total tracing 10.474 s -
Total appsec 10.658 s 183.78 ms (1.8%)
Total iast 10.87 s 395.963 ms (3.8%)
Total profiling 10.939 s 464.722 ms (4.4%) 
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~2be1b73b0e, baseline=1.50.0-SNAPSHOT~7c80dbede3

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (683.006 ms) : 0, 683006
BytebuddyAgent [candidate] (682.702 ms) : 0, 682702
GlobalTracer [baseline] (240.372 ms) : 0, 240372
GlobalTracer [candidate] (240.371 ms) : 0, 240371
AppSec [baseline] (54.905 ms) : 0, 54905
AppSec [candidate] (54.456 ms) : 0, 54456
Debugger [baseline] (6.887 ms) : 0, 6887
Debugger [candidate] (8.984 ms) : 0, 8984
Remote Config [baseline] (710.403 µs) : 0, 710
Remote Config [candidate] (702.323 µs) : 0, 702
Telemetry [baseline] (9.108 ms) : 0, 9108
Telemetry [candidate] (9.735 ms) : 0, 9735
section appsec
BytebuddyAgent [baseline] (708.67 ms) : 0, 708670
BytebuddyAgent [candidate] (700.165 ms) : 0, 700165
GlobalTracer [baseline] (238.897 ms) : 0, 238897
GlobalTracer [candidate] (236.85 ms) : 0, 236850
AppSec [baseline] (176.83 ms) : 0, 176830
AppSec [candidate] (175.975 ms) : 0, 175975
Debugger [baseline] (6.012 ms) : 0, 6012
Debugger [candidate] (5.945 ms) : 0, 5945
Remote Config [baseline] (633.072 µs) : 0, 633
Remote Config [candidate] (629.389 µs) : 0, 629
Telemetry [baseline] (7.505 ms) : 0, 7505
Telemetry [candidate] (7.4 ms) : 0, 7400
IAST [baseline] (21.656 ms) : 0, 21656
IAST [candidate] (21.524 ms) : 0, 21524
section iast
BytebuddyAgent [baseline] (810.901 ms) : 0, 810901
BytebuddyAgent [candidate] (801.053 ms) : 0, 801053
GlobalTracer [baseline] (232.903 ms) : 0, 232903
GlobalTracer [candidate] (230.699 ms) : 0, 230699
AppSec [baseline] (48.051 ms) : 0, 48051
AppSec [candidate] (50.1 ms) : 0, 50100
Debugger [baseline] (5.946 ms) : 0, 5946
Debugger [candidate] (5.848 ms) : 0, 5848
Remote Config [baseline] (596.565 µs) : 0, 597
Remote Config [candidate] (586.864 µs) : 0, 587
Telemetry [baseline] (7.98 ms) : 0, 7980
Telemetry [candidate] (7.838 ms) : 0, 7838
IAST [baseline] (29.903 ms) : 0, 29903
IAST [candidate] (27.993 ms) : 0, 27993
section profiling
ProfilingAgent [baseline] (104.243 ms) : 0, 104243
ProfilingAgent [candidate] (102.42 ms) : 0, 102420
BytebuddyAgent [baseline] (673.633 ms) : 0, 673633
BytebuddyAgent [candidate] (673.195 ms) : 0, 673195
GlobalTracer [baseline] (375.857 ms) : 0, 375857
GlobalTracer [candidate] (374.231 ms) : 0, 374231
AppSec [baseline] (62.515 ms) : 0, 62515
AppSec [candidate] (61.689 ms) : 0, 61689
Debugger [baseline] (6.327 ms) : 0, 6327
Debugger [candidate] (6.266 ms) : 0, 6266
Remote Config [baseline] (674.559 µs) : 0, 675
Remote Config [candidate] (651.992 µs) : 0, 652
Telemetry [baseline] (8.377 ms) : 0, 8377
Telemetry [candidate] (8.189 ms) : 0, 8189
Profiling [baseline] (104.267 ms) : 0, 104267
Profiling [candidate] (102.444 ms) : 0, 102444
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~2be1b73b0e, baseline=1.50.0-SNAPSHOT~7c80dbede3

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.02 s) : 0, 1020217
Total [baseline] (8.638 s) : 0, 8638146
Agent [candidate] (1.036 s) : 0, 1035833
Total [candidate] (8.723 s) : 0, 8722809
section iast
Agent [baseline] (1.149 s) : 0, 1149145
Total [baseline] (9.226 s) : 0, 9225662
Agent [candidate] (1.158 s) : 0, 1157802
Total [candidate] (9.302 s) : 0, 9302361
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.157 s) : 0, 1157019
Total [baseline] (9.191 s) : 0, 9191405
Agent [candidate] (1.157 s) : 0, 1156567
Total [candidate] (9.212 s) : 0, 9212027
section iast_TELEMETRY_OFF
Agent [baseline] (1.152 s) : 0, 1151775
Total [baseline] (9.224 s) : 0, 9224109
Agent [candidate] (1.153 s) : 0, 1152911
Total [candidate] (9.261 s) : 0, 9260899
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.02 s -
Agent iast 1.149 s 128.928 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.157 s 136.802 ms (13.4%)
Agent iast_TELEMETRY_OFF 1.152 s 131.558 ms (12.9%)
Total tracing 8.638 s -
Total iast 9.226 s 587.516 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.191 s 553.259 ms (6.4%)
Total iast_TELEMETRY_OFF 9.224 s 585.963 ms (6.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent iast 1.158 s 121.968 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.157 s 120.734 ms (11.7%)
Agent iast_TELEMETRY_OFF 1.153 s 117.077 ms (11.3%)
Total tracing 8.723 s -
Total iast 9.302 s 579.552 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.212 s 489.218 ms (5.6%)
Total iast_TELEMETRY_OFF 9.261 s 538.091 ms (6.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~2be1b73b0e, baseline=1.50.0-SNAPSHOT~7c80dbede3

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (681.669 ms) : 0, 681669
BytebuddyAgent [candidate] (691.862 ms) : 0, 691862
GlobalTracer [baseline] (240.219 ms) : 0, 240219
GlobalTracer [candidate] (243.25 ms) : 0, 243250
AppSec [baseline] (54.258 ms) : 0, 54258
AppSec [candidate] (54.985 ms) : 0, 54985
Debugger [baseline] (9.963 ms) : 0, 9963
Debugger [candidate] (10.052 ms) : 0, 10052
Remote Config [baseline] (712.058 µs) : 0, 712
Remote Config [candidate] (699.611 µs) : 0, 700
Telemetry [baseline] (9.859 ms) : 0, 9859
Telemetry [candidate] (11.262 ms) : 0, 11262
section iast
BytebuddyAgent [baseline] (801.819 ms) : 0, 801819
BytebuddyAgent [candidate] (807.146 ms) : 0, 807146
GlobalTracer [baseline] (230.499 ms) : 0, 230499
GlobalTracer [candidate] (232.24 ms) : 0, 232240
IAST [baseline] (28.435 ms) : 0, 28435
IAST [candidate] (26.346 ms) : 0, 26346
AppSec [baseline] (50.448 ms) : 0, 50448
AppSec [candidate] (52.816 ms) : 0, 52816
Debugger [baseline] (5.944 ms) : 0, 5944
Debugger [candidate] (6.046 ms) : 0, 6046
Remote Config [baseline] (597.529 µs) : 0, 598
Remote Config [candidate] (628.484 µs) : 0, 628
Telemetry [baseline] (7.937 ms) : 0, 7937
Telemetry [candidate] (8.063 ms) : 0, 8063
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (807.591 ms) : 0, 807591
BytebuddyAgent [candidate] (807.306 ms) : 0, 807306
GlobalTracer [baseline] (232.303 ms) : 0, 232303
GlobalTracer [candidate] (232.007 ms) : 0, 232007
IAST [baseline] (29.405 ms) : 0, 29405
IAST [candidate] (29.363 ms) : 0, 29363
AppSec [baseline] (49.676 ms) : 0, 49676
AppSec [candidate] (49.752 ms) : 0, 49752
Debugger [baseline] (5.913 ms) : 0, 5913
Debugger [candidate] (5.955 ms) : 0, 5955
Remote Config [baseline] (609.899 µs) : 0, 610
Remote Config [candidate] (610.272 µs) : 0, 610
Telemetry [baseline] (7.923 ms) : 0, 7923
Telemetry [candidate] (8.006 ms) : 0, 8006
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (803.639 ms) : 0, 803639
BytebuddyAgent [candidate] (804.336 ms) : 0, 804336
GlobalTracer [baseline] (231.484 ms) : 0, 231484
GlobalTracer [candidate] (231.716 ms) : 0, 231716
IAST [baseline] (24.785 ms) : 0, 24785
IAST [candidate] (23.309 ms) : 0, 23309
AppSec [baseline] (53.821 ms) : 0, 53821
AppSec [candidate] (55.423 ms) : 0, 55423
Debugger [baseline] (5.978 ms) : 0, 5978
Debugger [candidate] (6.046 ms) : 0, 6046
Remote Config [baseline] (610.978 µs) : 0, 611
Remote Config [candidate] (608.4 µs) : 0, 608
Telemetry [baseline] (7.849 ms) : 0, 7849
Telemetry [candidate] (7.839 ms) : 0, 7839
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-05-14T18:33:22 2025-05-14T18:41:09
git_branch master gary/llmobs-sdk-merge
git_commit_date 1747229243 1747246977
git_commit_sha 7c80dbe 2be1b73
release_version 1.50.0-SNAPSHOT~7c80dbede3 1.50.0-SNAPSHOT~2be1b73b0e
start_time 2025-05-14T18:33:08 2025-05-14T18:40:55
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1747248465 1747248465
ci_job_id 938665299 938665299
ci_pipeline_id 65116884 65116884
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-px6-mzwm-project-304-concurrent-0-j2v3kz2g 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-px6-mzwm-project-304-concurrent-0-j2v3kz2g 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~2be1b73b0e, baseline=1.50.0-SNAPSHOT~7c80dbede3
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.975 µs) : 359, 399
.   : milestone, 379,
iast (518.568 µs) : 497, 541
.   : milestone, 519,
iast_FULL (738.358 µs) : 716, 760
.   : milestone, 738,
iast_GLOBAL (565.946 µs) : 544, 588
.   : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (525.633 µs) : 503, 549
.   : milestone, 526,
iast_INACTIVE (471.497 µs) : 448, 495
.   : milestone, 471,
iast_TELEMETRY_OFF (521.444 µs) : 498, 545
.   : milestone, 521,
tracing (459.971 µs) : 439, 481
.   : milestone, 460,
section candidate
no_agent (382.032 µs) : 362, 402
.   : milestone, 382,
iast (514.2 µs) : 492, 536
.   : milestone, 514,
iast_FULL (732.557 µs) : 711, 754
.   : milestone, 733,
iast_GLOBAL (554.738 µs) : 533, 576
.   : milestone, 555,
iast_HARDCODED_SECRET_DISABLED (518.485 µs) : 495, 542
.   : milestone, 518,
iast_INACTIVE (461.743 µs) : 441, 483
.   : milestone, 462,
iast_TELEMETRY_OFF (507.823 µs) : 484, 531
.   : milestone, 508,
tracing (456.824 µs) : 435, 479
.   : milestone, 457,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.975 µs [359.364 µs, 398.586 µs] -
iast 518.568 µs [496.541 µs, 540.596 µs] 139.594 µs (36.8%)
iast_FULL 738.358 µs [716.344 µs, 760.373 µs] 359.383 µs (94.8%)
iast_GLOBAL 565.946 µs [543.902 µs, 587.989 µs] 186.971 µs (49.3%)
iast_HARDCODED_SECRET_DISABLED 525.633 µs [502.564 µs, 548.702 µs] 146.658 µs (38.7%)
iast_INACTIVE 471.497 µs [448.413 µs, 494.581 µs] 92.523 µs (24.4%)
iast_TELEMETRY_OFF 521.444 µs [497.977 µs, 544.912 µs] 142.47 µs (37.6%)
tracing 459.971 µs [439.214 µs, 480.728 µs] 80.996 µs (21.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 382.032 µs [361.958 µs, 402.106 µs] -
iast 514.2 µs [492.3 µs, 536.101 µs] 132.168 µs (34.6%)
iast_FULL 732.557 µs [710.839 µs, 754.274 µs] 350.524 µs (91.8%)
iast_GLOBAL 554.738 µs [533.196 µs, 576.28 µs] 172.706 µs (45.2%)
iast_HARDCODED_SECRET_DISABLED 518.485 µs [495.242 µs, 541.727 µs] 136.452 µs (35.7%)
iast_INACTIVE 461.743 µs [440.563 µs, 482.923 µs] 79.711 µs (20.9%)
iast_TELEMETRY_OFF 507.823 µs [484.32 µs, 531.326 µs] 125.791 µs (32.9%)
tracing 456.824 µs [434.986 µs, 478.662 µs] 74.792 µs (19.6%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~2be1b73b0e, baseline=1.50.0-SNAPSHOT~7c80dbede3
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.369 ms) : 1350, 1389
.   : milestone, 1369,
appsec (1.746 ms) : 1722, 1770
.   : milestone, 1746,
appsec_no_iast (1.745 ms) : 1721, 1768
.   : milestone, 1745,
code_origins (1.678 ms) : 1651, 1704
.   : milestone, 1678,
iast (1.515 ms) : 1491, 1539
.   : milestone, 1515,
profiling (1.553 ms) : 1528, 1578
.   : milestone, 1553,
tracing (1.498 ms) : 1474, 1523
.   : milestone, 1498,
section candidate
no_agent (1.354 ms) : 1335, 1374
.   : milestone, 1354,
appsec (1.749 ms) : 1725, 1773
.   : milestone, 1749,
appsec_no_iast (1.732 ms) : 1709, 1756
.   : milestone, 1732,
code_origins (1.659 ms) : 1632, 1686
.   : milestone, 1659,
iast (1.522 ms) : 1498, 1546
.   : milestone, 1522,
profiling (1.516 ms) : 1493, 1540
.   : milestone, 1516,
tracing (1.496 ms) : 1472, 1521
.   : milestone, 1496,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.369 ms [1.35 ms, 1.389 ms] -
appsec 1.746 ms [1.722 ms, 1.77 ms] 376.937 µs (27.5%)
appsec_no_iast 1.745 ms [1.721 ms, 1.768 ms] 375.372 µs (27.4%)
code_origins 1.678 ms [1.651 ms, 1.704 ms] 308.185 µs (22.5%)
iast 1.515 ms [1.491 ms, 1.539 ms] 145.548 µs (10.6%)
profiling 1.553 ms [1.528 ms, 1.578 ms] 183.255 µs (13.4%)
tracing 1.498 ms [1.474 ms, 1.523 ms] 128.859 µs (9.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.354 ms [1.335 ms, 1.374 ms] -
appsec 1.749 ms [1.725 ms, 1.773 ms] 394.365 µs (29.1%)
appsec_no_iast 1.732 ms [1.709 ms, 1.756 ms] 378.124 µs (27.9%)
code_origins 1.659 ms [1.632 ms, 1.686 ms] 305.027 µs (22.5%)
iast 1.522 ms [1.498 ms, 1.546 ms] 167.8 µs (12.4%)
profiling 1.516 ms [1.493 ms, 1.54 ms] 161.93 µs (12.0%)
tracing 1.496 ms [1.472 ms, 1.521 ms] 142.081 µs (10.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master gary/llmobs-sdk-merge
git_commit_date 1747229243 1747246977
git_commit_sha 7c80dbe 2be1b73
release_version 1.50.0-SNAPSHOT~7c80dbede3 1.50.0-SNAPSHOT~2be1b73b0e
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1747248964 1747248964
ci_job_id 938665300 938665300
ci_pipeline_id 65116884 65116884
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-xzg4hwwu-project-304-concurrent-2-v42r68kg 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-xzg4hwwu-project-304-concurrent-2-v42r68kg 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~2be1b73b0e, baseline=1.50.0-SNAPSHOT~7c80dbede3
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.639 s) : 15639000, 15639000
.   : milestone, 15639000,
appsec (15.073 s) : 15073000, 15073000
.   : milestone, 15073000,
iast (18.581 s) : 18581000, 18581000
.   : milestone, 18581000,
iast_GLOBAL (18.022 s) : 18022000, 18022000
.   : milestone, 18022000,
profiling (15.177 s) : 15177000, 15177000
.   : milestone, 15177000,
tracing (15.08 s) : 15080000, 15080000
.   : milestone, 15080000,
section candidate
no_agent (15.438 s) : 15438000, 15438000
.   : milestone, 15438000,
appsec (15.042 s) : 15042000, 15042000
.   : milestone, 15042000,
iast (18.523 s) : 18523000, 18523000
.   : milestone, 18523000,
iast_GLOBAL (17.73 s) : 17730000, 17730000
.   : milestone, 17730000,
profiling (15.151 s) : 15151000, 15151000
.   : milestone, 15151000,
tracing (14.724 s) : 14724000, 14724000
.   : milestone, 14724000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.639 s [15.639 s, 15.639 s] -
appsec 15.073 s [15.073 s, 15.073 s] -566.0 ms (-3.6%)
iast 18.581 s [18.581 s, 18.581 s] 2.942 s (18.8%)
iast_GLOBAL 18.022 s [18.022 s, 18.022 s] 2.383 s (15.2%)
profiling 15.177 s [15.177 s, 15.177 s] -462.0 ms (-3.0%)
tracing 15.08 s [15.08 s, 15.08 s] -559.0 ms (-3.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.438 s [15.438 s, 15.438 s] -
appsec 15.042 s [15.042 s, 15.042 s] -396.0 ms (-2.6%)
iast 18.523 s [18.523 s, 18.523 s] 3.085 s (20.0%)
iast_GLOBAL 17.73 s [17.73 s, 17.73 s] 2.292 s (14.8%)
profiling 15.151 s [15.151 s, 15.151 s] -287.0 ms (-1.9%)
tracing 14.724 s [14.724 s, 14.724 s] -714.0 ms (-4.6%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~2be1b73b0e, baseline=1.50.0-SNAPSHOT~7c80dbede3
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (2.422 ms) : 2372, 2472
.   : milestone, 2422,
iast (2.201 ms) : 2139, 2264
.   : milestone, 2201,
iast_GLOBAL (2.248 ms) : 2185, 2312
.   : milestone, 2248,
profiling (2.067 ms) : 2015, 2118
.   : milestone, 2067,
tracing (2.023 ms) : 1974, 2071
.   : milestone, 2023,
section candidate
no_agent (1.481 ms) : 1470, 1493
.   : milestone, 1481,
appsec (2.433 ms) : 2383, 2484
.   : milestone, 2433,
iast (2.22 ms) : 2157, 2283
.   : milestone, 2220,
iast_GLOBAL (2.246 ms) : 2182, 2309
.   : milestone, 2246,
profiling (2.076 ms) : 2024, 2128
.   : milestone, 2076,
tracing (2.026 ms) : 1977, 2074
.   : milestone, 2026,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.465 ms, 1.488 ms] -
appsec 2.422 ms [2.372 ms, 2.472 ms] 945.501 µs (64.0%)
iast 2.201 ms [2.139 ms, 2.264 ms] 724.914 µs (49.1%)
iast_GLOBAL 2.248 ms [2.185 ms, 2.312 ms] 771.878 µs (52.3%)
profiling 2.067 ms [2.015 ms, 2.118 ms] 589.973 µs (40.0%)
tracing 2.023 ms [1.974 ms, 2.071 ms] 546.166 µs (37.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.47 ms, 1.493 ms] -
appsec 2.433 ms [2.383 ms, 2.484 ms] 952.183 µs (64.3%)
iast 2.22 ms [2.157 ms, 2.283 ms] 738.777 µs (49.9%)
iast_GLOBAL 2.246 ms [2.182 ms, 2.309 ms] 764.371 µs (51.6%)
profiling 2.076 ms [2.024 ms, 2.128 ms] 595.152 µs (40.2%)
tracing 2.026 ms [1.977 ms, 2.074 ms] 544.464 µs (36.8%)

@gary-huang
add APIs for llm obs sdk (#8135)
2be1b73 
* add APIs for llm obs

* add llm message class to support llm spans

* follow java convention of naming Id instead of ID

* add codeowners
@gary-huang gary-huang force-pushed the gary/llmobs-sdk-merge branch from 896f40a to 2be1b73 Compare May 14, 2025 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@gary-huang