Skip to content

🍒 8885, 8952 - Optimize IAST Vulnerability Detection... #9241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

🍒 8885, 8952 - Optimize IAST Vulnerability Detection... #9241

wants to merge 2 commits into from

Conversation

sarahchen6
Copy link
Contributor

@sarahchen6 sarahchen6 commented Jul 24, 2025 •
edited
Loading

Backports #8885 and #8952 to release/v1.50.x

@jandro996 @sarahchen6
Optimize IAST Vulnerability Detection (#8885)
972fecd 
What Does This Do
Implements the new algorithm for detecting IAST vulnerabilities, where vulnerabilities that were already explored in previous runs for a given endpoint are skipped, ensuring that all remaining ones are eventually explored.

This addresses the current limitation where only the first matching vulnerabilities are consistently reported, causing others to remain hidden.

Changes to OverheadContext
The OverheadContext class has been extended to support three separate tracking maps:

globalMap
Used to track vulnerability detection counts per endpoint across all requests.
Keys are strings combining the request method and route (GET /login, POST /submit, etc.).
Values are maps from vulnerabilityType → int (count of occurrences).
Capped at 4,096 entries using a clear‐on‐overflow strategy, to ensure bounded memory usage.
Oldest entries are cleared once the limit is reached.
copyMap
Created per request to copy the global counts at the start of the request, ensuring a consistent baseline to compare against throughout the lifecycle of the request.
requestMap
Tracks vulnerability type counts within the request.
An additional field, isGlobal, has been added to indicate whether the context is global or request-scoped. If isGlobal is true, the maps are not used, and quota checks proceed using the global strategy only.

A new method, resetMaps(), has been added to update globalMap when the request ends and vulnerability data has been reported. Two scenarios are supported:

Case 1: Budget not fully used → The entry for the endpoint in globalMap is cleared, since the request stayed within budget.
Case 2: Budget fully used → The counts from requestMap are compared to those in copyMap. For each vulnerability type, if the value in requestMap is greater, it is used to update the corresponding entry in globalMap.
Changes to OverheadController
The method consumeQuota() has been extended to receive a vulnerabilityType and modified to support the new logic:

If an OverheadContext is present and not global, and there is remaining quota and a valid span, the controller now invokes a new method maybeSkipVulnerability() to determine whether quota should actually be consumed or not, based on endpoint-specific history.

It's better to check the Algorithm execution example flow diagram to understand how this should work

Changes to IastRequestContext
In releaseRequestContext(), the request now calls resetMaps() on the associated OverheadContext, ensuring globalMap is updated at the end of each request.

Motivation
[RFC-1029] Optimizing IAST Vulnerability Detection implementation

Additional Notes
java tracer needs to implement also [RFC-1029-A1] Solution for dynamic http routes

(cherry picked from commit 99ecab7)
@sarahchen6 sarahchen6 added type: enhancement Enhancements and improvements comp: asm iast Application Security Management (IAST) labels Jul 24, 2025
@sarahchen6 sarahchen6 requested a review from a team as a code owner July 24, 2025 16:02
@sarahchen6 sarahchen6 added type: enhancement Enhancements and improvements comp: asm iast Application Security Management (IAST) labels Jul 24, 2025
@sarahchen6 sarahchen6 added this to the 1.50.2 milestone Jul 24, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jul 24, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarah.chen/backport-pr-8885
git_commit_date 1753378716 1753382428
git_commit_sha b94ca59 c7b1417
release_version 1.51.1-SNAPSHOT~b94ca598bb 1.51.0-SNAPSHOT~c7b14171e2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753384910 1753384910
ci_job_id 1046693491 1046693491
ci_pipeline_id 71639197 71639197
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-18gk8fel 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-18gk8fel 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 9 performance improvements and 4 performance regressions! Performance is the same for 24 metrics, 16 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:BytebuddyAgent better
[-50.979ms; -23.869ms] or [-5.996%; -2.807%]		 812.854ms 850.278ms
scenario:startup:insecure-bank:tracing:BytebuddyAgent better
[-47.878ms; -42.171ms] or [-6.542%; -5.762%]		 686.873ms 731.897ms
scenario:startup:petclinic:appsec:Agent.start better
[-60.956ms; -27.836ms] or [-4.968%; -2.269%]		 1.183s 1.227s
scenario:startup:petclinic:appsec:BytebuddyAgent better
[-58.112ms; -35.688ms] or [-7.666%; -4.708%]		 711.166ms 758.066ms
scenario:startup:petclinic:appsec:AppSec worse
[+4.397ms; +10.190ms] or [+2.595%; +6.013%]		 176.756ms 169.463ms
scenario:startup:petclinic:appsec:Telemetry better
[-1.331ms; -0.767ms] or [-15.898%; -9.166%]		 7.322ms 8.371ms
scenario:startup:petclinic:iast:BytebuddyAgent better
[-43.791ms; -36.141ms] or [-5.170%; -4.267%]		 806.986ms 846.952ms
scenario:startup:petclinic:profiling:Agent.start worse
[+81.078ms; +89.335ms] or [+6.809%; +7.503%]		 1.276s 1.191s
scenario:startup:petclinic:profiling:BytebuddyAgent better
[-82.032ms; -76.427ms] or [-10.811%; -10.072%]		 679.583ms 758.813ms
scenario:startup:petclinic:profiling:GlobalTracer worse
[+141.314ms; +143.289ms] or [+63.969%; +64.863%]		 363.212ms 220.911ms
scenario:startup:petclinic:profiling:AppSec worse
[+31.469ms; +32.200ms] or [+104.417%; +106.845%]		 61.972ms 30.137ms
scenario:startup:petclinic:profiling:Telemetry better
[-7.760ms; -7.222ms] or [-49.209%; -45.797%]		 8.279ms 15.770ms
scenario:startup:petclinic:tracing:BytebuddyAgent better
[-47.563ms; -27.198ms] or [-6.503%; -3.719%]		 694.038ms 731.419ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~c7b14171e2, baseline=1.51.1-SNAPSHOT~b94ca598bb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1045149
Total [baseline] (10.718 s) : 0, 10718219
Agent [candidate] (1.036 s) : 0, 1035976
Total [candidate] (10.738 s) : 0, 10737617
section appsec
Agent [baseline] (1.227 s) : 0, 1227069
Total [baseline] (10.835 s) : 0, 10835231
Agent [candidate] (1.183 s) : 0, 1182673
Total [candidate] (10.822 s) : 0, 10821662
section iast
Agent [baseline] (1.174 s) : 0, 1174200
Total [baseline] (10.871 s) : 0, 10871109
Agent [candidate] (1.156 s) : 0, 1155613
Total [candidate] (10.94 s) : 0, 10940183
section profiling
Agent [baseline] (1.191 s) : 0, 1190659
Total [baseline] (10.812 s) : 0, 10812471
Agent [candidate] (1.276 s) : 0, 1275865
Total [candidate] (11.006 s) : 0, 11006233
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.045 s -
Agent appsec 1.227 s 181.92 ms (17.4%)
Agent iast 1.174 s 129.051 ms (12.3%)
Agent profiling 1.191 s 145.51 ms (13.9%)
Total tracing 10.718 s -
Total appsec 10.835 s 117.012 ms (1.1%)
Total iast 10.871 s 152.89 ms (1.4%)
Total profiling 10.812 s 94.252 ms (0.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent appsec 1.183 s 146.697 ms (14.2%)
Agent iast 1.156 s 119.637 ms (11.5%)
Agent profiling 1.276 s 239.889 ms (23.2%)
Total tracing 10.738 s -
Total appsec 10.822 s 84.045 ms (0.8%)
Total iast 10.94 s 202.567 ms (1.9%)
Total profiling 11.006 s 268.617 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~c7b14171e2, baseline=1.51.1-SNAPSHOT~b94ca598bb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (731.419 ms) : 0, 731419
BytebuddyAgent [candidate] (694.038 ms) : 0, 694038
GlobalTracer [baseline] (241.706 ms) : 0, 241706
GlobalTracer [candidate] (244.687 ms) : 0, 244687
AppSec [baseline] (30.41 ms) : 0, 30410
AppSec [candidate] (57.18 ms) : 0, 57180
Debugger [baseline] (5.978 ms) : 0, 5978
Debugger [candidate] (7.791 ms) : 0, 7791
Remote Config [baseline] (649.344 µs) : 0, 649
Remote Config [candidate] (688.191 µs) : 0, 688
Telemetry [baseline] (12.54 ms) : 0, 12540
Telemetry [candidate] (10.486 ms) : 0, 10486
crashtracking [baseline] (1.43 ms) : 0, 1430
section appsec
BytebuddyAgent [baseline] (758.066 ms) : 0, 758066
BytebuddyAgent [candidate] (711.166 ms) : 0, 711166
GlobalTracer [baseline] (235.976 ms) : 0, 235976
GlobalTracer [candidate] (236.743 ms) : 0, 236743
IAST [baseline] (23.732 ms) : 0, 23732
IAST [candidate] (23.218 ms) : 0, 23218
AppSec [baseline] (169.463 ms) : 0, 169463
AppSec [candidate] (176.756 ms) : 0, 176756
Debugger [baseline] (8.005 ms) : 0, 8005
Debugger [candidate] (5.941 ms) : 0, 5941
Remote Config [baseline] (629.587 µs) : 0, 630
Remote Config [candidate] (615.958 µs) : 0, 616
Telemetry [baseline] (8.371 ms) : 0, 8371
Telemetry [candidate] (7.322 ms) : 0, 7322
crashtracking [baseline] (1.437 ms) : 0, 1437
section iast
BytebuddyAgent [baseline] (846.952 ms) : 0, 846952
BytebuddyAgent [candidate] (806.986 ms) : 0, 806986
GlobalTracer [baseline] (231.946 ms) : 0, 231946
GlobalTracer [candidate] (232.343 ms) : 0, 232343
IAST [baseline] (28.409 ms) : 0, 28409
IAST [candidate] (29.818 ms) : 0, 29818
AppSec [baseline] (27.366 ms) : 0, 27366
AppSec [candidate] (50.807 ms) : 0, 50807
Debugger [baseline] (8.335 ms) : 0, 8335
Debugger [candidate] (5.962 ms) : 0, 5962
Remote Config [baseline] (580.966 µs) : 0, 581
Remote Config [candidate] (637.845 µs) : 0, 638
Telemetry [baseline] (8.121 ms) : 0, 8121
Telemetry [candidate] (8.215 ms) : 0, 8215
crashtracking [baseline] (1.433 ms) : 0, 1433
section profiling
BytebuddyAgent [baseline] (758.813 ms) : 0, 758813
BytebuddyAgent [candidate] (679.583 ms) : 0, 679583
GlobalTracer [baseline] (220.911 ms) : 0, 220911
GlobalTracer [candidate] (363.212 ms) : 0, 363212
AppSec [baseline] (30.137 ms) : 0, 30137
AppSec [candidate] (61.972 ms) : 0, 61972
Debugger [baseline] (6.206 ms) : 0, 6206
Debugger [candidate] (6.181 ms) : 0, 6181
Remote Config [baseline] (697.525 µs) : 0, 698
Remote Config [candidate] (700.582 µs) : 0, 701
Telemetry [baseline] (15.77 ms) : 0, 15770
Telemetry [candidate] (8.279 ms) : 0, 8279
crashtracking [baseline] (1.402 ms) : 0, 1402
ProfilingAgent [baseline] (108.078 ms) : 0, 108078
ProfilingAgent [candidate] (107.178 ms) : 0, 107178
Profiling [baseline] (108.102 ms) : 0, 108102
Profiling [candidate] (107.202 ms) : 0, 107202
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~c7b14171e2, baseline=1.51.1-SNAPSHOT~b94ca598bb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1044475
Total [baseline] (8.613 s) : 0, 8613317
Agent [candidate] (1.026 s) : 0, 1026312
Total [candidate] (8.573 s) : 0, 8573188
section iast
Agent [baseline] (1.179 s) : 0, 1179426
Total [baseline] (9.33 s) : 0, 9330057
Agent [candidate] (1.163 s) : 0, 1163010
Total [candidate] (9.259 s) : 0, 9258658
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.044 s -
Agent iast 1.179 s 134.951 ms (12.9%)
Total tracing 8.613 s -
Total iast 9.33 s 716.74 ms (8.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.026 s -
Agent iast 1.163 s 136.698 ms (13.3%)
Total tracing 8.573 s -
Total iast 9.259 s 685.47 ms (8.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~c7b14171e2, baseline=1.51.1-SNAPSHOT~b94ca598bb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (731.897 ms) : 0, 731897
BytebuddyAgent [candidate] (686.873 ms) : 0, 686873
GlobalTracer [baseline] (242.369 ms) : 0, 242369
GlobalTracer [candidate] (242.937 ms) : 0, 242937
AppSec [baseline] (30.403 ms) : 0, 30403
AppSec [candidate] (58.92 ms) : 0, 58920
Debugger [baseline] (6.011 ms) : 0, 6011
Debugger [candidate] (7.015 ms) : 0, 7015
Remote Config [baseline] (659.049 µs) : 0, 659
Remote Config [candidate] (667.289 µs) : 0, 667
Telemetry [baseline] (10.693 ms) : 0, 10693
Telemetry [candidate] (8.997 ms) : 0, 8997
crashtracking [baseline] (1.441 ms) : 0, 1441
section iast
BytebuddyAgent [baseline] (850.278 ms) : 0, 850278
BytebuddyAgent [candidate] (812.854 ms) : 0, 812854
GlobalTracer [baseline] (233.633 ms) : 0, 233633
GlobalTracer [candidate] (233.315 ms) : 0, 233315
IAST [baseline] (27.638 ms) : 0, 27638
IAST [candidate] (28.557 ms) : 0, 28557
AppSec [baseline] (30.037 ms) : 0, 30037
AppSec [candidate] (52.624 ms) : 0, 52624
Debugger [baseline] (6.632 ms) : 0, 6632
Debugger [candidate] (5.978 ms) : 0, 5978
Remote Config [baseline] (590.85 µs) : 0, 591
Remote Config [candidate] (606.337 µs) : 0, 606
Telemetry [baseline] (8.158 ms) : 0, 8158
Telemetry [candidate] (8.166 ms) : 0, 8166
crashtracking [baseline] (1.428 ms) : 0, 1428
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarah.chen/backport-pr-8885
git_commit_date 1753378716 1753382428
git_commit_sha b94ca59 c7b1417
release_version 1.51.1-SNAPSHOT~b94ca598bb 1.51.0-SNAPSHOT~c7b14171e2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753384611 1753384611
ci_job_id 1046693492 1046693492
ci_pipeline_id 71639197 71639197
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-8ph0odo1 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-8ph0odo1 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 4 performance improvements and 1 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:tracing:high_load worse
[+223.455µs; +462.894µs] or [+3.029%; +6.274%]		 unstable
[-108.934op/s; +53.996op/s] or [-17.363%; +8.607%]		 7.722ms 599.906op/s 7.378ms 627.375op/s
scenario:load:petclinic:appsec:high_load better
[-2.844ms; -1.914ms] or [-5.664%; -3.812%]		 unstable
[-2.641op/s; +11.791op/s] or [-2.832%; +12.643%]		 47.833ms 97.838op/s 50.212ms 93.263op/s
scenario:load:petclinic:iast:high_load better
[-2.131ms; -1.281ms] or [-4.619%; -2.778%]		 unstable
[-4.034op/s; +11.909op/s] or [-3.977%; +11.740%]		 44.422ms 105.375op/s 46.128ms 101.438op/s
scenario:load:petclinic:tracing:high_load better
[-2.099ms; -1.308ms] or [-4.621%; -2.878%]		 unstable
[-3.993op/s; +11.918op/s] or [-3.877%; +11.572%]		 43.733ms 106.950op/s 45.436ms 102.987op/s
scenario:load:petclinic:profiling:high_load better
[-2.604ms; -1.637ms] or [-5.278%; -3.319%]		 unstable
[-3.271op/s; +11.796op/s] or [-3.447%; +12.432%]		 47.203ms 99.150op/s 49.323ms 94.888op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~c7b14171e2, baseline=1.51.1-SNAPSHOT~b94ca598bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.269 ms) : 4220, 4317
.   : milestone, 4269,
iast (9.165 ms) : 9015, 9315
.   : milestone, 9165,
iast_FULL (13.831 ms) : 13559, 14103
.   : milestone, 13831,
iast_GLOBAL (10.365 ms) : 10181, 10548
.   : milestone, 10365,
profiling (8.705 ms) : 8564, 8846
.   : milestone, 8705,
tracing (7.378 ms) : 7274, 7483
.   : milestone, 7378,
section candidate
no_agent (4.279 ms) : 4225, 4333
.   : milestone, 4279,
iast (9.354 ms) : 9201, 9506
.   : milestone, 9354,
iast_FULL (13.558 ms) : 13288, 13828
.   : milestone, 13558,
iast_GLOBAL (10.239 ms) : 10060, 10418
.   : milestone, 10239,
profiling (8.817 ms) : 8682, 8952
.   : milestone, 8817,
tracing (7.722 ms) : 7604, 7839
.   : milestone, 7722,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.269 ms [4.22 ms, 4.317 ms] -
iast 9.165 ms [9.015 ms, 9.315 ms] 4.896 ms (114.7%)
iast_FULL 13.831 ms [13.559 ms, 14.103 ms] 9.562 ms (224.0%)
iast_GLOBAL 10.365 ms [10.181 ms, 10.548 ms] 6.096 ms (142.8%)
profiling 8.705 ms [8.564 ms, 8.846 ms] 4.436 ms (103.9%)
tracing 7.378 ms [7.274 ms, 7.483 ms] 3.11 ms (72.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.279 ms [4.225 ms, 4.333 ms] -
iast 9.354 ms [9.201 ms, 9.506 ms] 5.075 ms (118.6%)
iast_FULL 13.558 ms [13.288 ms, 13.828 ms] 9.279 ms (216.9%)
iast_GLOBAL 10.239 ms [10.06 ms, 10.418 ms] 5.96 ms (139.3%)
profiling 8.817 ms [8.682 ms, 8.952 ms] 4.538 ms (106.1%)
tracing 7.722 ms [7.604 ms, 7.839 ms] 3.443 ms (80.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~c7b14171e2, baseline=1.51.1-SNAPSHOT~b94ca598bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.7 ms) : 37399, 38000
.   : milestone, 37700,
appsec (50.212 ms) : 49763, 50662
.   : milestone, 50212,
code_origins (45.271 ms) : 44875, 45668
.   : milestone, 45271,
iast (46.128 ms) : 45743, 46513
.   : milestone, 46128,
profiling (49.323 ms) : 48845, 49802
.   : milestone, 49323,
tracing (45.436 ms) : 45056, 45817
.   : milestone, 45436,
section candidate
no_agent (37.264 ms) : 36960, 37568
.   : milestone, 37264,
appsec (47.833 ms) : 47419, 48247
.   : milestone, 47833,
code_origins (44.301 ms) : 43934, 44667
.   : milestone, 44301,
iast (44.422 ms) : 44018, 44826
.   : milestone, 44422,
profiling (47.203 ms) : 46785, 47621
.   : milestone, 47203,
tracing (43.733 ms) : 43378, 44087
.   : milestone, 43733,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.7 ms [37.399 ms, 38.0 ms] -
appsec 50.212 ms [49.763 ms, 50.662 ms] 12.512 ms (33.2%)
code_origins 45.271 ms [44.875 ms, 45.668 ms] 7.571 ms (20.1%)
iast 46.128 ms [45.743 ms, 46.513 ms] 8.428 ms (22.4%)
profiling 49.323 ms [48.845 ms, 49.802 ms] 11.623 ms (30.8%)
tracing 45.436 ms [45.056 ms, 45.817 ms] 7.736 ms (20.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.264 ms [36.96 ms, 37.568 ms] -
appsec 47.833 ms [47.419 ms, 48.247 ms] 10.569 ms (28.4%)
code_origins 44.301 ms [43.934 ms, 44.667 ms] 7.037 ms (18.9%)
iast 44.422 ms [44.018 ms, 44.826 ms] 7.158 ms (19.2%)
profiling 47.203 ms [46.785 ms, 47.621 ms] 9.939 ms (26.7%)
tracing 43.733 ms [43.378 ms, 44.087 ms] 6.469 ms (17.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarah.chen/backport-pr-8885
git_commit_date 1753378716 1753382428
git_commit_sha b94ca59 c7b1417
release_version 1.51.1-SNAPSHOT~b94ca598bb 1.51.0-SNAPSHOT~c7b14171e2
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1753385241 1753385241
ci_job_id 1046693493 1046693493
ci_pipeline_id 71639197 71639197
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-g4d0pzqx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-g4d0pzqx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.372ms; -1.038ms] or [-37.693%; -28.522%]		 2.435ms 3.640ms
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~c7b14171e2, baseline=1.51.1-SNAPSHOT~b94ca598bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (3.64 ms) : 3426, 3853
.   : milestone, 3640,
iast (2.228 ms) : 2165, 2292
.   : milestone, 2228,
iast_GLOBAL (2.276 ms) : 2211, 2340
.   : milestone, 2276,
profiling (2.082 ms) : 2029, 2135
.   : milestone, 2082,
tracing (2.039 ms) : 1990, 2088
.   : milestone, 2039,
section candidate
no_agent (1.487 ms) : 1476, 1499
.   : milestone, 1487,
appsec (2.435 ms) : 2385, 2485
.   : milestone, 2435,
iast (2.218 ms) : 2155, 2280
.   : milestone, 2218,
iast_GLOBAL (2.256 ms) : 2193, 2319
.   : milestone, 2256,
profiling (2.053 ms) : 2003, 2104
.   : milestone, 2053,
tracing (2.03 ms) : 1982, 2078
.   : milestone, 2030,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.478 ms [1.466 ms, 1.489 ms] -
appsec 3.64 ms [3.426 ms, 3.853 ms] 2.162 ms (146.3%)
iast 2.228 ms [2.165 ms, 2.292 ms] 750.436 µs (50.8%)
iast_GLOBAL 2.276 ms [2.211 ms, 2.34 ms] 797.644 µs (54.0%)
profiling 2.082 ms [2.029 ms, 2.135 ms] 604.204 µs (40.9%)
tracing 2.039 ms [1.99 ms, 2.088 ms] 560.977 µs (38.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.487 ms [1.476 ms, 1.499 ms] -
appsec 2.435 ms [2.385 ms, 2.485 ms] 947.387 µs (63.7%)
iast 2.218 ms [2.155 ms, 2.28 ms] 730.213 µs (49.1%)
iast_GLOBAL 2.256 ms [2.193 ms, 2.319 ms] 769.02 µs (51.7%)
profiling 2.053 ms [2.003 ms, 2.104 ms] 566.01 µs (38.1%)
tracing 2.03 ms [1.982 ms, 2.078 ms] 542.665 µs (36.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~c7b14171e2, baseline=1.51.1-SNAPSHOT~b94ca598bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.003 s) : 15003000, 15003000
.   : milestone, 15003000,
appsec (14.854 s) : 14854000, 14854000
.   : milestone, 14854000,
iast (18.864 s) : 18864000, 18864000
.   : milestone, 18864000,
iast_GLOBAL (18.183 s) : 18183000, 18183000
.   : milestone, 18183000,
profiling (15.319 s) : 15319000, 15319000
.   : milestone, 15319000,
tracing (14.81 s) : 14810000, 14810000
.   : milestone, 14810000,
section candidate
no_agent (15.274 s) : 15274000, 15274000
.   : milestone, 15274000,
appsec (14.92 s) : 14920000, 14920000
.   : milestone, 14920000,
iast (19.13 s) : 19130000, 19130000
.   : milestone, 19130000,
iast_GLOBAL (17.846 s) : 17846000, 17846000
.   : milestone, 17846000,
profiling (15.815 s) : 15815000, 15815000
.   : milestone, 15815000,
tracing (14.884 s) : 14884000, 14884000
.   : milestone, 14884000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.003 s [15.003 s, 15.003 s] -
appsec 14.854 s [14.854 s, 14.854 s] -149.0 ms (-1.0%)
iast 18.864 s [18.864 s, 18.864 s] 3.861 s (25.7%)
iast_GLOBAL 18.183 s [18.183 s, 18.183 s] 3.18 s (21.2%)
profiling 15.319 s [15.319 s, 15.319 s] 316.0 ms (2.1%)
tracing 14.81 s [14.81 s, 14.81 s] -193.0 ms (-1.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.274 s [15.274 s, 15.274 s] -
appsec 14.92 s [14.92 s, 14.92 s] -354.0 ms (-2.3%)
iast 19.13 s [19.13 s, 19.13 s] 3.856 s (25.2%)
iast_GLOBAL 17.846 s [17.846 s, 17.846 s] 2.572 s (16.8%)
profiling 15.815 s [15.815 s, 15.815 s] 541.0 ms (3.5%)
tracing 14.884 s [14.884 s, 14.884 s] -390.0 ms (-2.6%)

@sarahchen6 sarahchen6 requested review from a team as code owners July 24, 2025 18:40
@sarahchen6 sarahchen6 requested a review from PerfectSlayer July 24, 2025 18:40
@sarahchen6 sarahchen6 changed the title 🍒 8885 - Optimize IAST Vulnerability Detection 🍒 8885, 8952 - Optimize IAST Vulnerability Detection... Jul 24, 2025
@sarahchen6
Copy link
Contributor Author

We no longer need to proceed with the 1.50.2 release.

@sarahchen6 sarahchen6 closed this Jul 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@manuel-alvarez-alvarez manuel-alvarez-alvarez manuel-alvarez-alvarez approved these changes

@robertpi robertpi Awaiting requested review from robertpi robertpi is a code owner automatically assigned from DataDog/asm-java

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer PerfectSlayer is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: asm iast Application Security Management (IAST) type: enhancement Enhancements and improvements
Projects
None yet
Milestone
1.50.2
Development

Successfully merging this pull request may close these issues.
3 participants
@sarahchen6 @manuel-alvarez-alvarez @jandro996