Simplify context propagation #8719
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PerfectSlayer
added
type: enhancement
tag: do not merge
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 16 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~30b0291160, baseline=1.50.0-SNAPSHOT~0903cf408b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.021 s) : 0, 1020933
Total [baseline] (10.446 s) : 0, 10446353
Agent [candidate] (1.018 s) : 0, 1018136
Total [candidate] (10.461 s) : 0, 10461041
section appsec
Agent [baseline] (1.164 s) : 0, 1163500
Total [baseline] (10.665 s) : 0, 10665048
Agent [candidate] (1.16 s) : 0, 1160343
Total [candidate] (10.695 s) : 0, 10695480
section iast
Agent [baseline] (1.151 s) : 0, 1150608
Total [baseline] (10.908 s) : 0, 10908148
Agent [candidate] (1.16 s) : 0, 1159663
Total [candidate] (10.926 s) : 0, 10925822
section profiling
Agent [baseline] (1.29 s) : 0, 1289542
Total [baseline] (10.944 s) : 0, 10944249
Agent [candidate] (1.278 s) : 0, 1277827
Total [candidate] (10.823 s) : 0, 10822741
gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~30b0291160, baseline=1.50.0-SNAPSHOT~0903cf408b
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.721 ms) : 0, 684721
BytebuddyAgent [candidate] (682.752 ms) : 0, 682752
GlobalTracer [baseline] (241.001 ms) : 0, 241001
GlobalTracer [candidate] (240.184 ms) : 0, 240184
AppSec [baseline] (54.58 ms) : 0, 54580
AppSec [candidate] (54.176 ms) : 0, 54176
Debugger [baseline] (6.977 ms) : 0, 6977
Debugger [candidate] (7.601 ms) : 0, 7601
Remote Config [baseline] (714.529 µs) : 0, 715
Remote Config [candidate] (708.079 µs) : 0, 708
Telemetry [baseline] (9.133 ms) : 0, 9133
Telemetry [candidate] (9.068 ms) : 0, 9068
section appsec
BytebuddyAgent [baseline] (702.03 ms) : 0, 702030
BytebuddyAgent [candidate] (699.998 ms) : 0, 699998
GlobalTracer [baseline] (237.077 ms) : 0, 237077
GlobalTracer [candidate] (236.571 ms) : 0, 236571
AppSec [baseline] (175.845 ms) : 0, 175845
AppSec [candidate] (175.062 ms) : 0, 175062
Debugger [baseline] (5.933 ms) : 0, 5933
Debugger [candidate] (5.958 ms) : 0, 5958
Remote Config [baseline] (626.461 µs) : 0, 626
Remote Config [candidate] (624.096 µs) : 0, 624
Telemetry [baseline] (7.41 ms) : 0, 7410
Telemetry [candidate] (7.777 ms) : 0, 7777
IAST [baseline] (21.853 ms) : 0, 21853
IAST [candidate] (21.542 ms) : 0, 21542
section iast
BytebuddyAgent [baseline] (803.697 ms) : 0, 803697
BytebuddyAgent [candidate] (809.813 ms) : 0, 809813
GlobalTracer [baseline] (230.546 ms) : 0, 230546
GlobalTracer [candidate] (232.421 ms) : 0, 232421
AppSec [baseline] (51.954 ms) : 0, 51954
AppSec [candidate] (52.369 ms) : 0, 52369
Debugger [baseline] (5.945 ms) : 0, 5945
Debugger [candidate] (5.86 ms) : 0, 5860
Remote Config [baseline] (593.724 µs) : 0, 594
Remote Config [candidate] (610.468 µs) : 0, 610
Telemetry [baseline] (7.809 ms) : 0, 7809
Telemetry [candidate] (7.859 ms) : 0, 7859
IAST [baseline] (25.784 ms) : 0, 25784
IAST [candidate] (26.918 ms) : 0, 26918
section profiling
BytebuddyAgent [baseline] (678.708 ms) : 0, 678708
BytebuddyAgent [candidate] (673.142 ms) : 0, 673142
GlobalTracer [baseline] (376.558 ms) : 0, 376558
GlobalTracer [candidate] (374.055 ms) : 0, 374055
AppSec [baseline] (62.646 ms) : 0, 62646
AppSec [candidate] (62.293 ms) : 0, 62293
Debugger [baseline] (6.409 ms) : 0, 6409
Debugger [candidate] (6.3 ms) : 0, 6300
Remote Config [baseline] (672.905 µs) : 0, 673
Remote Config [candidate] (675.403 µs) : 0, 675
Telemetry [baseline] (8.412 ms) : 0, 8412
Telemetry [candidate] (8.137 ms) : 0, 8137
ProfilingAgent [baseline] (104.962 ms) : 0, 104962
ProfilingAgent [candidate] (102.501 ms) : 0, 102501
Profiling [baseline] (104.987 ms) : 0, 104987
Profiling [candidate] (102.524 ms) : 0, 102524
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~30b0291160, baseline=1.50.0-SNAPSHOT~0903cf408b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.022 s) : 0, 1022273
Total [baseline] (8.661 s) : 0, 8660790
Agent [candidate] (1.022 s) : 0, 1022117
Total [candidate] (8.67 s) : 0, 8669918
section iast
Agent [baseline] (1.155 s) : 0, 1154521
Total [baseline] (9.204 s) : 0, 9203917
Agent [candidate] (1.15 s) : 0, 1150302
Total [candidate] (9.218 s) : 0, 9218049
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.157 s) : 0, 1156868
Total [baseline] (9.197 s) : 0, 9196506
Agent [candidate] (1.165 s) : 0, 1165016
Total [candidate] (9.237 s) : 0, 9237011
section iast_TELEMETRY_OFF
Agent [baseline] (1.145 s) : 0, 1145258
Total [baseline] (9.256 s) : 0, 9255951
Agent [candidate] (1.15 s) : 0, 1149746
Total [candidate] (9.254 s) : 0, 9254485
gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~30b0291160, baseline=1.50.0-SNAPSHOT~0903cf408b
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (683.981 ms) : 0, 683981
BytebuddyAgent [candidate] (682.385 ms) : 0, 682385
GlobalTracer [baseline] (240.549 ms) : 0, 240549
GlobalTracer [candidate] (239.702 ms) : 0, 239702
AppSec [baseline] (55.975 ms) : 0, 55975
AppSec [candidate] (55.338 ms) : 0, 55338
Debugger [baseline] (7.55 ms) : 0, 7550
Debugger [candidate] (8.331 ms) : 0, 8331
Remote Config [baseline] (706.451 µs) : 0, 706
Remote Config [candidate] (710.431 µs) : 0, 710
Telemetry [baseline] (9.848 ms) : 0, 9848
Telemetry [candidate] (12.046 ms) : 0, 12046
section iast
BytebuddyAgent [baseline] (806.643 ms) : 0, 806643
BytebuddyAgent [candidate] (802.536 ms) : 0, 802536
GlobalTracer [baseline] (231.112 ms) : 0, 231112
GlobalTracer [candidate] (230.841 ms) : 0, 230841
IAST [baseline] (28.483 ms) : 0, 28483
IAST [candidate] (27.581 ms) : 0, 27581
AppSec [baseline] (50.272 ms) : 0, 50272
AppSec [candidate] (51.367 ms) : 0, 51367
Debugger [baseline] (5.933 ms) : 0, 5933
Debugger [candidate] (5.921 ms) : 0, 5921
Remote Config [baseline] (613.742 µs) : 0, 614
Remote Config [candidate] (598.898 µs) : 0, 599
Telemetry [baseline] (7.906 ms) : 0, 7906
Telemetry [candidate] (7.924 ms) : 0, 7924
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (808.252 ms) : 0, 808252
BytebuddyAgent [candidate] (813.794 ms) : 0, 813794
GlobalTracer [baseline] (231.539 ms) : 0, 231539
GlobalTracer [candidate] (233.399 ms) : 0, 233399
IAST [baseline] (28.582 ms) : 0, 28582
IAST [candidate] (29.594 ms) : 0, 29594
AppSec [baseline] (50.436 ms) : 0, 50436
AppSec [candidate] (49.846 ms) : 0, 49846
Debugger [baseline] (5.884 ms) : 0, 5884
Debugger [candidate] (5.952 ms) : 0, 5952
Remote Config [baseline] (602.449 µs) : 0, 602
Remote Config [candidate] (606.282 µs) : 0, 606
Telemetry [baseline] (7.957 ms) : 0, 7957
Telemetry [candidate] (8.036 ms) : 0, 8036
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (799.203 ms) : 0, 799203
BytebuddyAgent [candidate] (800.607 ms) : 0, 800607
GlobalTracer [baseline] (230.577 ms) : 0, 230577
GlobalTracer [candidate] (232.283 ms) : 0, 232283
IAST [baseline] (22.987 ms) : 0, 22987
IAST [candidate] (23.378 ms) : 0, 23378
AppSec [baseline] (54.851 ms) : 0, 54851
AppSec [candidate] (54.648 ms) : 0, 54648
Debugger [baseline] (5.885 ms) : 0, 5885
Debugger [candidate] (5.982 ms) : 0, 5982
Remote Config [baseline] (592.171 µs) : 0, 592
Remote Config [candidate] (626.066 µs) : 0, 626
Telemetry [baseline] (7.685 ms) : 0, 7685
Telemetry [candidate] (7.794 ms) : 0, 7794
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~30b0291160, baseline=1.50.0-SNAPSHOT~0903cf408b
dateFormat X
axisFormat %s
section baseline
no_agent (1.356 ms) : 1336, 1375
. : milestone, 1356,
appsec (1.726 ms) : 1703, 1750
. : milestone, 1726,
appsec_no_iast (1.745 ms) : 1721, 1768
. : milestone, 1745,
code_origins (1.673 ms) : 1645, 1700
. : milestone, 1673,
iast (1.52 ms) : 1496, 1544
. : milestone, 1520,
profiling (1.574 ms) : 1549, 1599
. : milestone, 1574,
tracing (1.491 ms) : 1466, 1516
. : milestone, 1491,
section candidate
no_agent (1.361 ms) : 1341, 1380
. : milestone, 1361,
appsec (1.712 ms) : 1689, 1736
. : milestone, 1712,
appsec_no_iast (1.726 ms) : 1703, 1750
. : milestone, 1726,
code_origins (1.649 ms) : 1622, 1676
. : milestone, 1649,
iast (1.509 ms) : 1484, 1533
. : milestone, 1509,
profiling (1.534 ms) : 1509, 1559
. : milestone, 1534,
tracing (1.496 ms) : 1472, 1521
. : milestone, 1496,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~30b0291160, baseline=1.50.0-SNAPSHOT~0903cf408b
dateFormat X
axisFormat %s
section baseline
no_agent (385.976 µs) : 366, 406
. : milestone, 386,
iast (521.471 µs) : 499, 544
. : milestone, 521,
iast_FULL (731.506 µs) : 709, 754
. : milestone, 732,
iast_GLOBAL (571.172 µs) : 549, 593
. : milestone, 571,
iast_HARDCODED_SECRET_DISABLED (526.626 µs) : 503, 550
. : milestone, 527,
iast_INACTIVE (461.326 µs) : 439, 484
. : milestone, 461,
iast_TELEMETRY_OFF (506.399 µs) : 484, 529
. : milestone, 506,
tracing (466.703 µs) : 444, 489
. : milestone, 467,
section candidate
no_agent (382.459 µs) : 362, 403
. : milestone, 382,
iast (514.054 µs) : 492, 536
. : milestone, 514,
iast_FULL (736.171 µs) : 714, 758
. : milestone, 736,
iast_GLOBAL (557.15 µs) : 535, 579
. : milestone, 557,
iast_HARDCODED_SECRET_DISABLED (524.506 µs) : 502, 547
. : milestone, 525,
iast_INACTIVE (469.423 µs) : 447, 492
. : milestone, 469,
iast_TELEMETRY_OFF (515.865 µs) : 493, 539
. : milestone, 516,
tracing (460.479 µs) : 438, 483
. : milestone, 460,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~30b0291160, baseline=1.50.0-SNAPSHOT~0903cf408b
dateFormat X
axisFormat %s
section baseline
no_agent (15.267 s) : 15267000, 15267000
. : milestone, 15267000,
appsec (15.015 s) : 15015000, 15015000
. : milestone, 15015000,
iast (19.04 s) : 19040000, 19040000
. : milestone, 19040000,
iast_GLOBAL (18.144 s) : 18144000, 18144000
. : milestone, 18144000,
profiling (15.411 s) : 15411000, 15411000
. : milestone, 15411000,
tracing (15.099 s) : 15099000, 15099000
. : milestone, 15099000,
section candidate
no_agent (15.656 s) : 15656000, 15656000
. : milestone, 15656000,
appsec (14.677 s) : 14677000, 14677000
. : milestone, 14677000,
iast (18.934 s) : 18934000, 18934000
. : milestone, 18934000,
iast_GLOBAL (18.292 s) : 18292000, 18292000
. : milestone, 18292000,
profiling (15.63 s) : 15630000, 15630000
. : milestone, 15630000,
tracing (14.915 s) : 14915000, 14915000
. : milestone, 14915000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~30b0291160, baseline=1.50.0-SNAPSHOT~0903cf408b
dateFormat X
axisFormat %s
section baseline
no_agent (1.47 ms) : 1458, 1481
. : milestone, 1470,
appsec (2.409 ms) : 2359, 2458
. : milestone, 2409,
iast (2.196 ms) : 2134, 2258
. : milestone, 2196,
iast_GLOBAL (2.235 ms) : 2172, 2297
. : milestone, 2235,
profiling (2.07 ms) : 2019, 2121
. : milestone, 2070,
tracing (2.021 ms) : 1972, 2070
. : milestone, 2021,
section candidate
no_agent (1.479 ms) : 1467, 1490
. : milestone, 1479,
appsec (2.408 ms) : 2359, 2458
. : milestone, 2408,
iast (2.191 ms) : 2128, 2254
. : milestone, 2191,
iast_GLOBAL (2.237 ms) : 2174, 2300
. : milestone, 2237,
profiling (2.037 ms) : 1987, 2087
. : milestone, 2037,
tracing (2.021 ms) : 1972, 2069
. : milestone, 2021,
|
mhlidd
requested changes
Apr 22, 2025
Comment on lines
-1531
to
-1527
| links = new ArrayList<>(); | ||
| parentContext = null; | ||
| } else if (behaviorExtract == TracePropagationBehaviorExtract.RESTART) { | ||
| links = new ArrayList<>(); |
There was a problem hiding this comment.
One feature in the RFC is that we don't want to include any span links that are created from terminated_context during the extraction phase. The resetting of the links ArrayList was to remove those, since there is a call to addTerminatedContextAsLinks() in buildSpan, prior to the call to buildSpanContext(). Maybe that call should be removed?
There was a problem hiding this comment.
That was only an early try to check how the CI would behave.
Yes, the call from buildSpan went away, the main goal is to handle the terminated context / span link in a single location (buildSpanContext) to avoid multiple allocation.
I keep trying thing in this PR, because I feel the isRemote() behavior on DDSpanContext is wrong compared to its interface behavior. It's only usage what for last parent id tag, which also need improvements / refactoring.
| .build()) | ||
| : SpanLink.from(parentContext); | ||
| links.add(link); | ||
| if (Config.get().getTracePropagationBehaviorExtract() == RESTART) { |
There was a problem hiding this comment.
Suggested change
| if (Config.get().getTracePropagationBehaviorExtract() == RESTART) { | |
| if (Config.get().getTracePropagationBehaviorExtract() == RESTART || Config.get().getTracePropagationBehaviorExtract() == IGNORE) { |
PerfectSlayer
force-pushed
the
bbujon/trace-propagation
branch
2 times, most recently
from
8aeb60d to
cf070a5
Compare
mhlidd
approved these changes
Apr 24, 2025
| if (propagationTags.getLastParentId() == null) { | ||
| if (context.isRemote()) { | ||
| CharSequence lastParentId = context.getLastParentId(); | ||
| if (lastParentId != null) { | ||
| propagationTags.updateLastParentId(lastParentId); | ||
| } | ||
| } else { | ||
| propagationTags.updateLastParentId(DDSpanId.toHexStringPadded(context.getSpanId())); | ||
| } | ||
| } | ||
|
|
||
| propagationTags.updateLastParentId(DDSpanId.toHexStringPadded(context.getSpanId())); |
There was a problem hiding this comment.
What is the reason for removing this here? It feels like to me that the end effect would be different since we are no longer checking for null before doing an update of the parentId?
There was a problem hiding this comment.
Yeah, and the previous behavior feels bogus to me.
We want the last parent id to be the current span id when injected, right?
The last parent id is stored in the root span (of the local pending trace).
So if at some point the last parent id is set, and later on another span of the same local trace we do injection, the last parent id will already be set (to some other span, not the current one) and it won’t be updated.
So with this change, I’m enforcing to always use the current span id as last parent id.
Should I write some test to check for the expected behavior? -- and test it again master and this PR
There was a problem hiding this comment.
I added some unit tests and @zacharycmontoya is writing some system tests here: DataDog/system-tests#4595
There was a problem hiding this comment.
FYI @PerfectSlayer the system-tests have been merged now with a Java skip
PerfectSlayer
force-pushed
the
bbujon/trace-propagation
branch
from
a84184b to
b9ce87a
Compare
Datadog Summary✅ Code Quality ✅ Code Security ✅ Dependencies Was this helpful? Give us feedback! |
Avoid adding terminated contexts as span links and clearing them if not needed.
PerfectSlayer
force-pushed
the
bbujon/trace-propagation
branch
from
b9ce87a to
30b0291
Compare
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Avoid adding terminated contexts as span links and clearing them if not needed.
Motivation
The recent trace propagation behavior complicated the context propagation and ended up creating span links that can be clear if context is restart.
In addition, it can unexpectedly void customer manually added span links too.
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: LANGPLAT-452