[🍒 8086] Fix tracing JUnit5 tests in Maven projects with multiple forks #8087
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nikita-tkachenko-datadog
added
type: bug
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
anmarchenko
approved these changes
Dec 13, 2024
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 56 metrics, 7 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~d5970c3996, baseline=1.45.0-SNAPSHOT~baedf8dd57
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.097 s) : 0, 1096654
Total [baseline] (8.658 s) : 0, 8657632
Agent [candidate] (1.091 s) : 0, 1091352
Total [candidate] (8.619 s) : 0, 8619471
section iast
Agent [baseline] (1.222 s) : 0, 1221649
Total [baseline] (9.214 s) : 0, 9213904
Agent [candidate] (1.229 s) : 0, 1228834
Total [candidate] (9.248 s) : 0, 9248293
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.221 s) : 0, 1221155
Total [baseline] (9.168 s) : 0, 9167534
Agent [candidate] (1.22 s) : 0, 1219811
Total [candidate] (9.193 s) : 0, 9193433
section iast_TELEMETRY_OFF
Agent [baseline] (1.217 s) : 0, 1217030
Total [baseline] (9.198 s) : 0, 9197830
Agent [candidate] (1.226 s) : 0, 1226020
Total [candidate] (9.222 s) : 0, 9222022
gantt
title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~d5970c3996, baseline=1.45.0-SNAPSHOT~baedf8dd57
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (698.478 ms) : 0, 698478
BytebuddyAgent [candidate] (695.01 ms) : 0, 695010
GlobalTracer [baseline] (317.669 ms) : 0, 317669
GlobalTracer [candidate] (316.574 ms) : 0, 316574
AppSec [baseline] (54.585 ms) : 0, 54585
AppSec [candidate] (54.627 ms) : 0, 54627
Remote Config [baseline] (676.416 µs) : 0, 676
Remote Config [candidate] (688.966 µs) : 0, 689
Telemetry [baseline] (11.499 ms) : 0, 11499
Telemetry [candidate] (10.687 ms) : 0, 10687
section iast
BytebuddyAgent [baseline] (812.929 ms) : 0, 812929
BytebuddyAgent [candidate] (819.53 ms) : 0, 819530
GlobalTracer [baseline] (307.263 ms) : 0, 307263
GlobalTracer [candidate] (308.229 ms) : 0, 308229
AppSec [baseline] (57.471 ms) : 0, 57471
AppSec [candidate] (58.104 ms) : 0, 58104
Remote Config [baseline] (619.447 µs) : 0, 619
Remote Config [candidate] (613.835 µs) : 0, 614
Telemetry [baseline] (7.511 ms) : 0, 7511
Telemetry [candidate] (7.496 ms) : 0, 7496
IAST [baseline] (22.127 ms) : 0, 22127
IAST [candidate] (20.999 ms) : 0, 20999
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (813.792 ms) : 0, 813792
BytebuddyAgent [candidate] (813.374 ms) : 0, 813374
GlobalTracer [baseline] (306.539 ms) : 0, 306539
GlobalTracer [candidate] (305.364 ms) : 0, 305364
AppSec [baseline] (57.963 ms) : 0, 57963
AppSec [candidate] (58.163 ms) : 0, 58163
Remote Config [baseline] (625.615 µs) : 0, 626
Remote Config [candidate] (605.71 µs) : 0, 606
Telemetry [baseline] (7.376 ms) : 0, 7376
Telemetry [candidate] (7.454 ms) : 0, 7454
IAST [baseline] (21.132 ms) : 0, 21132
IAST [candidate] (21.098 ms) : 0, 21098
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (811.037 ms) : 0, 811037
BytebuddyAgent [candidate] (817.649 ms) : 0, 817649
GlobalTracer [baseline] (305.754 ms) : 0, 305754
GlobalTracer [candidate] (307.259 ms) : 0, 307259
AppSec [baseline] (57.184 ms) : 0, 57184
AppSec [candidate] (58.278 ms) : 0, 58278
Remote Config [baseline] (629.207 µs) : 0, 629
Remote Config [candidate] (645.544 µs) : 0, 646
Telemetry [baseline] (7.315 ms) : 0, 7315
Telemetry [candidate] (7.461 ms) : 0, 7461
IAST [baseline] (21.397 ms) : 0, 21397
IAST [candidate] (20.915 ms) : 0, 20915
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~d5970c3996, baseline=1.45.0-SNAPSHOT~baedf8dd57
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.101 s) : 0, 1101226
Total [baseline] (10.463 s) : 0, 10463312
Agent [candidate] (1.101 s) : 0, 1100530
Total [candidate] (10.463 s) : 0, 10462868
section appsec
Agent [baseline] (1.229 s) : 0, 1228910
Total [baseline] (10.716 s) : 0, 10716040
Agent [candidate] (1.234 s) : 0, 1234066
Total [candidate] (10.741 s) : 0, 10740672
section iast
Agent [baseline] (1.225 s) : 0, 1225420
Total [baseline] (11.007 s) : 0, 11006666
Agent [candidate] (1.22 s) : 0, 1219539
Total [candidate] (11.012 s) : 0, 11012115
section profiling
Agent [baseline] (1.324 s) : 0, 1323685
Total [baseline] (10.814 s) : 0, 10813740
Agent [candidate] (1.315 s) : 0, 1315287
Total [candidate] (10.809 s) : 0, 10809345
gantt
title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~d5970c3996, baseline=1.45.0-SNAPSHOT~baedf8dd57
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (700.84 ms) : 0, 700840
BytebuddyAgent [candidate] (700.136 ms) : 0, 700136
GlobalTracer [baseline] (319.137 ms) : 0, 319137
GlobalTracer [candidate] (318.926 ms) : 0, 318926
AppSec [baseline] (54.691 ms) : 0, 54691
AppSec [candidate] (54.772 ms) : 0, 54772
Remote Config [baseline] (672.526 µs) : 0, 673
Remote Config [candidate] (689.138 µs) : 0, 689
Telemetry [baseline] (12.063 ms) : 0, 12063
Telemetry [candidate] (12.138 ms) : 0, 12138
section appsec
BytebuddyAgent [baseline] (714.315 ms) : 0, 714315
BytebuddyAgent [candidate] (717.184 ms) : 0, 717184
GlobalTracer [baseline] (314.931 ms) : 0, 314931
GlobalTracer [candidate] (317.049 ms) : 0, 317049
AppSec [baseline] (167.956 ms) : 0, 167956
AppSec [candidate] (167.764 ms) : 0, 167764
Remote Config [baseline] (651.483 µs) : 0, 651
Remote Config [candidate] (645.557 µs) : 0, 646
Telemetry [baseline] (7.41 ms) : 0, 7410
Telemetry [candidate] (8.111 ms) : 0, 8111
IAST [baseline] (19.869 ms) : 0, 19869
IAST [candidate] (19.097 ms) : 0, 19097
section iast
BytebuddyAgent [baseline] (816.451 ms) : 0, 816451
BytebuddyAgent [candidate] (812.906 ms) : 0, 812906
GlobalTracer [baseline] (308.088 ms) : 0, 308088
GlobalTracer [candidate] (305.662 ms) : 0, 305662
AppSec [baseline] (57.021 ms) : 0, 57021
AppSec [candidate] (57.152 ms) : 0, 57152
Remote Config [baseline] (616.816 µs) : 0, 617
Remote Config [candidate] (618.507 µs) : 0, 619
Telemetry [baseline] (7.52 ms) : 0, 7520
Telemetry [candidate] (7.444 ms) : 0, 7444
IAST [baseline] (22.011 ms) : 0, 22011
IAST [candidate] (22.006 ms) : 0, 22006
section profiling
BytebuddyAgent [baseline] (693.332 ms) : 0, 693332
BytebuddyAgent [candidate] (689.744 ms) : 0, 689744
GlobalTracer [baseline] (435.122 ms) : 0, 435122
GlobalTracer [candidate] (430.68 ms) : 0, 430680
AppSec [baseline] (53.811 ms) : 0, 53811
AppSec [candidate] (53.536 ms) : 0, 53536
Remote Config [baseline] (684.388 µs) : 0, 684
Remote Config [candidate] (659.11 µs) : 0, 659
Telemetry [baseline] (7.736 ms) : 0, 7736
Telemetry [candidate] (7.661 ms) : 0, 7661
ProfilingAgent [baseline] (93.896 ms) : 0, 93896
ProfilingAgent [candidate] (93.86 ms) : 0, 93860
Profiling [baseline] (93.92 ms) : 0, 93920
Profiling [candidate] (93.884 ms) : 0, 93884
LoadDacapo |
nikita-tkachenko-datadog
deleted the
nikita-tkachenko/fix-maven-multiple-forks-backport
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry pick #8086
What Does This Do
Fixes a regression introduced in release v1.44.0 (#8055).
When tracing JUnit 5 tests in Maven projects with multiple forks the following exception was seen:
Maven Surefire plugin can be configured to fork multiple JVMs per test task in order to parallelise tests execution in multiple processes:
When configure like this, Maven Surefire will invoke
AbstractPlatformProvider.getSuites()in order to list available test cases and divide them between the forks.If the tests are in JUnit 5, the platform provide will in turn invoke
JupiterTestEngine.discover()to get the list of tests.Release v1.44.0 introduced some changes in the way JUnit 5 is instrumented.
Among them was early initialisation of
TestEventHandler(needed to have the handler available by the time class and method orderers were requested from JUnit).Early initialisation happens at
JupiterTestEngine.discover()- the assumption was that this method was only executed in child JVMs.Since in the parent JVM (Maven build process)
TestEventHandlerfactory is not configured, early initialisation caused NPEs.The fix is to create a no-op TestEventHandler implementation in the parent process as a stub, so that the instrumentation does not fail.
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: SDTEST-1336