Skip to content

Conversation

dd-octo-sts[bot]
Copy link
Contributor

@dd-octo-sts dd-octo-sts bot commented Sep 14, 2025

Updates the package versions for integration tests.

@dd-octo-sts dd-octo-sts bot requested review from a team as code owners September 14, 2025 00:30
@dd-octo-sts dd-octo-sts bot added area:dependabot dependabot updates area:test-apps apps used to test integrations dependencies Pull requests that update a dependency file labels Sep 14, 2025
@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Sep 14, 2025

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7515) - mean (72ms)  : 71, 73
     .   : milestone, 72,
    master - mean (72ms)  : 71, 73
     .   : milestone, 72,

    section Baseline
    This PR (7515) - mean (68ms)  : 66, 70
     .   : milestone, 68,
    master - mean (68ms)  : 66, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7515) - mean (1,000ms)  : 974, 1027
     .   : milestone, 1000,
    master - mean (999ms)  : 976, 1022
     .   : milestone, 999,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7515) - mean (106ms)  : 105, 107
     .   : milestone, 106,
    master - mean (106ms)  : 105, 108
     .   : milestone, 106,

    section Baseline
    This PR (7515) - mean (106ms)  : 102, 109
     .   : milestone, 106,
    master - mean (105ms)  : 103, 108
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (7515) - mean (715ms)  : 696, 734
     .   : milestone, 715,
    master - mean (713ms)  : 698, 728
     .   : milestone, 713,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7515) - mean (94ms)  : 93, 95
     .   : milestone, 94,
    master - mean (94ms)  : 93, 95
     .   : milestone, 94,

    section Baseline
    This PR (7515) - mean (93ms)  : 91, 96
     .   : milestone, 93,
    master - mean (93ms)  : 91, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (7515) - mean (668ms)  : 648, 688
     .   : milestone, 668,
    master - mean (670ms)  : 652, 688
     .   : milestone, 670,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7515) - mean (93ms)  : 91, 94
     .   : milestone, 93,
    master - mean (93ms)  : 91, 94
     .   : milestone, 93,

    section Baseline
    This PR (7515) - mean (92ms)  : 89, 94
     .   : milestone, 92,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (7515) - mean (597ms)  : 584, 610
     .   : milestone, 597,
    master - mean (598ms)  : 586, 611
     .   : milestone, 598,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7515) - mean (193ms)  : 188, 199
     .   : milestone, 193,
    master - mean (193ms)  : 188, 198
     .   : milestone, 193,

    section Baseline
    This PR (7515) - mean (191ms)  : 185, 196
     .   : milestone, 191,
    master - mean (192ms)  : 187, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (7515) - mean (1,103ms)  : 1068, 1138
     .   : milestone, 1103,
    master - mean (1,106ms)  : 1073, 1139
     .   : milestone, 1106,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7515) - mean (274ms)  : 270, 279
     .   : milestone, 274,
    master - mean (274ms)  : 267, 282
     .   : milestone, 274,

    section Baseline
    This PR (7515) - mean (274ms)  : 268, 280
     .   : milestone, 274,
    master - mean (274ms)  : 267, 280
     .   : milestone, 274,

    section CallTarget+Inlining+NGEN
    This PR (7515) - mean (896ms)  : 863, 929
     .   : milestone, 896,
    master - mean (895ms)  : 856, 934
     .   : milestone, 895,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7515) - mean (267ms)  : 261, 274
     .   : milestone, 267,
    master - mean (267ms)  : 257, 276
     .   : milestone, 267,

    section Baseline
    This PR (7515) - mean (269ms)  : 264, 274
     .   : milestone, 269,
    master - mean (266ms)  : 258, 274
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (7515) - mean (879ms)  : 847, 911
     .   : milestone, 879,
    master - mean (877ms)  : 833, 921
     .   : milestone, 877,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7515) - mean (266ms)  : 258, 274
     .   : milestone, 266,
    master - mean (267ms)  : 260, 274
     .   : milestone, 267,

    section Baseline
    This PR (7515) - mean (266ms)  : 260, 272
     .   : milestone, 266,
    master - mean (264ms)  : 254, 273
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (7515) - mean (782ms)  : 763, 801
     .   : milestone, 782,
    master - mean (788ms)  : 760, 815
     .   : milestone, 788,

Loading

Copy link

datadog-official bot commented Sep 14, 2025

⚠️ Tests

⚠️ Warnings

🧪 5 Tests failed

SubmitsTraces from Datadog.Trace.ClrProfiler.IntegrationTests.MongoDbTests (Datadog) (✨ Fix with BitsAI)
Expected exit code: 0, actual exit code: 134.
SubmitsTraces from Datadog.Trace.ClrProfiler.IntegrationTests.MongoDbTests (Datadog) (✨ Fix with BitsAI)
Expected exit code: 0, actual exit code: 134.
SubmitsTraces from Datadog.Trace.ClrProfiler.IntegrationTests.MongoDbTests (Datadog) (✨ Fix with BitsAI)
Expected exit code: 0, actual exit code: 134.
View all

ℹ️ Info

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4b239dc | Docs | Was this helpful? Give us feedback!

Comment on lines +13 to +14
<!-- Latest package https://www.nuget.org/packages/MongoDB.Driver/3.5.0 -->
<PackageReference Include="MongoDB.Driver" Version="3.5.0" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is failing because it requires a newer version of the mongodb itself. That means we definitely need to update the mongodb image (and so update all the VMs), but it may also mean that we actually need to run both old and new versions of mongodb, and split which one we talk to based on the package version 🙁

01:25:04 [DBG]  Unhandled exception. MongoDB.Driver.MongoIncompatibleDriverException: Server at mongo:27017 reports wire version 7, but this version of the driver requires at least 8 (MongoDB 4.2.0).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😭

@dd-octo-sts dd-octo-sts bot force-pushed the bot/test-package-versions-bump branch from e645e2c to 4b239dc Compare September 21, 2025 00:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dependabot dependabot updates area:test-apps apps used to test integrations dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants