Enhance GraphQL OTEL instrumentation with custom metrics and traces #2673

RubenCerna2079 · 2025-05-02T16:59:41Z

Why make this change?

This closes #2642

What is this change?

This PR enhances the OTEL instrumentation for the GraphQL APIs by adding custom traces and metrics.
Metrics can be filtered for status_code, api_type, endpoint and method.

How was this tested?

Local Testing
Integration Tests
Unit Tests

All of the tests were done locally to check if the log information that was provided was correct, for both scenarios in which the query gave the proper information or when an exception was raised.
Another thing that was tested is that when we open GraphQL it would send a few requests called Introspection Queries used to ensure that GraphQL is working properly. However, we do not want the user to see these requests as part of the total count as this is done automatically, which may confuse the users.

Sample Request(s)

Instructions on how to use DAB Workbench

Clone the following repo https://github.com/tommasodotNET/dab-workbench.git
Run your DAB version in CLI so the files from the out folder are created, and make sure to stop it before running the DAB Workbench since both cannot be running at the same time.
Find the path to the Microsoft.DataApiBuilder.exe, which should look something like <PATH_TO_REPO>\data-api-builder\src\out\cli\net8.0\Microsoft.DataApiBuilder.exe
Copy the path of the .exe file and paste it in the file /DABWorkbench.AppHost/Program.cs in the variable dabCLIPath which is found in line 3 as follows:
var dabCLIPath = @"<PATH_TO_REPO>\data-api-builder\src\out\cli\net8.0\Microsoft.DataApiBuilder.exe";
Now you should be able to run DAB Workbench with your version of DAB.

Copilot

Pull Request Overview

This PR enhances the OTEL instrumentation for GraphQL APIs by incorporating custom traces and metrics. Key changes include adding new telemetry using directives and namespaces, integrating activity tracking in query execution, and updating the BuildRequestStateMiddleware to support custom telemetry for GraphQL operations.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/Service/Startup.cs	Added telemetry using directive.
src/Service/Controllers/RestController.cs	Updated telemetry namespace usage to align with telemetry enhancements.
src/Core/Telemetry/TelemetryTracesHelper.cs	Updated namespace and provided clarified telemetry trace comments.
src/Core/Telemetry/TelemetryMetricsHelper.cs	Updated namespace and removed unneeded using statements.
src/Core/Services/ExecutionHelper.cs	Added telemetry activity tracking for both query and mutation execution.
src/Core/Services/BuildRequestStateMiddleware.cs	Integrated telemetry, runtime config provider, and error tracking.

src/Core/Services/BuildRequestStateMiddleware.cs

RubenCerna2079 · 2025-05-02T17:01:52Z

/azp run

azure-pipelines · 2025-05-02T17:02:16Z

Azure Pipelines successfully started running 6 pipeline(s).

RubenCerna2079 · 2025-05-02T17:13:51Z

/azp run

azure-pipelines · 2025-05-02T17:14:16Z

Azure Pipelines successfully started running 6 pipeline(s).

RubenCerna2079 · 2025-05-02T20:57:27Z

/azp run

azure-pipelines · 2025-05-02T20:57:50Z

Azure Pipelines successfully started running 6 pipeline(s).

src/Core/Services/BuildRequestStateMiddleware.cs

src/Core/Services/ExecutionHelper.cs

src/Core/Services/BuildRequestStateMiddleware.cs

sourabh1007 · 2025-05-06T00:57:53Z

You should follow open telemetry semantic conventions for otel metrics and trace.
For HTTP related information, this should be the semantic convention: https://opentelemetry.io/docs/specs/semconv/http/http-spans/

There are few HTTP metrics already available, https://opentelemetry.io/docs/specs/semconv/http/http-metrics/.

Similar exercise, we did for Cosmos DB SDK. ref. https://opentelemetry.io/docs/specs/semconv/database/cosmosdb/. Right now, it is under preview.

@Aniruddh25 what do you think about it?

RubenCerna2079 · 2025-05-07T20:35:29Z

@sourabh1007 I will be addressing the semantic conventions for OTEL in a future PR.
Thank you for letting us know about this!

RubenCerna2079 · 2025-05-08T17:27:13Z

/azp run

azure-pipelines · 2025-05-08T17:27:43Z

Azure Pipelines successfully started running 6 pipeline(s).

RubenCerna2079 · 2025-05-08T18:19:28Z

/azp run

azure-pipelines · 2025-05-08T18:19:50Z

Azure Pipelines successfully started running 6 pipeline(s).

src/Core/Telemetry/TelemetryTracesHelper.cs

RubenCerna2079 · 2025-05-14T18:30:05Z

/azp run

azure-pipelines · 2025-05-14T18:30:31Z

Azure Pipelines successfully started running 6 pipeline(s).

RubenCerna2079 · 2025-05-15T17:04:40Z

/azp run

azure-pipelines · 2025-05-15T17:05:03Z

Azure Pipelines successfully started running 6 pipeline(s).

…2673) This closes #2642 This PR enhances the OTEL instrumentation for the GraphQL APIs by adding custom traces and metrics. Metrics can be filtered for `status_code`, `api_type`, `endpoint` and `method`. - [X] Local Testing - [ ] Integration Tests - [ ] Unit Tests All of the tests were done locally to check if the log information that was provided was correct, for both scenarios in which the query gave the proper information or when an exception was raised. Another thing that was tested is that when we open GraphQL it would send a few requests called `Introspection Queries` used to ensure that GraphQL is working properly. However, we do not want the user to see these requests as part of the total count as this is done automatically, which may confuse the users. ![image](https://github.com/user-attachments/assets/1f66da36-d537-47cc-95d6-fd19cf73242e) ![image](https://github.com/user-attachments/assets/91b3801f-b48e-4841-99a6-72de1b8b482a) ![image](https://github.com/user-attachments/assets/48a8ee9a-4d17-4b52-9f66-7e5c745aeef4) - Clone the following repo `https://github.com/tommasodotNET/dab-workbench.git` - Run your DAB version in CLI so the files from the `out` folder are created, and make sure to stop it before running the DAB Workbench since both cannot be running at the same time. - Find the path to the `Microsoft.DataApiBuilder.exe`, which should look something like `<PATH_TO_REPO>\data-api-builder\src\out\cli\net8.0\Microsoft.DataApiBuilder.exe` - Copy the path of the `.exe` file and paste it in the file `/DABWorkbench.AppHost/Program.cs` in the variable `dabCLIPath` which is found in line 3 as follows: `var dabCLIPath = @"<PATH_TO_REPO>\data-api-builder\src\out\cli\net8.0\Microsoft.DataApiBuilder.exe";` - Now you should be able to run DAB Workbench with your version of DAB. --------- Co-authored-by: Tommaso Stocchi <[email protected]> Co-authored-by: Jerry Nixon <[email protected]> Co-authored-by: Ruben Cerna <[email protected]>

## Why make this change? This change is made in order to add all of the commits for milestone 1.5 into its respective branch. ## What is this change? This change cherry-picks all of the commits that were added after the first release candidate. Cherry-picked commits: - #2648 #2657 #2617 #2659 #2655 #2633 #2667 #2673 #2650 #2695 #2702 #2688 ## How was this tested? - [ ] Integration Tests - [ ] Unit Tests ## Sample Request(s) --------- Co-authored-by: Sezal Chug <[email protected]> Co-authored-by: sezalchug <[email protected]> Co-authored-by: Tommaso Stocchi <[email protected]> Co-authored-by: Aaron Powell <[email protected]> Co-authored-by: aaronburtle <[email protected]> Co-authored-by: Aniruddh Munde <[email protected]> Co-authored-by: Jerry Nixon <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Michael Staib <[email protected]> Co-authored-by: souvikghosh04 <[email protected]> Co-authored-by: Souvik Ghosh <[email protected]>

…2673) This closes #2642 This PR enhances the OTEL instrumentation for the GraphQL APIs by adding custom traces and metrics. Metrics can be filtered for `status_code`, `api_type`, `endpoint` and `method`. - [X] Local Testing - [ ] Integration Tests - [ ] Unit Tests All of the tests were done locally to check if the log information that was provided was correct, for both scenarios in which the query gave the proper information or when an exception was raised. Another thing that was tested is that when we open GraphQL it would send a few requests called `Introspection Queries` used to ensure that GraphQL is working properly. However, we do not want the user to see these requests as part of the total count as this is done automatically, which may confuse the users. ![image](https://github.com/user-attachments/assets/1f66da36-d537-47cc-95d6-fd19cf73242e) ![image](https://github.com/user-attachments/assets/91b3801f-b48e-4841-99a6-72de1b8b482a) ![image](https://github.com/user-attachments/assets/48a8ee9a-4d17-4b52-9f66-7e5c745aeef4) - Clone the following repo `https://github.com/tommasodotNET/dab-workbench.git` - Run your DAB version in CLI so the files from the `out` folder are created, and make sure to stop it before running the DAB Workbench since both cannot be running at the same time. - Find the path to the `Microsoft.DataApiBuilder.exe`, which should look something like `<PATH_TO_REPO>\data-api-builder\src\out\cli\net8.0\Microsoft.DataApiBuilder.exe` - Copy the path of the `.exe` file and paste it in the file `/DABWorkbench.AppHost/Program.cs` in the variable `dabCLIPath` which is found in line 3 as follows: `var dabCLIPath = @"<PATH_TO_REPO>\data-api-builder\src\out\cli\net8.0\Microsoft.DataApiBuilder.exe";` - Now you should be able to run DAB Workbench with your version of DAB. --------- Co-authored-by: Tommaso Stocchi <[email protected]> Co-authored-by: Jerry Nixon <[email protected]> Co-authored-by: Ruben Cerna <[email protected]>

RubenCerna2079 added this to the 1.5 milestone May 2, 2025

RubenCerna2079 self-assigned this May 2, 2025

RubenCerna2079 added this to Data API builder May 2, 2025

Copilot AI review requested due to automatic review settings May 2, 2025 16:59

RubenCerna2079 requested review from Aniruddh25, aaronburtle, neeraj-sharma2592, ravishetye, sezal98 and sourabh1007 as code owners May 2, 2025 16:59

github-project-automation bot moved this to Todo in Data API builder May 2, 2025

Copilot AI reviewed May 2, 2025

View reviewed changes

src/Core/Services/BuildRequestStateMiddleware.cs Outdated Show resolved Hide resolved

RubenCerna2079 changed the title ~~Jerry/otel ruben~~ Enhance GraphQL OTEL instrumentation with custom metrics and traces May 2, 2025

Aniruddh25 reviewed May 6, 2025

View reviewed changes

src/Core/Services/BuildRequestStateMiddleware.cs Outdated Show resolved Hide resolved

Aniruddh25 reviewed May 6, 2025

View reviewed changes

src/Core/Services/BuildRequestStateMiddleware.cs Outdated Show resolved Hide resolved

Aniruddh25 reviewed May 6, 2025

View reviewed changes

src/Core/Services/ExecutionHelper.cs Show resolved Hide resolved

Aniruddh25 reviewed May 6, 2025

View reviewed changes

src/Core/Services/BuildRequestStateMiddleware.cs Show resolved Hide resolved

Aniruddh25 reviewed May 8, 2025

View reviewed changes

src/Core/Telemetry/TelemetryTracesHelper.cs Outdated Show resolved Hide resolved

tommasodotNET and others added 15 commits May 14, 2025 11:27

adds docs in TelemetryMetricsHelper

273b7ba

removes unnecessary usings in program.cs

290b19d

fixes missing meter name

17ec1cd

fixes logs

adf7e4d

ONLY NINE FILES CHANGED! Technically 5.

3b0faed

Merge conflict changes

c50af46

Get httpstatuscode for graphql

f28fb20

Fix otel

9d77b6a

Finish OTEL changes

721b020

Fix test errors

bc70551

Fix test errors

596b0f7

Changes based on comments

bb70182

Fix library changes

f5f8967

Changes based on comments

c87c892

Changes based on comments

8de770a

RubenCerna2079 force-pushed the jerry/otel-ruben branch from fe4f6e3 to 8de770a Compare May 14, 2025 18:29

Merge branch 'main' into jerry/otel-ruben

c29028f

sezal98 approved these changes May 15, 2025

View reviewed changes

RubenCerna2079 merged commit 7f8fa6b into main May 15, 2025
11 checks passed

RubenCerna2079 deleted the jerry/otel-ruben branch May 15, 2025 22:10

github-project-automation bot moved this from Review In Progress to Done in Data API builder May 15, 2025

RubenCerna2079 mentioned this pull request May 29, 2025

Cherry-Picking Commits for 1.5 #2689

Merged

2 tasks

Enhance GraphQL OTEL instrumentation with custom metrics and traces #2673

Enhance GraphQL OTEL instrumentation with custom metrics and traces #2673

Uh oh!

Conversation

RubenCerna2079 commented May 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why make this change?

What is this change?

How was this tested?

Sample Request(s)

Instructions on how to use DAB Workbench

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

RubenCerna2079 commented May 2, 2025

Uh oh!

azure-pipelines bot commented May 2, 2025

Uh oh!

RubenCerna2079 commented May 2, 2025

Uh oh!

azure-pipelines bot commented May 2, 2025

Uh oh!

RubenCerna2079 commented May 2, 2025

Uh oh!

azure-pipelines bot commented May 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sourabh1007 commented May 6, 2025

Uh oh!

RubenCerna2079 commented May 7, 2025

Uh oh!

RubenCerna2079 commented May 8, 2025

Uh oh!

azure-pipelines bot commented May 8, 2025

Uh oh!

RubenCerna2079 commented May 8, 2025

Uh oh!

azure-pipelines bot commented May 8, 2025

Uh oh!

Uh oh!

RubenCerna2079 commented May 14, 2025

Uh oh!

azure-pipelines bot commented May 14, 2025

Uh oh!

RubenCerna2079 commented May 15, 2025

Uh oh!

azure-pipelines bot commented May 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

RubenCerna2079 commented May 2, 2025 •

edited

Loading