Skip to content

Improve tracing #1318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 13, 2024
Merged

Improve tracing #1318

merged 1 commit into from
Mar 13, 2024

Conversation

vlad-ivanov-name
Copy link
Collaborator

  • Add opentelemetry OTLP export support
  • Rewrite top-level HTTP request handler - fix instrumentation
  • Flush spans before shutdown
  • Use OT "semantic conventions" for common fields

commit-id:4f45746e

@vlad-ivanov-name vlad-ivanov-name marked this pull request as draft March 7, 2024 10:12
@vlad-ivanov-name
Copy link
Collaborator Author

Please don't merge yet -- otel seems to be broken -- testing a fix

* Add opentelemetry OTLP export support
* Rewrite top-level HTTP request handler - fix instrumentation
* Flush spans before shutdown
* Use OT "semantic conventions" for common fields

commit-id:4f45746e
@vlad-ivanov-name
Copy link
Collaborator Author

I've reworked span association in a few places -- there were two main issues:

  1. tracing::instrument by default records INFO spans, but there were parts of the code where TRACE spans were encapsulating INFO spans. So when level is set to INFO, some spans would be missing a parent because it would be filtered out
  2. some spans lived across .await yield points, which resulted in weird issues with duplicate span enter calls and missing associations

This is now how git http requests look like on google cloud trace:

image

CleanShot 2024-03-07 at 14 01 24

I will test with jaeger now

@vlad-ivanov-name
Copy link
Collaborator Author

with jaeger:

image

PR is ready now

@vlad-ivanov-name vlad-ivanov-name marked this pull request as ready for review March 7, 2024 14:05
@vlad-ivanov-name vlad-ivanov-name merged commit fa8a7e0 into master Mar 13, 2024
@vlad-ivanov-name vlad-ivanov-name deleted the improve-tracing-otlp branch March 13, 2024 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants