Skip to content

Improve Integration Tests #2099

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bitsandfoxes opened this issue Apr 7, 2025 · 2 comments
Open

Improve Integration Tests #2099

bitsandfoxes opened this issue Apr 7, 2025 · 2 comments
Labels

Comments

@bitsandfoxes
Copy link
Contributor

bitsandfoxes commented Apr 7, 2025

Problem Statement

The current integration test is limited to running the SmokeTester and crash tests with a mock server that only confirms whether events were sent, without providing visibility into their content.
This creates significant testing gaps. For example, we cannot verify if scope synchronization is functioning correctly. This limitation is particularly evident when trying to observe trace changes introduced in getsentry/sentry-dotnet#4026.

There currently is no way to validate events coming from different layers of the application.

Proposal

Rather than depending on the SmokeTester to intercept .NET SDK send-attempts and validate them on the running client, we should enhance the crash-test-server to persist envelopes to disk. When executing the CrashTest, we should then validate both the presence and contents of these stored envelopes.

Implementation steps:

  1. Start the fake server
  2. Execute the game (SmokeTester remains to trigger errors and send events)
  3. Server receives and persists events to disk
  4. Verify successful game execution
  5. Read persisted envelopes and validate their contents against expected values

Non-exhaustive list of things to check for:

  • Reporting SDKs name (sentry.cocoa.unity)
  • Synched Scope
    • Tags
    • User
    • Breadcrumbs
@vaind
Copy link
Collaborator

vaind commented Apr 7, 2025

As an alternative, a few other SDKs actually post events to sentry.io and then fetch them through an API. Besides resolving the pain points mentioned above, that would also enable testing symbolication.

@bruno-garcia
Copy link
Member

As an alternative, a few other SDKs actually post events to sentry.io and then fetch them through an API. Besides resolving the pain points mentioned above, that would also enable testing symbolication.

React Native does this but note: It is (was?) extremely flakey and slow.
So while I'm in favor of having this (I asked for that to be added back then). I wouldn't rely on that for lots of testing.
It should be just another layer, to have perhaps a single, e2e test. But I not necessarily having lots of test cases added to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

No branches or pull requests

3 participants