Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Turn on scenario app screenshots #49066

Merged
merged 3 commits into from
Dec 21, 2023
Merged

Turn on scenario app screenshots #49066

merged 3 commits into from
Dec 21, 2023

Conversation

jmagman
Copy link
Member

@jmagman jmagman commented Dec 15, 2023

Save a recording/screenshot to the uploaded xcresult when a Scenario test fails.

Screenshot 2023-12-14 at 6 34 47 PM Screenshot 2023-12-14 at 6 41 03 PM

No more errant spew as reported in #19668 when this was originally turned off.

 Test Case '-[StatusBarTest testTapStatusBar]' started.
     t =     0.00s Start Test at 2023-12-14 18:20:34.207
...
     t =     2.69s     Synthesize event
     t =     2.70s         Find the StatusBar at {{0.0, 0.0}, {375.0, 20.0}}
     t =     2.72s         Find the StatusBar at {{0.0, 0.0}, {375.0, 20.0}}
     t =     2.74s         Find the StatusBar at {{0.0, 0.0}, {375.0, 20.0}}
     t =     3.03s     Wait for com.apple.springboard to idle
     t =     3.43s Waiting 1.0s for "0,PointerChange.add,device=0,buttons=0,signalKind=PointerSignalKind.none" TextField to exist
     t =     4.43s     Checking `Expect predicate `exists == 1` for object "0,PointerChange.add,device=0,buttons=0,signalKind=PointerSignalKind.none" TextField`
     t =     4.43s         Checking existence of `"0,PointerChange.add,device=0,buttons=0,signalKind=PointerSignalKind.none" TextField`
     t =     4.45s         Capturing element debug description
     t =     4.45s Checking existence of `"0,PointerChange.add,device=0,buttons=0,signalKind=PointerSignalKind.none" TextField`
 /Volumes/Work/s/w/ir/cache/builder/src/out/ios_debug_sim/scenario_app/Scenarios/ScenariosUITests/StatusBarTest.m:37: error: -[StatusBarTest testTapStatusBar] : ((exists) is true) failed
     t =     4.48s Tear Down

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8761683198070320113/+/u/test:_Scenario_App_Integration_Tests__3_/stdout

Fixes flutter/flutter#140192

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
  • I listed at least one issue that this PR fixes in the description above.
  • I added new tests to check the change I am making or feature I am adding, or the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
  • I updated/added relevant documentation (doc comments with ///).
  • I signed the CLA.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@jmagman jmagman self-assigned this Dec 15, 2023
@jmagman jmagman marked this pull request as ready for review December 15, 2023 02:40
@jmagman jmagman requested a review from vashworth December 15, 2023 02:40
@flutter-dashboard
Copy link

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact "@text-exemption-reviewers" in the #hackers channel in Chat (don't just cc them here, they won't see it! Use Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@flutter-dashboard
Copy link

Golden file changes have been found for this pull request. Click here to view and triage (e.g. because this is an intentional change).

If you are still iterating on this change and are not ready to resolve the images on the Flutter Gold dashboard, consider marking this PR as a draft pull request above. You will still be able to view image results on the dashboard, commenting will be silenced, and the check will not try to resolve itself until marked ready for review.

Changes reported for pull request #49066 at sha c756ff3

Copy link
Contributor

@vashworth vashworth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jmagman
Copy link
Member Author

jmagman commented Dec 19, 2023

It looks like this pull request may not have tests.

I got distracted yak shaving flutter/cocoon#3358

@stuartmorgan-g
Copy link
Contributor

test-exempt: is a test

auto-submit bot pushed a commit to flutter/cocoon that referenced this pull request Dec 19, 2023
Treat [engine `scenario_app`](https://github.com/flutter/engine/tree/main/testing/scenario_app) as a test and do not add needs-test comment.  Example:  flutter/engine#49066 (comment)

Refactor tests, put all the code/test combo checks into one test to reduce boilerplate, otherwise I would have needed to add a new test for the `scenario_app` check.
@jmagman jmagman added autosubmit Merge PR when tree becomes green via auto submit App and removed will affect goldens labels Dec 21, 2023
@auto-submit auto-submit bot merged commit a2f296c into flutter:main Dec 21, 2023
@jmagman jmagman deleted the screenshot branch December 21, 2023 19:19
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Dec 21, 2023
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request Dec 22, 2023
…140527)

flutter/engine@1b1b2a1...737e6f8

2023-12-21 [email protected] Roll Dart SDK from bcf68d22f0fa to fe94d9b88531 (1 revision) (flutter/engine#49343)
2023-12-21 [email protected] Reland "[web] Enforce onDrawFrame/onBeginFrame render rule" (flutter/engine#49336)
2023-12-21 [email protected] Turn on scenario app screenshots (flutter/engine#49066)
2023-12-21 [email protected] Revert onrender change (flutter/engine#49333)
2023-12-21 [email protected] [web] autofocus in new routes (flutter/engine#47727)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
CoderDake pushed a commit to CoderDake/flutter that referenced this pull request Dec 28, 2023
…lutter#140527)

flutter/engine@1b1b2a1...737e6f8

2023-12-21 [email protected] Roll Dart SDK from bcf68d22f0fa to fe94d9b88531 (1 revision) (flutter/engine#49343)
2023-12-21 [email protected] Reland "[web] Enforce onDrawFrame/onBeginFrame render rule" (flutter/engine#49336)
2023-12-21 [email protected] Turn on scenario app screenshots (flutter/engine#49066)
2023-12-21 [email protected] Revert onrender change (flutter/engine#49333)
2023-12-21 [email protected] [web] autofocus in new routes (flutter/engine#47727)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
autosubmit Merge PR when tree becomes green via auto submit App
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Turn on screenshots for scenario test iOS failures
3 participants