Skip to content

Commit 382fc1e

Browse files
committed
test: Add integration test of _UnreadMarker animation
Added an initial integration test to capture render performance of _UnreadMarker animation. This method was helpful for comparing across different implementations of the marker to see if any method was more efficient than others. The test driver `integration_test/perf_driver.dart` is derived from BSD licensed example code in Flutter documentation about integration tests profiling, see: https://docs.flutter.dev/cookbook/testing/integration/profiling#3-save-the-results-to-disk Also added a `docs/integration_tests.md` to capture notes on the process of gathering performance metrics on physical devices.
1 parent 8f65770 commit 382fc1e

File tree

3 files changed

+145
-0
lines changed

3 files changed

+145
-0
lines changed

docs/integration_tests.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Integration Tests
2+
3+
Integration tests in Flutter allow self-driven end-to-end
4+
testing of app code running with the full GUI.
5+
6+
This document is about using integration tests to capture
7+
performance metrics on physical devices. For more
8+
information on that topic see
9+
[Flutter cookbook on integration profiling][profiling-cookbook].
10+
11+
For more background on integration testing in general
12+
see [Flutter docs on integration testing][flutter-docs].
13+
14+
[profiling-cookbook]: https://docs.flutter.dev/cookbook/testing/integration/profiling
15+
[flutter-docs]: https://docs.flutter.dev/testing/integration-tests
16+
17+
18+
## Capturing performance metrics
19+
20+
Capturing performance metrics involves two parts: an
21+
integration test that runs on a device and driver code that
22+
runs on the host.
23+
24+
Integration test code is written in a similar style as
25+
widget test code, using a `testWidgets` function as well as
26+
a `WidgetTester` instance to arrange widgets and run
27+
interactions. A difference is the usage of
28+
`IntegrationTestWidgetsFlutterBinding` which provides a
29+
`traceAction` method used to record Dart VM timelines.
30+
31+
Driver code runs on the host and is useful to configure
32+
output of captured timeline data. There is a baseline driver
33+
at `integration_test/perf_driver.dart` that additionally
34+
configures output of a timeline summary containing widget
35+
build times and frame rendering performance.
36+
37+
38+
## Obtaining performance metrics
39+
40+
First, obtain a device ID using `flutter devices`.
41+
42+
The command to run an integration test on a device:
43+
44+
```
45+
$ flutter drive \
46+
--driver=integration_test/perf_driver.dart \
47+
--target=integration_test/unreadmarker_test.dart \
48+
--profile \
49+
--no-dds \
50+
-d <device_id>
51+
```
52+
53+
A data file with raw event timings will be produced in
54+
`build/trace_output.timeline.json`.
55+
56+
A more readily consumable file will also be produced in
57+
`build/trace_output.timeline_summary.json`. This file
58+
contains widget build and render timing data in a JSON
59+
structure. See the fields `frame_build_times` and
60+
`frame_rasterizer_times` as well as the provided percentile
61+
scores of those. These values are useful for objective
62+
comparison between different runs.

integration_test/perf_driver.dart

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// This integration driver configures output of timeline data
2+
// and a summary thereof from integration tests. See
3+
// docs/integration_tests.md for background.
4+
5+
import 'package:flutter_driver/flutter_driver.dart' as driver;
6+
import 'package:integration_test/integration_test_driver.dart';
7+
8+
Future<void> main() {
9+
return integrationDriver(
10+
responseDataCallback: (data) async {
11+
if (data == null) return;
12+
final timeline = driver.Timeline.fromJson(data['timeline']);
13+
final summary = driver.TimelineSummary.summarize(timeline);
14+
await summary.writeTimelineToFile(
15+
'trace_output',
16+
pretty: true,
17+
includeSummary: true);
18+
});
19+
}
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
2+
import 'package:flutter/material.dart';
3+
import 'package:flutter_test/flutter_test.dart';
4+
import 'package:integration_test/integration_test.dart';
5+
import 'package:zulip/api/model/events.dart';
6+
import 'package:zulip/api/model/model.dart';
7+
import 'package:zulip/model/narrow.dart';
8+
import 'package:zulip/model/store.dart';
9+
import 'package:zulip/widgets/message_list.dart';
10+
import 'package:zulip/widgets/store.dart';
11+
12+
import '../test/api/fake_api.dart';
13+
import '../test/example_data.dart' as eg;
14+
import '../test/model/binding.dart';
15+
import '../test/model/message_list_test.dart';
16+
17+
void main() {
18+
final binding = IntegrationTestWidgetsFlutterBinding.ensureInitialized();
19+
TestZulipBinding.ensureInitialized();
20+
21+
late PerAccountStore store;
22+
late FakeApiConnection connection;
23+
24+
Future<List<Message>> setupMessageListPage(WidgetTester tester, int messageCount) async {
25+
addTearDown(testBinding.reset);
26+
await testBinding.globalStore.add(eg.selfAccount, eg.initialSnapshot());
27+
store = await testBinding.globalStore.perAccount(eg.selfAccount.id);
28+
connection = store.connection as FakeApiConnection;
29+
30+
// prepare message list data
31+
final messages = List.generate(messageCount,
32+
(i) => eg.streamMessage(flags: [MessageFlag.read]));
33+
connection.prepare(json:
34+
newestResult(foundOldest: true, messages: messages).toJson());
35+
36+
await tester.pumpWidget(
37+
MaterialApp(
38+
home: GlobalStoreWidget(
39+
child: PerAccountStoreWidget(
40+
accountId: eg.selfAccount.id,
41+
child: const MessageListPage(narrow: AllMessagesNarrow())))));
42+
await tester.pumpAndSettle();
43+
return messages;
44+
}
45+
46+
testWidgets('_UnreadMarker animation performance test', (tester) async {
47+
// This integration test is meant for measuring performance.
48+
// See docs/integration_test.md for how to use it.
49+
50+
final messages = await setupMessageListPage(tester, 500);
51+
await binding.traceAction(() async {
52+
store.handleEvent(eg.updateMessageFlagsRemoveEvent(
53+
MessageFlag.read,
54+
messages));
55+
await tester.pumpAndSettle();
56+
store.handleEvent(UpdateMessageFlagsAddEvent(
57+
id: 1,
58+
flag: MessageFlag.read,
59+
messages: messages.map((e) => e.id).toList(),
60+
all: false));
61+
await tester.pumpAndSettle();
62+
});
63+
});
64+
}

0 commit comments

Comments
 (0)