Store some user data locally

[gnprice]

We'll need to store some of the user's data in a local database on the device.

The minimum version of this for a fully-functioning Zulip client would be:

• Authentication credentials for each of the user's accounts (needed for Add a login flow #11)
• Some other per-account data: the state of registering for push notifications; the server's Zulip version and feature level; the user ID
• User settings that are global to the device (like dark mode vs. light mode), or that otherwise don't come from the server

In particular, that covers everything the existing RN-based app stores and doesn't discard when you switch between accounts.

A future version would also store the full server data: other users with their names and avatars, streams with their names and subscription colors, recent/unread messages, and so on. But we can go a long way without that — the RN-based app has never made great use of this anyway. So we'll file a separate issue for that later.

The scope of this issue is to have local storage in some reasonable form that we think could be good to use in a production app, and to have some data we store there that can change and is reflected in the UI. We'll then add particular pieces of data to it as needed.

[gnprice]

We'll use SQLite for the database. There are a couple of NoSQL databases written in Dart that have some popularity in the Flutter world: Isar, and its predecessor Hive. But:

• I don't want to trust someone's hip new database design. SQLite is standard, ubiquitous, and has a famously rigorous test suite.
• I want to be able to access the data from platform-native code. We might not end up needing that capability — it's likely we can be fine with just doing all the application logic from Dart, including things like presenting notifications — but if we find we do, I don't want to be stuck.

Then the next question is what library to use to make it convenient to access SQLite from Dart in a Flutter app. From a bit of searching, there are several options:

sqflite

Venerable and popular.

The interface sqflite provides is quite reminiscent of the Android SQLite library — probably inspired by it. It provides a SQL query builder, with good parametrization support (avoiding SQL injection), but no ORM-style layer. Instead the suggested pattern is to basically write your model classes by hand, with accessors and mutators that use the query builder.

The migration support is also very similar to the Android SQLite library, which means migrations are fairly manual: see upstream example.

This option is probably a reliable one as far as it goes, but leaving a lot to be done manually, much of that fairly error-prone.

Floor

The Android people are well aware that the Android SQLite library leaves a lot of room for useful higher-level abstractions. Their answer to that, introduced several years ago, is the Room persistence library. It offers an ORM-like layer, with compile-time verification of your SQL queries, and a migrations system that supports better-abstracted and more-testable migrations. It seems pretty good.

Floor is explicitly inspired by Room, which makes it an appealing idea. Under the hood, it uses sqflite.

Unfortunately, it gives me the feeling that it's incomplete and not actively maintained. For example here's the docs on migrations:
https://pinchbv.github.io/floor/migrations/
Two paragraphs and a short example. Much of the docs is similarly thin.

On the "not maintained" side, the issue tracker is at https://github.com/pinchbv/floor/issues; see PR 638. Or even before that, issue 577, noting in 2021 that it seemed unmaintained. Or issue 603, which seems a pretty core bug and has no reply from the author/maintainer since it was filed in 2021.

Drift

This is also atop SQLite (though with the same authors' sqlite3 rather than sqflite). It provides an ORM-style query builder, which it says is type-safe.

The migration support is less appealing than Floor's (or at least Room's) — it's a lot closer to the manual structure of sqflite:
https://drift.simonbinder.eu/docs/advanced-features/migrations/
But at least you don't have to write unvalidated raw SQL.

The issue tracker is quite encouraging — it seems to be actively and conscientiously maintained.

• There's no stale-bot in sight. Issues are closed because the maintainer replied, and either fixed the issue or explained why it was invalid or duplicate or wontfix etc.
• The great majority of the top-voted issues are closed. Spot-checking the top five of those, they all look like legit closes.
• The great majority of recent issues, and of all issues, are closed. (And not by stale-bot.)

---

Conclusion

I plan to try Drift first.

If that's unsatisfying, I may try Floor. If we went down to the level of sqflite, I suspect I would end up building something akin to parts of Room on top of it anyway — so even though Floor is incomplete, if it's of reasonable quality then starting from there and adding/fixing things as needed may give a head start.

One thing I'll be looking for with Drift, and with Floor if I try that, is that the underlying SQLite schemas are reasonably clean, so that accessing them through a plainer SQLite binding wouldn't be materially worse than it would be for a hand-written schema. We'll want that in order to access the data from platform-native code, and it will also help keep the door open to switching later from Floor to Room or something else if desired.

[gnprice]

Flutter has a "happy paths" recommendation:

Structured local storage increases app performance and improves the user experience by selectively saving expensive or slow data on a user’s device. This path suggests two plugins for local persistence: drift and hive. Which plugin you choose depends on your needs.

Drift, rated a Flutter favorite, offers a fully-typed object relational mapping (ORM) around SQLite, with support on all Flutter platforms. Developers who require a fully relational database on their users’ device will benefit most from this package.

Hive offers a fully-typed object document mapping (ODM) around a custom storage solution, with support on all Flutter platforms. Developers who do not require a fully relational database, especially if they use document-based storage on their server (like Cloud Firestore) will benefit most from this package.

Doesn't change any conclusions, I think. There's also a video about Drift, which I haven't watched.

Meanwhile I have a draft PR at #22 to use Drift; and I've been making several other changes to prepare for storing data:

• Start writing unit tests #17
• Reorganize data-managing widgets to accommodate global data #20
• Make API logic testable, and start testing some #23

More to come.