Responsive localization

[zbraniecki]

Historically, localization was always predominantly a server-side, relatively static operation. Majority of (non-l10n) engineers I worked with usually start with a concept that l10n is a phase done as early as possible - for example during pre-processing, maybe build time, or on the server-side in a server-client model. Even when localization is performed at runtime, it's heavily optimized out and cached with an assumption that any invalidation will be performed by throwing out the whole cached UI and rebuilding it in a new locale.

Fluent, very early on its life-cycle, went the opposite way - localization is performed as late as possible.
Fluent can still be used to localize a template that is, say, sent to the client side from Django, PHP etc., but in the core use case for us - Firefox UI - we perform the localization as the final step of the pipeline, right before layout and rendering.

I'm bringing it up here because I believe that depending on how useful this group will find such use case, it may have an impact on how we design several features that impact lifecycle of the runtime localization.

It changes the Build -> Package? -> Run -> Translate -> Display -> Close model into Build -> Package? -> Run -> Translate -> Display -> Retranslate -> Redisplay -> Close.

Adding the retranslate has impact on how we, among others, think of fallback (we may start an app with 80% es-MX falling back to es, and during runtime we may update es-MX to 90% without restarting an app), on the dominant API we use for develpers (declarative vs imperative), and on some constrains on more rich features like DOM Overlays because we have to take into account that a DOM may be translated into one locale, and then we need a way to apply a new translation.
Instead of just working with state Untranslated -> Translated we also have a state Translated -> Retranslated where one locale may have added/removed/modified some bits of the widget.

This late model has several advantages that were crucial to us, which I'd like to present:

1. it allows developers to work with declarative, state-full UI, updating the state as they go, without worrying about synchronous or asynchronous manner in which the actual localization is going to be applied.

To illustrate the difference, in one component (Firefox Preferences) with ~1000 strings which we migrated from the previous l10n API to Fluent, we reduced the number of imperative calls by 10x.

Instead of developers writing:

let value = Services.strings.getFormattedValue("processCount", {count: 5});
let accesskey = Services.strings.getFormattedValue("processCount-accesskey");
document.getElementById("process_count").textContent = value;
document.getElementById("process_count").setAttribute("accesskey", accesskey);

they now write:

document.l10n.setAttributes(
  document.getElementById("process_count"),
  "processCount",
  {count: 5}
);

setAttributes is a very simple DOM function which sets two attributes: data-l10n-id and data-l10n-args.

Separately, we have a MutationObserver which reacts to that change by adding the element to a pool of strings to be translated in the next animation frame, and then performs the localization.

From the developer perspective, they just set the state of DOM, and its out of their concern how and when the translation will happen.

For our discussed use case on the other hand, the value is that we always have a DOM tree available with all the information needed to apply new translation - we just need to traverse the DOM, find all data-l10n-id/data-l10n-args and translate them to a new locale.

Here you can see a very old demo of that feature combined with dynamic language packs.
Many UI toolkits try to emulate such feature by preserving state and rebuilding the UI and reconnecting the state, but FluentDOM allows you to just update the DOM on fly without ever touching the state (we can update your translation while you interact with the UI!).

This feature is already fully implemented in Firefox Desktop now, and we can change translation on fly for the subset of our UI that we already migrated to Fluent.

2. Runtime pseudolocalization

A natural side-effect of the above is that we can pseudolocalize on fly, at runtime, by pushing all translations via a transform(String) -> String function and applying them to DOM.

This means that shipping pseudolocalization has no cost on binary size (reason why Android avoids shipping pseudolocales to production) and one can provide many, even customizable, pseudolocalization strategies (for example adjustable length increase to stress test layout).

Here's a demo of that feature.

3. Runtime caching

Caching is still possible (we load untranslated UI, apply translation, cache, then load from cache unless locale changed), and its invalidation just becomes part of the translate -> retranslate state which also simplifies things.

4. Actual responsive l10n

This is a feature we prototyped, but never got to actually implement, which I see as one of the potential "north stars" - features we may not implement ourselves, but may want to make the outcome of our work be able to be build on top of.

The idea behind it is to use Fluent syntax to provide reactive retranslation on external conditions.
A common idea we wanted to tackle was a scenario where the UI operates in adjustable available space.
Imagine a UI which may be displayed on TV, Laptop, Tablet of Phone.
For the large space, we'd like to use a long string, but when space shrinks, we'd like to display a shorter version of the same message rather than cut out with ellipses.
What's more, different locales may face different challenges - while German may struggle to fit the full text even on large screen, Chinese won't likely need the large version at all.

Since the experience is per-locale and the condition of variant selection is per-locale, we wanted to use Fluent for it, more or less like so:

prefs-cursor-keys-option = { SCREEN_WITH() ->
    [wide] Always use the cursor keys to navigate within pages.
    [medium] Use the cursor keys to navigate in pages.
   *[narrow] Use keys to navigate in pages.

This allows an English translation to adjust the width of the message to available space.

What was the real goal was also ability to interpret the message at runtime by FluentDOM and hook onScreenSizeChange event handler to retranslate the message.
This handle will be hooked only if the locale actually depends on SCREEN_WIDTH.

We never put this feature in production but we validated that Fluent data model and API makes this possible.

Here's a demo.

====

Such flexibility may be seen as very high level, and I'd say that 90% of work to make such features work are.
But there's 10% of work that depends on how low-level data model is designed - how fallback works, how interpolation works, what I/O is possible.

I'd like to put this proposal in front of this group as a feature that we'd like to make sure our outcome doesn't make impossible.

[stasm]

We implemented or prototyped these in Fluent together, and while I'm on board with the ideas you presented, I think they are out of scope for the standard itself. Instead, I'd like the standard to enable a wide variety of userland implementations, including the late runtime retranslation model you described.

Ultimately, this goes back to the goals/non-goals discussion in #59. In my opinion, designing the standard on the lower level of abstraction can have the benefit of making it agnostic to how it's then integrated into the consumer code. This also includes preferring imperative APIs over declarative ones, for most part, as they tend to be more flexible as building blocks.

The responsive model you describe, in particular, requires many additional considerations: binding translations to UI elements, observing changes to variables, notifying about updates, scheduling updates, and more. Furthermore, some of these questions are already answered in some capacity by the existing front-end frameworks. If the standard is opinionated in these areas, it might be hard to integrate in existing software.

I know that the alternative to this low-level, agnostic, imperative, pluggable model which I'm advocating for is the "CSS for localization" model. I think it has a lot of merits and perhaps it could make a good extension to the standard in the future. It's good to have these ideas written down and documented here; thanks for taking the time to file this issue.

[zbraniecki]

Yep! I agree, in particular:

Instead, I'd like the standard to enable a wide variety of userland implementations,
including the late runtime retranslation model you described.

I dumped this use case here as a fairly extreme "moonshot" style feature that could be very appealing, and could be used as a mental test of "will what we do enable that".

I don't think we should aim to bring this feature into this standard :)

[aphillips]

@zbraniecki Should we keep this issue open? If so, can you reformulate into a specific request against the syntax/spec? Thanks!

[aphillips]

Closing per discussion in 2023-06-19 telecon. @zbraniecki feel free to create new requests.