Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Add Spell Check Support for Android Engine #30858

Merged
merged 72 commits into from
May 16, 2022

Conversation

camsim99
Copy link
Contributor

@camsim99 camsim99 commented Jan 13, 2022

Adds engine portion of Issue #34688 for Android.

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
  • I listed at least one issue that this PR fixes in the description above.
  • I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on
    writing and running engine tests.
  • I updated/added relevant documentation (doc comments with ///).
  • I signed the CLA.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@camsim99 camsim99 marked this pull request as ready for review April 1, 2022 23:24
@camsim99 camsim99 requested review from justinmc and blasten April 1, 2022 23:24
@skia-gold
Copy link

Gold has detected about 25 new digest(s) on patchset 63.
View them at https://flutter-engine-gold.skia.org/cl/github/30858


@Override
public void initiateSpellCheck(@NonNull String locale, @NonNull String text) {
performSpellCheck(locale, text);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle case when there's a pending initiateSpellCheck. Maybe, call result.error or result.success with an empty list. The general idea is that while the framework is waiting on a spellcheck request, a new request is issued, so the older one needs to be dropped.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach of dropping older requests has a similar fault to the approach I pursued previously of saving the result and text awaiting a response to that result in the SpellCheckChannel class, which is that we cannot cancel requests to the Android spell checker once they have been submitted.

We already knew that, but our assumption was that if the SpellCheckPlugin has detected a result awaiting a response when another request to initiate spell check has come in, then that request will not reach the onGetSentenceSuggestions(...) callback, meaning that the Android spell checker consumed the request. The consequence of assuming such is that sometimes the plugin will attempt to respond to a new request with old spell check results. Depending on how much the text has been modified between the time those old results come in and the time that the new request to spell check was submitted, those old results could lead to some hard-to-handle errors on the framework side.

Given all of that, I decided to have the plugin only handle one request at a time by dropping new requests when an old one is still pending. This approach may lead to results lagging slightly behind, but (i) the spell check results in any given response will correspond to the text in that response, (ii) the spell check results will be returned in chronological order, and (iii) any impacts of these results lagging behind can be handled by the logic on the framework side. This is all because with this approach, we do not have to battle the unpredictable behavior of the Android spell checker. I believe also that in further iterations, other speedup techniques can be more easily explored given these guarantees.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this approach seem to work with decent performance in practice? Maybe this is the most solid approach in the real world even though it's not the fastest theoretically.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, is there ever a risk of the spell checker never responding and locking up or is that another thing that we shouldn't worry about in practice?

Copy link
Contributor Author

@camsim99 camsim99 May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this approach seem to work with decent performance in practice? Maybe this is the most solid approach in the real world even though it's not the fastest theoretically.

Yes, exactly. From playing around with it (paired with making the corrections on the framework side if the process of returning results lag behind), I don't see any difference. If you are open to playing around with it, too, that'd be great!

Also, is there ever a risk of the spell checker never responding and locking up or is that another thing that we shouldn't worry about in practice?

I think we can rely on Android for this. Android seems to have some handling for issues like that (see here).

Copy link

@blasten blasten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ready to approve if the "one at a time" approach has been QA'd and seems to work correctly and fast enough in practice. And maybe it should be tested here. Otherwise everything looks great.

Also I think there's a legitimate formatting error on the Linux Unopt test: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8814609902934628049/+/u/test:_format_and_dart_test/stdout


@Override
public void initiateSpellCheck(@NonNull String locale, @NonNull String text) {
performSpellCheck(locale, text);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this approach seem to work with decent performance in practice? Maybe this is the most solid approach in the real world even though it's not the fastest theoretically.


@Override
public void initiateSpellCheck(@NonNull String locale, @NonNull String text) {
performSpellCheck(locale, text);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, is there ever a risk of the spell checker never responding and locking up or is that another thing that we shouldn't worry about in practice?

import org.junit.runner.RunWith;
import org.mockito.ArgumentCaptor;

// TODO(camillesimon): Fix theses tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are they still broken?

Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

I tried this out on a bunch of different emulators and one physical device, and with some logging in the engine I'm convinced that there's no major problem with this PR. The feel of the "one at a time" approach seems fine to me in practice. It seems lots of spell checkers at different API levels do different things, which can lead to strange behavior. Later we may want to add some code in the framework that normalizes or works around some of this weirdness, or just stop support for those devices. But I think this PR can be merged now as-is.

@camsim99 camsim99 added the waiting for tree to go green This PR is approved and tested, but waiting for the tree to be green to land. label May 16, 2022
@fluttergithubbot fluttergithubbot merged commit 05ddf05 into flutter:main May 16, 2022
camsim99 added a commit that referenced this pull request May 16, 2022
@camsim99 camsim99 mentioned this pull request May 16, 2022
8 tasks
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request May 16, 2022
zanderso pushed a commit to flutter/flutter that referenced this pull request May 16, 2022
* 05ddf05 Add Spell Check Support for Android Engine (flutter/engine#30858)

* a277fe2 Roll Fuchsia Linux SDK from sg5JUw-y-... to F0_B8WVKY... (flutter/engine#33395)
houhuayong pushed a commit to houhuayong/engine that referenced this pull request Jun 21, 2022
* {@link SpellCheckChannel}.
*
* <p>Spell check results will be encoded as a string representing the span of that result, with
* the format "start_index.end_index.suggestion_1/nsuggestion_2/nsuggestion_3", where there may be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the code, it is \n, not /n :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out! I need to make another engine PR to remove my usage of JSONMethodCodec, so I'll fix this there.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
platform-android platform-web Code specifically for the web engine waiting for tree to go green This PR is approved and tested, but waiting for the tree to be green to land.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants