Skip to content

[Linux] accessibility of widgets with text content #103191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jpnurmi opened this issue May 6, 2022 · 4 comments
Closed

[Linux] accessibility of widgets with text content #103191

jpnurmi opened this issue May 6, 2022 · 4 comments
Assignees
@jpnurmi
Labels
a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) a: desktop Running on desktop a: typography Text rendering, possibly libtxt found in release: 2.10 Found to occur in 2.10 found in release: 2.13 Found to occur in 2.13 framework flutter/packages/flutter repository. See also f: labels. has reproducible steps The issue has been confirmed reproducible and is ready to work on P2 Important issues not at the top of the work list platform-linux Building on or for Linux specifically r: solved Issue is closed as solved

Comments

@jpnurmi
Copy link
Member

jpnurmi commented May 6, 2022

The AtkText interface is missing implementation.

Steps to Reproduce

  1. Execute flutter run on the code sample
  2. Start Accerciser
  3. Inspect the "Text" accessibility interface of labels and fields

Expected results:
Flutter widgets report text content through the Text accessibility interface.

Actual results:
Flutter reports that the Text accessibility interface is implemented, but all values are missing.

Flutter sample 
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      theme: ThemeData(
        inputDecorationTheme: const InputDecorationTheme(
          border: OutlineInputBorder(),
        ),
      ),
      home: Scaffold(
        body: Padding(
          padding: const EdgeInsets.all(20),
          child: Column(
            crossAxisAlignment: CrossAxisAlignment.start,
            children: [
              const Text('Text', semanticsLabel: 'Text'),
              const SizedBox(height: 20),
              const SelectableText(
                'SelectableText',
                semanticsLabel: 'SelectableText',
              ),
              const SizedBox(height: 20),
              TextField(
                controller: TextEditingController(text: 'TextField'),
              ),
              const SizedBox(height: 20),
              TextField(
                readOnly: true,
                controller:
                    TextEditingController(text: 'TextField (read-only)'),
              ),
              const SizedBox(height: 20),
              TextField(
                obscureText: true,
                controller: TextEditingController(text: 'password'),
              ),
            ],
          ),
        ),
      ),
    );
  }
}
GTK sample 
<?xml version="1.0" encoding="UTF-8"?>
<!-- Generated with glade 3.38.2 -->
<interface>
  <requires lib="gtk+" version="3.24"/>
  <object class="GtkWindow">
    <property name="can-focus">False</property>
    <child>
      <object class="GtkBox">
        <property name="visible">True</property>
        <property name="can-focus">False</property>
        <property name="margin-start">20</property>
        <property name="margin-end">20</property>
        <property name="margin-top">20</property>
        <property name="margin-bottom">20</property>
        <property name="orientation">vertical</property>
        <property name="spacing">20</property>
        <child>
          <object class="GtkLabel">
            <property name="visible">True</property>
            <property name="can-focus">False</property>
            <property name="label" translatable="yes">GtkLabel</property>
            <property name="xalign">0</property>
          </object>
          <packing>
            <property name="expand">False</property>
            <property name="fill">True</property>
            <property name="position">0</property>
          </packing>
        </child>
        <child>
          <object class="GtkLabel">
            <property name="visible">True</property>
            <property name="can-focus">False</property>
            <property name="label" translatable="yes">GtkLabel (selectable)</property>
            <property name="selectable">True</property>
            <property name="xalign">0</property>
          </object>
          <packing>
            <property name="expand">False</property>
            <property name="fill">True</property>
            <property name="position">1</property>
          </packing>
        </child>
        <child>
          <object class="GtkEntry">
            <property name="visible">True</property>
            <property name="can-focus">True</property>
            <property name="text" translatable="yes">GtkEntry</property>
          </object>
          <packing>
            <property name="expand">False</property>
            <property name="fill">True</property>
            <property name="position">2</property>
          </packing>
        </child>
        <child>
          <object class="GtkEntry">
            <property name="visible">True</property>
            <property name="can-focus">True</property>
            <property name="editable">False</property>
            <property name="text" translatable="yes">GtkEntry (read-only)</property>
          </object>
          <packing>
            <property name="expand">False</property>
            <property name="fill">True</property>
            <property name="position">3</property>
          </packing>
        </child>
        <child>
          <object class="GtkEntry">
            <property name="visible">True</property>
            <property name="can-focus">True</property>
            <property name="visibility">False</property>
            <property name="text" translatable="yes">password</property>
          </object>
          <packing>
            <property name="expand">False</property>
            <property name="fill">True</property>
            <property name="position">4</property>
          </packing>
        </child>
      </object>
    </child>
  </object>
  <object class="GtkTextBuffer" id="textbuffer1">
    <property name="text" translatable="yes">GtkTextView</property>
  </object>
  <object class="GtkTextBuffer" id="textbuffer2">
    <property name="text" translatable="yes">GtkTextView (read-only)</property>
  </object>
</interface>
Flutter GTK
image image
image image
Flutter doctor 
[✓] Flutter (Channel master, 2.13.0-0.0.pre.940, on Ubuntu 22.04 LTS 5.15.0-27-generic, locale en_US.UTF-8)
    • Flutter version 2.13.0-0.0.pre.940 at /home/jpnurmi/Flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 8094263cb7 (3 hours ago), 2022-05-06 04:59:06 -0400
    • Engine revision eb4f476fa6
    • Dart version 2.18.0 (build 2.18.0-91.0.dev)
    • DevTools version 2.13.0

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /home/jpnurmi/Android/Sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_SDK_ROOT = /home/jpnurmi/Android/Sdk
    • Java binary at: /home/jpnurmi/Android/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[✓] Linux toolchain - develop for Linux desktop
    • Ubuntu clang version 14.0.0-1ubuntu1
    • cmake version 3.22.1
    • ninja version 1.10.1
    • pkg-config version 0.29.2

[✓] Android Studio (version 2020.3)
    • Android Studio at /home/jpnurmi/Android/android-studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] VS Code
    • VS Code at /snap/code/current
    • Flutter extension version 3.40.0

[✓] Connected device (1 available)
    • Linux (desktop) • linux • linux-x64 • Ubuntu 22.04 LTS 5.15.0-27-generic

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!
@maheshj01 maheshj01 added the in triage Presently being triaged by the triage team label May 6, 2022
@maheshj01
Copy link
Member

HI @jpnurmi, thanks for filing the issue. I am able to reproduce the issue on stable and the master channel. When inspecting the Text widget the text content is not returned.

image

flutter doctor -v (Linux) 
[✓] Flutter (Channel stable, 2.10.5, on Ubuntu 20.04.4 LTS 5.13.0-30-generic, locale en_GB.UTF-8)
    • Flutter version 2.10.5 at /home/mahesh/Development/stable
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 5464c5bac7 (3 weeks ago), 2022-04-18 09:55:37 -0700
    • Engine revision 57d3bac3dd
    • Dart version 2.16.2
    • DevTools version 2.9.2

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /home/mahesh/Android/Sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /home/mahesh/Downloads/android-studio-2020.3.1.25-linux/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[✓] Linux toolchain - develop for Linux desktop
    • clang version 10.0.0-4ubuntu1
    • cmake version 3.16.3
    • ninja version 1.10.0
    • pkg-config version 0.29.1

[✓] Android Studio (version 2020.3)
    • Android Studio at /home/mahesh/Downloads/android-studio-2020.3.1.25-linux/android-studio
    • Flutter plugin version 63.2.1
    • Dart plugin version 203.8452
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] VS Code (version 1.66.1)
    • VS Code at /usr/share/code
    • Flutter extension version 3.38.1

[✓] Connected device (2 available)
    • Linux (desktop) • linux  • linux-x64      • Ubuntu 20.04.4 LTS 5.13.0-30-generic
    • Chrome (web)    • chrome • web-javascript • Google Chrome 100.0.4896.75

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

[✓] Flutter (Channel master, 2.13.0-0.0.pre.942, on Ubuntu 20.04.4 LTS 5.13.0-30-generic, locale en_GB.UTF-8)
    • Flutter version 2.13.0-0.0.pre.942 at /home/mahesh/Development/master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 20fd55737d (3 hours ago), 2022-05-06 06:04:06 -0400
    • Engine revision a73b884510
    • Dart version 2.18.0 (build 2.18.0-91.0.dev)
    • DevTools version 2.13.0

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /home/mahesh/Android/Sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /home/mahesh/Downloads/android-studio-2020.3.1.25-linux/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[✓] Linux toolchain - develop for Linux desktop
    • clang version 10.0.0-4ubuntu1
    • cmake version 3.16.3
    • ninja version 1.10.0
    • pkg-config version 0.29.1

[✓] Android Studio (version 2020.3)
    • Android Studio at /home/mahesh/Downloads/android-studio-2020.3.1.25-linux/android-studio
    • Flutter plugin version 63.2.1
    • Dart plugin version 203.8452
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] VS Code (version 1.66.1)
    • VS Code at /usr/share/code
    • Flutter extension version 3.38.1

[✓] Connected device (2 available)
    • Linux (desktop) • linux  • linux-x64      • Ubuntu 20.04.4 LTS 5.13.0-30-generic
    • Chrome (web)    • chrome • web-javascript • Google Chrome 100.0.4896.75

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

@maheshj01 maheshj01 added a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) platform-linux Building on or for Linux specifically a: desktop Running on desktop has reproducible steps The issue has been confirmed reproducible and is ready to work on found in release: 2.10 Found to occur in 2.10 found in release: 2.13 Found to occur in 2.13 a: typography Text rendering, possibly libtxt framework flutter/packages/flutter repository. See also f: labels. and removed in triage Presently being triaged by the triage team labels May 6, 2022
@jpnurmi jpnurmi mentioned this issue May 6, 2022
Merged
8 tasks
@gspencergoog gspencergoog added the P2 Important issues not at the top of the work list label May 12, 2022
@jpnurmi
Copy link
Member Author

jpnurmi commented Jun 7, 2022

I only mentioned AtkText but AtkEditableText would be also important for text input.

GTK

a11y-gtk-entry.mp4

Flutter

a11y-flutter-text-field.mp4

jpnurmi added a commit to jpnurmi/engine that referenced this issue Jun 10, 2022
@jpnurmi
Add FlAccessibleTextField that implements AtkEditableText
d71dfa3 
Solves: flutter/flutter#103191
@jpnurmi jpnurmi mentioned this issue Jun 10, 2022
Merged
8 tasks
robert-ancell pushed a commit to flutter/engine that referenced this issue Jun 26, 2022
@jpnurmi
[Linux][A11y] implement AtkEditableText interface (#33955)
5ca80bd 
* Make FlAccessibleNode derivable

This allows introducing specialized subclasses that implement such a11y
interfaces as AtkEditableText that should not be implemented for the
generic node.

* FlAccessibleNode: add set value, text selection & perform action

Prepare API required for the upcoming AtkEditableText implementation.

* FlViewAccessible: postpone child node creation

This changes the a11y node creation flow so that (unknown) child nodes
are not pre-created when their parent is updated but "pending" child
node IDs are collected for later. Parent-child relationships are
established at the end of update batches when all nodes have been
created.

* Add FlAccessibleTextField that implements AtkEditableText

Solves: flutter/flutter#103191
@jpnurmi
Copy link
Member Author

jpnurmi commented Jun 27, 2022

AtkEditableText and AtkText have been implemented for text fields in #33955. Originally, I proposed implementing AtkText even for text labels but the docs suggest that AtkObject::name is more appropriate for short and simple text labels so I'm leaving it as is.

AtkText should be implemented by AtkObjects on behalf of widgets that have text content which is either attributed or otherwise non-trivial. AtkObjects whose text content is simple, unattributed, and very brief may expose that content via atk_object_get_name instead; however if the text is editable, multi-line, typically longer than three or four words, attributed, selectable, or if the object already uses the 'name' ATK property for other information, the AtkText interface should be used to expose the text content. In the case of editable text content, AtkEditableText (a subtype of the AtkText interface) should be implemented instead.

@jpnurmi jpnurmi closed this as completed Jun 27, 2022
@maheshj01 maheshj01 added the r: solved Issue is closed as solved label Jun 28, 2022
@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 12, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jpnurmi jpnurmi

Labels
a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) a: desktop Running on desktop a: typography Text rendering, possibly libtxt found in release: 2.10 Found to occur in 2.10 found in release: 2.13 Found to occur in 2.13 framework flutter/packages/flutter repository. See also f: labels. has reproducible steps The issue has been confirmed reproducible and is ready to work on P2 Important issues not at the top of the work list platform-linux Building on or for Linux specifically r: solved Issue is closed as solved
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jpnurmi @gspencergoog @maheshj01