Skip to content

Extend testdriver.js and wptrunner to test AAMs (platform accessibility APIs) #53733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

spectranaut
Copy link
Contributor

This PR adds an extension to testdriver.js in order to test accessibility API, as defined by the ARIA, Core-AAM and HTML-AAM specifications. The RFC to extend testdriver.js in this way can be found here. The decision to open a PR with these changes was made at TPAC 2024.

The change in this CL:

  1. Tests in the core-aam directory: These tests are marked as tentative because some work may need to be done on the design of the API endpoint, and some technical problems may need to be solve in order for them to run in wpt.fyi and in browser CIs.
  2. Changes to wptrunner:
    a. The product name ("chrome", "firefox", etc) is passed back to the executor level. This is necessary because accessibility APIs are exposed externally to the browser, and we need to find the browser in the accessibility API by name. Ideally, this will be done by PID (to allow for parallelization), but that requires some changes to chromedriver and safaridriver.
    b. The option --force-renderer-accessibility has been added when running wpt run against chrome.
  3. executorplatformaccessibility.py and tools/wptrunner/wptrunner/executors/platformaccessibility: these executors run tests against platform specific accessibility APIs, so there is an executor for Linux, Windows, and macOS. The Windows and macOS ones are proof of concepts and will need more work. The Linux one is fully featured and can run all of the tests added in this PR.

To run tests on Linux:

# Chrome
./wpt run --force-renderer-accessibility --log-mach-level debug --log-mach - chrome core-aam/role/blockquote.tentative.html

# Chromium
./wpt run --binary <chromiumbinary> --force-renderer-accessibility --log-mach-level debug --log-mach - chromium core-aam/role/blockquote.tentative.html

# Firefox
# To run against Firefox, you need accessibility turned on. You can run on the Orca screen reader or `export GNOME_ACCESSIBILITY=1` and use `--no-headless`.
./wpt run --no-headless --log-mach-level debug --log-mach - firefox core-aam/role/blockquote.tentative.html

Notes about macOS and Windows

On Windows and macOS, even with the following dependencies, these tests will fail with the message "Fail: Test not implemented".

Thanks to @alice for the macOS code and thanks @jcsteh, the code to get IA2 interface was very inspired by the firefox test infrastructure: https://searchfox.org/mozilla-central/source/accessible/tests/browser/windows/a11y_setup.py

macOS dependencies:

python -m pip install pyobjc-framework-Accessibility pyobjc-framework-ApplicationServices

To run against Firefox, you need to turn on VoiceOver the screen reader.
To run against Chrome, you need to include --no-headless.

Windows dependencies:

Requirements: comtypes python library

pip install comtypes

@jcsteh
Copy link
Contributor

jcsteh commented Jul 14, 2025

# To run against Firefox, you need accessibility turned on. You can run on the Orca screen reader or `export GNOME_ACCESSIBILITY=1` and use `--no-headless`.

I wonder if we can make the name --force-renderer-accessibility more generic (e.g. --enable-accessibility or --force-accessibility) and implement it for various browsers as required? For Firefox, we can use the pref accessibility.force_disabled=-1 (I know, it's horrible, I'm sorry). I'm not quite sure where we'd wire that up, but I see that we do set extra prefs here, so we have the capability to do that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-aam infra testdriver.js wg-wai_aria wpt wptrunner The automated test runner, commonly called through ./wpt run
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants