Skip to content

feat(android): publish clickable, focusable, long-clickable and scrollable on snapshot nodes - #4633

Draft
dannyhw wants to merge 2 commits into
callstack:mainfrom
dannyhw:snapshot-node-interaction-flags
Draft

dannyhw wants to merge 2 commits into
callstack:mainfrom
dannyhw:snapshot-node-interaction-flags

Conversation

@dannyhw

@dannyhw dannyhw commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

What

Android snapshot nodes publish four accessibility facts next to hittable:

Why

hittable is true for a node that is clickable, focusable or focused. That is the right question for driving a screen, and it hides a difference an accessibility check needs: a node a screen reader can land on but cannot activate is hittable, and so is one it can activate.

My case is a suite that tests native controls hosted in a React Native app (Compose inside React Native views). It asserts things like "the node carrying the control's name is the clickable one" and "a control with a long-press menu offers a long click". Today those few assertions need adb shell uiautomator dump, which cannot run while the helper session holds the accessibility connection, so they live in a separate pass after the suite.

From a build of this branch, on an API 35 emulator (snapshot --json --raw):

"Message from Grace"   hittable: true   clickable: false  focusable: true   longClickable: true
a plain container                        clickable: false  focusable: false  longClickable: false
ScrollView             hittable: true   clickable: false  focusable: true   scrollable: true

The first line is the case hittable cannot tell apart from a button.

The part I am unsure about

snapshot-private-evidence.ts says clickability is deliberately kept outside SnapshotNode, and two tests in snapshot-clickability.test.ts asserted that clickable never appears on a published node. This change reverses that, and I changed those two tests to match. If you would rather keep the default node as it is, I am happy to put these behind an opt-in capture option (the way --actions works), or publish them on --raw captures only. Tell me which you prefer.

It also adds four booleans to every Android node in --json output. The text snapshot is unchanged.

State

  • pnpm lint and pnpm typecheck pass. The platform-android and capture unit tests pass (1,548).
  • Not done yet: the full check:affected run, a test in the helper's own suite for the new attribute, docs.

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

View guided diff

…lable on snapshot nodes

A snapshot node carries hittable, which is true for a node that is clickable,
focusable or focused. A reader that audits what an accessibility service is
offered needs those apart: a node a screen reader can land on but cannot
activate is hittable, and so is one it can activate.

The Android reader already parses clickable, focusable and scrollable; this
publishes them on the node. The snapshot helper now writes long-clickable,
with both answers so a capture from an older helper is not read as a node
that offers no long click.
@thymikee

Copy link
Copy Markdown
Member

The code change at d56b15a looks reasonable, but it has no live device evidence yet, so it needs one recorded run before merge.

The PR changes the device-facing helper (a new XML attribute on every node) and the Android snapshot projection. The PR body has only three pasted node lines from an unnamed build, and its State section says the full check:affected, a helper-suite test and docs are not done. Nothing shows that the rebuilt helper serializes long-clickable on a device, that the default snapshot -i --json route publishes the four fields, or that an older installed helper degrades cleanly. Please run pnpm build:android, pnpm build and pnpm clean:daemon, then on an API 35 emulator capture snapshot -i --json and snapshot --json --raw at d56b15a (see AccessibilityTreeXml.java:64). Record the device, the source SHA, androidSnapshot.backend: android-helper with helperVersion equal to package.json, and one node each for clickable true, long-clickable true and false, and scrollable true. Add one capture with the previously installed helper APK that shows longClickable absent (not false) and no error.

The four new keys are also written as explicit booleans on every node in the default projection, not only raw, at ui-hierarchy-builder.ts:481. Is that intended? snapshot --json and MCP/AI-SDK results gain 3-4 mostly-false keys per node, which is tens of KB on a 500-node screen. Could the four facts be published only from the raw projection, carried through AndroidPassiveNodeFacts so the builder gains no lines, which keeps default snapshot --json byte-identical? If you want them by default, publishing only the true values (omitting false, as heading does) keeps absent cheap. This needs a maintainer decision first, recorded by amending the "deliberately kept outside SnapshotNode" rule in snapshot-private-evidence.ts. After that, please check whether Android's private clickableByNodeIndex map is still needed once the node carries clickable.

Not blocking, and fine to take or leave: longClickable is a pure passive fact, so it fits in AndroidPassiveNodeFacts (spread at ui-hierarchy-builder.ts:473) instead of a new top-level field and a conditional spread, which is unneeded because undefined keys already drop out; two comments are now false (android.ts:48 says the helper serializes no long-clickable, and snapshot-private-evidence.ts:5-8 says clickability is never serialized into the snapshot contract); once the publication choice is made, snapshots.md needs one bullet per published field (absent means unavailable) and SELECTOR_DIGEST_NODE_FIELDS needs a decision on the four keys, since digest-level get attrs omits them while default level carries them; and both changed tests run with { raw: true }, so a non-raw case (one clickable row, one fully clipped row) and a helper unit assertion for long-clickable true/false would cover the default route.

The failing check is oxlint . --deny-warnings on scripts/layering/package-boundaries.test.ts:1001 ("File has too many lines (1001)"). The PR does not touch scripts/, and that file has 1000 lines at the PR head and 1001 on main fce2fa6, so this failure looks unrelated to the PR. Typecheck and formatting passed. There are no conflicts. Before merge, we need the maintainer decision on default, raw-only or opt-in publication, and then the recorded helper run at the head SHA.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants