Repository navigation
fix(apple-runner): fill only the input a tap at the point reaches - #4616
miospotdevteam wants to merge 5 commits into
Conversation
A sheet or overlay can cover a text field that stays in the accessibility tree. When a fill's point lies on both fields, the point lookup ordered inputs smallest first with no occlusion check, so it picked the covered field. The focus tap still reached the sheet's field and raised the keyboard, the keyboard counted as focus evidence (or, already up, skipped the check), and the clear, typing and read-back went to the covered field: XCTest reported that it had no keyboard focus, or the read-back mismatched. - The point lookup puts inputs XCTest reports hittable ahead of covered ones when more than one input answers the point. - When more than one input answers the point, the iOS focus step accepts no keyboard evidence: the tapped input must report keyboard focus with its identity unchanged, and the target binds that identity for the clear, the typing and the read-back. Without that report it types nothing and answers TEXT_INPUT_NOT_FOCUSED. - A point with a single input keeps its current focus path, so secure fields and moved or replaced inputs behave as before. Adds an overlapping-sheet fixture and runner tests for the keyboard hidden and already shown, plain and with an 80 ms delay, and a sheet field that refuses focus.
When a fill's point lies on more than one input, both routes chose the target without proving that a tap at that point reached it: the XCTest route ordered candidates by size and isHittable (an input's own hit point, not the caller's), and the synthesized route bound the probe's first candidate and sent its clear to whichever field held focus. Both now tap at the caller's point and accept only the input that took keyboard focus from it (or an already focused input that is the only uncovered one there), before any clear or typing; anything else is TEXT_INPUT_NOT_FOCUSED with nothing typed. New fixtures cover a popover that leaves the covered field's centre exposed and a sheet over a focused, populated neighbour.
There was a problem hiding this comment.
All reported issues were addressed across 7 files
You’re at about 98% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Reply to a comment to ask cubic a question or push back. It learns from your replies.
View guided diff | Re-trigger cubic
…ized route The synthesized coordinate route discarded the read that confirmed the tapped input took focus and queried the input again before clearing. Pass it on as the XCTest route does.
|
I found no blocking problem in e64804c, but the iOS evidence is not yet recorded well enough to merge on. The new tests run only in the nightly iOS lane, so the checks on this head show nothing for the changed route (no checks are reported). No conflicts. Before merge, please add the device and runtime record to the PR body (details below) and decide on the Not blocking, and you can take or leave each one: (1) TextInputProbe.swift:212 now runs Simplicity question: could The inline thread on the synthesized route is fixed at this head: #4616 (comment) ( |
|
Thanks for the careful read. I am taking the simplification: |
…l reaches - The point lookup no longer puts hittable inputs first: it is back to smallest-first, so tap booking, read at a point and refresh resolution behave as before. - An input that takes focus from the tap wins, even when its focus lands late. An input that already had focus counts only when no other input took focus within the wait and it is then the one uncovered input there. - When synthesis stops after the clear, the fallback reuses the bound input while it still has keyboard focus instead of resolving the point again.
|
Thanks, that plan answers the simplicity question: one owner for which input a tap reaches, with the reorder gone. I will review the new head once it is pushed, together with the device and runtime record. |
…essibility A plain view narrower than the field left its corners exposed and is not an accessibility element, so the field still read hittable. A labelled UILabel past every edge covers all the points the app reports from, while taking no touches.
|
Pushed
|
|
This PR is ready at cb1f1af. The post-tap read is now carried on the synthesized route and on the new fallback, so the earlier concern from e64804c is fixed (#4616 (comment), which you can resolve). The checks show no failures. No checks are reported on this cross-repo head, and the changed route is iOS-runner-only, so CI shows nothing for it either way. There are no conflicts. Not blocking, and you can take or leave it: the promised CLI The device record, the 45/0/0 result and the e64804c 0/2 run come from the PR body. Accepting an already-focused input at an overlapping point now always waits the full 2 s readiness timeout. The PR documents this as a trade-off, and it is not measured. The next step is the maintainer merge decision. |
|
Thanks. You're right about the checkout CLI; I corrected the body: the CLI run there is before-evidence only, and the fix's evidence is the XCUITest run. I'll leave the checkout |
Summary
On iOS, a sheet or popover can cover a text field that stays in the accessibility tree. When a
fillpoint lay on both, the runner picked a target without proving the tap reached it: the point lookup ordered inputs by size, and the keyboard counted as focus. The clear, typing and read-back then went to the covered field, or the clear went to whichever field already had focus.filltaps that point andtextInputTheTapReachedalone decides which input the tap reached: the first candidate that newly takes keyboard focus within the readiness wait (2 s). An input that already held focus counts as reached only when the full wait finds no newly focused candidate and it is the only uncovered candidate there. The chosen input stays bound through clear, typing and read-back, including the synthesized route's fallback when synthesis stops early, which reuses the bound input while it still holds focus.TEXT_INPUT_NOT_FOCUSED.queryTextInputs) keep the basesmallestElementFirstorder.Trade-offs:
keyboard dismiss) and retry.Files: 4 runner sources, the fixture app, runner tests,
commands.md. No linked issue.Validation
Base
ea414582b, headcb1f1af81, compared with the previous heade64804c37.Simulator: iPhone 17, arm64, iOS 27.0 (24A434). Xcode 27.0 (27A266a). Generic simulator build, then
test-without-buildingon one device, parallel testing off.Test filter (45 tests)
cb1f1af81, 45 testsXCTExpectFailureself-testtestTextInputProbeContainmentExcludesRequiredReadsAndLaterIssues)e64804c37with this head's fixture and test file, the two new review testsOn
e64804c37the two review tests fail before their value assertions:testFillTypesIntoThePopoverFieldThatTakesFocusLateNotTheFieldThatHeldFocus: the old code accepts the field that held focus on the first poll and dispatches the clear to it; XCTest then refuses because the popover field has taken focus:Failed to synthesize event: Neither element nor any descendant has keyboard focus. Event dispatch snapshot: TextField, {{121.0, 294.0}, {160.0, 34.0}}, identifier: 'agent-device-popover-partly-covered-input', value: background.testFillWhoseSynthesisStopsAfterTheClearTypesIntoTheBoundPopoverField: the old fallback re-resolves by point after the clear and refuses:caught error: "TextEntryFocusNotConfirmed()".testFillTypesIntoTheFocusedSheetInputWhenTheTapMovesNoFocuspasses on this head. The late-focus fixture's shape check (popoverField.isHittable == false) passed on both heads.Before-evidence from the released CLI on the overlapping-sheet fixture:
fillat the shared point exited 1 withTEXT_ENTRY_MISMATCH, expected"sheetvalue", observed"background"; the sheet field stayedstaleand the covered field stayedbackground. No checkout CLI run from this head was made, so that run is evidence of the bug only, not of this fix; the fix's evidence is the XCUITest run above.Other checks at this head:
pnpm check:quick: passes.pnpm check:xctest-selection: passes. Output:xctest selection: 746 declared AgentDeviceRunnerUITests methods — host lane (.github/actions/smoke-macos/action.yml, macOS, every PR) reaches 440, PR iOS branch lane (.github/actions/smoke-ios-run/action.yml, iOS Simulator, selected PRs and main) selects 6, nightly (.github/workflows/xctest-nightly.yml, iOS Simulator) reaches 737, its iPad leg selects 4; 0 reachable by no lane; RunnerTests/testCommand skipped everywhere.pnpm build:xcuitest:iosandpnpm build:xcuitest:macos: succeed.CI: the new tests are iOS-only, so only the nightly iOS lane runs them.
🤖 Generated with Claude Code