Repository navigation
Conversation
Size Report
Startup median (7 runs, lower is better):
|
Apex Sentinel · ✅ ApproveSince the last review: 0 new · 0 still open · 0 no longer raised No open findings. Not in the reviewers' prompt (6 files)
Reviewed with this repository's own Sentinel configuration ( 💬 Talking to SentinelIn a finding's thread, reply with:
On this conversation, comment Earlier reviewsReview of
|
There was a problem hiding this comment.
All reported issues were addressed across 6 files
You’re at about 96% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
35230e2 to
59b2272
Compare
59b2272 to
aadf30b
Compare
|
I found nothing that blocks this change at 59b2272. All 17 checks are green and there are no conflicts. The delta is read against the base head 2e90b57, since the audit code lives only in open PR #4592. Not blocking, take or leave: the audit line in skills/agent-device/SKILL.md:18 has no support or trigger condition, and Vega has no snapshot (commands.md:340), so you could prefix it with "When the platform supports snapshot, and after a screen change,". The sentence in skills/dogfood/SKILL.md:27 holds two instructions, so you could split it, and do the same for the first line of Platform checks in the help. The comment at scripts/help-conformance-cases.mjs:149 says audit on the initial screen, but the regex matches an audit line anywhere in the plan, so "plans an audit" would be accurate. The open review thread on the missing Vega condition (#4600 (comment)) still apply, as do the three lower-priority threads on the same notes. The thread at #4600 (comment) does not apply: audit keeps the ref frame and issues only a partial publication, so snapshot refs stay valid after it. You can resolve that one. This can merge after base #4592 lands. |
|
This PR is ready at aadf30b: I found no problems in the code, and all 17 checks are green. There are no conflicts. Not blocking, and you can take or leave these: the audit line at https://github.com/callstack/agent-device/blob/aadf30b/skills/agent-device/SKILL.md#L18 (and the matching clause in agent-setup.md) tells every app-driving task to run audit, while the sibling lines say a normal task should start immediately, and the issue scopes the audit nudge to help dogfood, so you could drop it or point it at help dogfood; the matcher at https://github.com/callstack/agent-device/blob/aadf30b/scripts/help-conformance-cases.mjs#L151 only matches |
aadf30b to
a4ba7fc
Compare
|
Took the first two, kept the third pair.
Kept as is: the Bench after both changes: |
Summary
help dogfoodnow has aPlatform checks:block after Goal/Loop: on the first screen and after a screen changes meaningfully, runauditfirst and thensnapshot -i(audit performs its own capture, so it goes first), act on the snapshot refs or the pinned refs audit prints, and report its findings as accessibility findings; on iOS test Back and the edge-back swipe where a screen was pushed; on Android test system Back and applicable sheet dismissal; probe a dismissible sheet's unchanged swipe-down (expect dismiss), edited dismissal (expect confirm or save), and explicit Cancel; check destructive confirmation or undo, swipe-action alternatives, and each field's return key against its task; exercise each control once with reversible test data. One line carries the exceptions: autosave, explicit discard semantics, mandatory flows, nondismissible presentations. It links the Apple Human Interface Guidelines and Material accessibility guidance and does not restate whatauditalready checks (target size, names).The new command is discoverable outside that topic: one line each in
skills/agent-device/SKILL.mdandskills/dogfood/SKILL.md, and the recommended agent rule inwebsite/docs/docs/agent-setup.mdnow mentionsaudit. Both routing lines send an agent toagent-device help dogfoodfirst, so a normal app-driving task still starts immediately and only QA or accessibility work takes the audit pass. The dogfood skill line carries the trigger (first screen, and after a screen changes meaningfully) and applies only when the target supports snapshots, so snapshotless targets (Vega) skip it.Stacked on #4592.
Evidence
agent-device help dogfoodrenders the block on lines 18-25, inside the first 30 lines.--case dogfood-mode, Sonnet, 3 runs): 3/3 with the block, 9/9 checks each, and every planned pass now emitsauditbeforesnapshot -i. With the earlier draft that only said "run audit", Sonnet stopped planningsnapshot -iand the case failedusesSnapshotI(2/3), which is why the line names both commands in order. The newrunsPlatformAuditexpectation is falsified by the base text: base runs produce plans with noauditcommand.logs-only evidence and missed the pre-existingcapturesStrongIssueEvidencecheck; the runs before and after it score 3/3.Test plan
pnpm formatpnpm buildnpx vitest runoversrc/commands/schema/cli-help-{first-screen,workflows}.test.ts,scripts/__tests__/help-conformance-{bench,topic-coverage,expectation-falsification,error-recovery-coverage,find-verification,sample-outputs}.test.ts,scripts/__tests__/agent-setup-startup-contract.test.ts,scripts/__tests__/simulator-skills-contract.test.ts,scripts/__tests__/npm-skills-exclusion.test.ts(155 tests)node scripts/help-conformance-bench.mjs --case dogfood-mode --runner claude:sonnet --repeat 3→ 3/3, 9/9 checks eacha4ba7fcba4: all gates green (20 pass, 2 skipping). The firstCoveragerun failed only ontest/integration/interaction-contract/coordinate.contract.test.tstiming out at 5s under runner load; it passes locally on this head and a rerun is green. This PR touches no test file or device command.🤖 Generated with Apex