Repository navigation
test(agent): answer the credential store's keyring prompts in the harness - #8577
Conversation
…ness Since the session moved into the OS keyring, signing in asks the user to approve storing it. Run without the test scripts' environment, the agent harness E2E stalled at that prompt in builds with TinySecurity: its sign-in timed out and the approval tests never ran. boot_stack now opts the test build's keyring responder in unless the run already chose.
|
Warning Review limit reached
This review includes 1 billable file and costs up to $0.25. Or wait 48 minutes for your next included review. View limit details
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Tiny Sweeper reviewReviewed head: Verdict: Ready for maintainer review · Priority: none Diagnostics and supporting detailsFull summaryPull request purpose (title): test(agent): answer the credential store's keyring prompts in the harness. The test-only responder still answers only the credential store's prompts and remains absent from product builds, so product code and user-visible behavior are unchanged. Latest-push delta: unavailable — no trusted base-to-head diff comparison was available, so no push-delta behavior is asserted. Review snapshot
Completeness: Complete What changedNo product code or user-visible behavior changes. Latest-push delta: unavailable — the prior reviewed head exists, but there was no trusted base-to-head diff comparison, and cached prompt differences are not a proven push delta. FeaturesNone identified with supported citations. TestsNo supported feature-to-test mapping was produced. Test execution is not inferred. FindingsNo active actionable findings. Before mergeNone. Agent review detailscritique
security
tests
commits
description
Evidence and run details
|
Summary
Run directly against a staged TinySecurity, the agent harness E2E (
tests/agent_harness_e2e.rs) hung for 120 s at sign-in in every test. This PR makes the harness answer the credential store's keyring prompts itself, so its tests get past sign-in.Problem
Since #8536, signing in stores the session token in the OS keyring, and TinySecurity asks the user to approve that write. Tests answer it through the test-only keyring responder (
test-keyring-consent, compiled into the CLI test targets). That responder acts only whenOPENHUMAN_TEST_KEYRING_CONSENTis set, and onlyscripts/test-rust-e2e.shandscripts/ci/rust-coverage.shset it. Run directly withcargo test --test agent_harness_e2e, withsecurity-moduleand a staged TinySecurity, the boot'sauth_store_sessionwaits on a prompt nobody answers. Each test fails after its 120 s timeout (error sending request,agent_harness_e2e.rs:636) (finding TS-E2E-018).Fix
boot_stack_with_optionssetsOPENHUMAN_TEST_KEYRING_CONSENT=approvefor its own test process, unless the run already set a value. The responder still answers only the credential store's prompts, and it still exists only in test builds.Scope
This removes the sign-in hang only. Past sign-in, the harness hits TinySecurity refusals it doesn't account for yet: the prompt screen has no model under the scripted mock, the temporary action directory is a protected location, and a LocalOnly refusal of the scripted model. So against a staged TinySecurity most of its tests still fail,
approval_gate_approve_flowamong them. Those gaps are tracked as TS-E2E-034. The official runner (scripts/test-rust-e2e.sh) doesn't stage TinySecurity and already sets the variable, so its runs are unchanged.Tests
security-module, a staged TinySecurity andOPENHUMAN_TEST_KEYRING_CONSENTunset, no test inagent_harness_e2etimes out at sign-in.:636,error sending requestafter 120 s).Validation
Cuttlefish job CF-W4-h18 on
b0f4d8bd41(base6043171902), with a staged TinySecurity,security-module, andOPENHUMAN_TEST_KEYRING_CONSENTunset:--tests,-D warnings)type_complexityerror atcore/observability_reporting.rs:535, which stops the build before the CLI tests compile; it's in every clippy run of main6043171902this sessionagent_harness_e2e, headerror sending request. 2 passed, 25 failed and 1 ignored, every failure after sign-in from the TinySecurity refusals above (TS-E2E-034)approval_gate_approve_flowon main:636,error sending request, after 121 sRelated: TS-E2E-018, TS-E2E-034.
Submission Checklist
agent_harness_e2eitself; the red control (main's harness) times out at sign-in (:636, "error sending request" after 121 s).docs/TEST-COVERAGE-MATRIX.md.docs/RELEASE-MANUAL-SMOKE.md).AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
test/agent-harness-keyring-consentb0f4d8bd41a9dfc08251f5e2aa4283f1db57f780Validation Run
pnpm --filter openhuman-app format:check, no frontend changepnpm typecheck, no TypeScript changeValidation Blocked
command:N/Aerror:N/Aimpact:N/ABehavior Changes
Parity Contract
Duplicate / Superseded PR Handling