Repository navigation
PTYCAP-05: throwaway-repo due->PR proof (code + owner runbook) - #511
Conversation
Zero Exposure AI ReviewResult: NOT REVIEWED -- PR TOO LARGE: after path exclusions this PR has 555,694 characters of diff across 74 files, which needs 3 review parts; one review covers at most 2 parts of 200,000 characters (400,000 characters). It was not reviewed and will not be auto-merged. To get an AI review, split it into at least 2 smaller PRs (for example one per phase), each under 400,000 characters of reviewable diff. Otherwise a human must review and merge it. Planning docs, lockfiles, generated specs and build output are already excluded (per-project override: review_path_exclude). Reviewer models (requested pin, not independently confirmed): |
d1156c3 to
f5f2c75
Compare
…ue->PR proof Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tracer-first plan set for Phase 5. Builds and tests ALL machinery (allowlist entry mechanism, hard per-proof budget, branch-only, never-merge guard with ruleset assertion + detective layer, terminal "did nothing and why" outcomes, counter-key pin tests, rollback ordering), flag-gated default OFF. The live proof is OWNER-GATED: no plan arms prod or touches Coolify/prod DB; plan 09 writes the arming + rollback runbook as owner steps. Waves: 01 tracer+state | 02 github+protection, 03 verifier, 04 read API | 05 gates, 06 guard, 07 arm CLI | 08 e2e+noop, 09 runbook/docs. Every locked-verdict amendment (F1-1..6, F2-1..4, F3-1..4) maps to a task and a test in 05-VALIDATION.md. ROADMAP Phase 5 gets the plan list. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… true no-op - orchestrator_proof_runs table (additive DDL, named CHECKs, partial unique indexes) - proof-types, tracer DAL subset, proofGuard + single-winner terminalizer, proof prompt - macro-cycle: optional proof hooks, loaded only when the flag stack is ON; fail closed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ribution-safe token sum - armProofRunTx/disarmProofRunTx/enableProofTaskTx: one transaction each under a per-user advisory lock - findProofRunForRequest, findActiveProofRunForRepo, countProofInjects, recordProofObservation (monotone tokens_hwm), listProofRunsForUser - sumSessionTokensSince reports sessionExists so a deleted session fails closed - DB-free fake sql fixture, DAL and schema static-pin tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… fake - ProofGithub interface (10 read-only methods), typed ProofGithubError (forbidden|not_found|unavailable) - makeFakeProofGithub with call log, per-method fail map, unmodeled-method throw - shared scenario helpers greenPr/redPr/mainAdvanced/unreadableRulesets Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…act plan Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… transport - GET-only injectable request, 403/404/other mapped to forbidden/not_found/unavailable - bypass_actors absent normalizes to null (never empty), check state truncation flag - five-probe permission check, no network in tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…permission and agent-admin checks - branch_lock or review_gate shapes only; PR plus green check is insufficient - zero visible bypass actors; null bypass_actors is unverifiable - digest/updated_at/ids/default-branch re-assert, never throws Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e detection - verifyProof classifies from ProofRun frozen fields + read-only ProofGithub only - compare(base_sha, default) not HEAD equality: ahead/behind/diverged/not_found are terminal main_advanced - merged proof PR, foreign proof/ head, wrong base are terminal; unreadable reads are ci_unverifiable Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cision - green requires open PR, >=1 passing check, every frozen required context success, no red/pending, not truncated, re-compare identical - neutral/skipped are not success; unreadable, truncated or sha-less checks are ci_unverifiable - 47-test scenario matrix over the shared fake Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- tenant-scoped (user id only from auth context), allowlisted response mapping - no installation id, ruleset ids, raw preconditions or counter key; digest truncated - mounted after the auth catch-all; mount-order test pins 401 unauth Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd regenerate docs - _openapi.ts spec-only registration (200/401/500), bearerAuth - docs/openapi.json and docs/api.md produced by bun run docs:sync (stable on rerun) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- proofBudgetGate: injects / four-bucket tokens (monotone hwm) / wall, counter-key pin, terminal block - proofProtectionGate: re-asserts frozen ruleset before every proof dispatch - flag OFF is zero-call pass-through; flag ON fails closed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- kill switch first, proofBudgetGate + proofProtectionGate next, counter last - coverage scans assert position with non-vacuous self-checks (additive) - counterkey test pinned the old list: extended additively Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kup args) - proof DAL/gate lookup args use requestKey/key; counterKey stays a DispatchRequest-only identity Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- runMergeToMain refuses proof_repo_never_merges for a repo with an armed/running proof run - one run-log row, nothing injected; flag OFF skips the lookup; lookup error fails closed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rmination - killSwitch, repoMatch, protection, verifier, agentDeclined, budget (locked order) - wall always terminates on the tick; spent budget waits only while a turn is live or CI pending - selectBudgetOutcome + frozenFromRun (owner/repo from the run row) - guard shares one evaluator ctx so the verifier observation reaches later evaluators Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…unter-key pin - observeInject maps refusals onto the closed nothing:/terminated: sets; recoverable ones are recorded only - hook seam now receives the full inject outcome and guard input (reason was previously dropped) - realProofHooks wires defaultProofEvaluators over the real DAL, kill switch, GitHub adapter and token ledger - pin test: runMacroCycle threads taskId; real injectOrchestratorPrompt builds task:<taskId> online + autospawn Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ep, no-scrub scan - 11 scenarios over runMacroCycle + real evaluators + shared fake GitHub, exactly-once side effects - final injected string: no banned token, push form pinned, only the run branch ref - no-scrub scan with negative fixture - terminal notification now carries the PR url when known Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…an summary Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dry run - 13 refusal codes, first failure wins, zero writes on refusal - budgets frozen from flags then deps.env then defaults; garbage refuses - writes arm_preconditions.bypass_verification = 'verified' (05-04 key) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the proof as budget_attribution_lost (codex #4) The agent ws handler logged and dropped a failed token_usage insert, so the proof's hard token budget silently under-counted while the session row still existed. The handler now marks the session in an in-process registry; sumSessionTokensSince reports attributionLost for any proof window the loss falls in, and the budget gate and tick evaluator both fail closed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…R to come from the armed repo (codex #5) The adapter dropped every off-prefix PR before the verifier could judge scope, and a fork PR reusing the exact proof/<run-id> branch name could be reported pr_open_green. listPulls now returns all recent PRs with head repository, author and creation time; the verifier terminates a PR created after arming from any non-proof/ head, and a run-branch PR whose head repo is not the armed repo (or is deleted), as pr_scope_violation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… allowlist, proof row, session match and merge refusal (codex #6) owner/repo path names are case-insensitive, so github://Owner/Repo and github://owner/repo were two idents for one repository and bypassed single-flight and the merge refusal. armProof and armProofRunTx canonicalize through lib/repo-key.ts, findActiveProofRunForRepo canonicalizes the lookup, and schema.sql adds an additive NOT VALID CHECK (repo_ident = lower(repo_ident)) so the existing partial unique indexes are single-flight over the canonical value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o repo key (lens-correctness MINOR 2) With the proof flag stack on, runMergeToMain skipped the proof-repo refusal entirely when ctx.repoKey was null, leaving the only auto-merge path unguarded for a keyless session. It now refuses with proof_repo_never_merges:unverifiable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…unds the budgets (adversary MINOR 3, 6) An admin agent identity can edit the ruleset and merge in one turn, yet arm only warned when --agent-login was missing or unreadable; it now refuses agent_permission_unverified. The owner-supplied "hard" budget had no upper bound; arm refuses above injects 50 / tokens 50M / wall 24h and schema.sql adds matching additive NOT VALID CHECK ceilings. Runbook updated (agent-login required, ceilings, canonical repo ident, attribution-loss and PR-scope behavior). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m and enable (codex r2 new #1) The exclusivity count ran under a per-user lock, so two users could both see zero enabled orchestrator tasks and both enable. All proof state changes now share one global lock and recheck the count under it. The fake sql models real blocking advisory-lock semantics so the race is exercised. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-send boundary under the arm/disarm lock (codex r2 #2, #3) buildSend persisted the prompt (awaited) after the pipeline's final halt check, so an arm or disarm that committed in that window was followed by a stale send. With proof mode ON the delivery now runs after persistence behind sendAtProofBoundary: a fresh kill-switch + active-run identity recheck while holding the same global advisory lock arm/disarm/enable take. Flag OFF is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…emoves allowlist case twins (codex r2 new #2, sonnet MINOR 1, MINOR 3) Arm now refuses (session_halt_engaged) when a foreign session halt is engaged, releases only stale halts left by the proof tooling, records the id of the row it engaged on the run (additive kill_switch_id column) and --enable releases exactly that row, so an owner emergency stop engaged after arm survives. Disarm deletes the allowlist row by lower(repo_ident), so a mixed-case twin no longer survives. Runbook gets the new refusal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nspected, fails closed otherwise (codex r2 #5) The adapter read only the newest 50 PRs, so a post-arm scope violation on a later page was invisible. It now pages newest-first until a page is short or its last PR predates armed_at (the verifier passes it); hitting the page cap or any page error throws and the verifier stays pending (never green). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… durable across restart (codex r2 #4) sumSessionTokensSince now also compares completed assistant turns of the session since arming (older than a 120s grace) with token_usage rows over the same window; fewer rows than turns reports attributionLost, which the budget gate and tick evaluator already terminalize as budget_attribution_lost. The in-process mark stays as an additional early signal. No new DDL; nothing is evicted or forgotten on restart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-owner, SHIP-AFTER-OWNER-GO) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…agged session halts Budget attribution: chose the nullable additive turn_id column on messages and token_usage (supervisor stamps one id on a turn's assistant_message and result/usage_event) over a usage outbox: smaller, no new table, no backfill. The durable detector now anti-joins each completed assistant message to a token_usage row of the SAME (session, turn_id); aggregate counts are gone and a message without a turn_id fails closed. Kill switch: session halts are unique per (session_id, engaged_by) while engaged, so a manual emergency engage after arm is a separate row, effective on its own, and proof --enable releases only its own row by id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tive MAJOR recorded) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#515) * fix(PTYCAP-05): e2e attribution-loss check uses a real proof identity QC round-2 added proofIdentityRefusal ahead of the attribution check in the proof budget gate. The e2e section 6 seeded a raw run with a synthetic counter key and no proofRunId, so the gate now refused lifetime_counter_key_missing instead of proof_budget_attribution_lost. Seed the run with the task:<id> key, a wall budget the seeding cannot outlast, and pass proofRunId. Test-only; no baseline change (the real failure was fail=1, not drift). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ci: retrigger woodpecker (pull_request opened hook returned 204, no qc pipeline created) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…-measure baseline; QC repair record Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rm; spawn tests get a realistic timeout; re-QC record Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f5f2c75 to
a418613
Compare
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a418613295
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| WHERE user_id = ${args.userId} AND status IN ('armed','running') | ||
| RETURNING * | ||
| `) as ProofRun[]; | ||
| if (ended.length === 0) return { disarmed: false as const }; |
There was a problem hiding this comment.
Allow disarm cleanup after a run reaches a terminal outcome
After a proof succeeds or terminates, the documented --disarm cleanup does nothing: the UPDATE only selects armed/running runs, so this early return skips removing the allowlist entry and disabling the proof row. finishProofRun does not perform either cleanup itself. Consequently, following the runbook leaves the repository allowlisted and attempting to arm a different repository fails with other_repo_allowlisted. Clean up the completed run's configuration as well, while preserving its recorded terminal outcome.
Useful? React with 👍 / 👎.
| await tx` | ||
| INSERT INTO orchestrator_rows (task_id, command, enabled, schedule_rule, sort_order) | ||
| VALUES (${taskId}, 'proof', true, ${tx.json(args.scheduleRule as any)}, 0) | ||
| `; |
There was a problem hiding this comment.
Isolate the proof cadence when reusing an orchestrator task
When arming a session with an existing orchestrator task, this adds the proof row but leaves all existing command rows enabled. scanAndEnqueueDueCycles enqueues a cycle whenever any row is due, and that cycle runs the proof guard and injects the proof prompt regardless of which command triggered it. An existing one-minute row therefore makes a proof armed with --every 30 run every minute and consume its inject budget prematurely; previous proof rows can likewise survive a terminal run and affect a re-arm. Disable or otherwise exclude the old rows so only the current proof schedule drives this task.
Useful? React with 👍 / 👎.
Keep Phase 5 proof DDL and Phase 7 heal DDL, both CLAUDE.md sections, and a re-measured hub test baseline (pass 4006, skip 291, fail 0). Bound the proof schema pin to its own section, compare the heal ledger to main once it is already there, and send the active proof run id in the real-Postgres budget check so that check asserts the inject ceiling.
…strator task The five-way single-flight check shares one database with the parallel e2e suite. Arm refuses every caller while any other orchestrator task is enabled, so a concurrent due-waves scan made all five arms lose. Wait out that transient refusal, then still require exactly one winner and four proof_already_active losses.
The proof arm is hub-global, and orchestrator:e2e runs the e2e files concurrently. An 8s poll expired while a peer test still held an enabled orchestrator task, so all five arms lost with other_orchestrator_task_enabled and the single-flight assertion never ran. Wait up to 90s, inside the enclosing test timeout. Co-authored-by: Michael Smith <finedesignz@users.noreply.github.com>
Baseline counts stay on main's floor; phase 5 adds no new skips. Co-authored-by: Michael Smith <finedesignz@users.noreply.github.com>
The proof arm is hub-global, so a concurrent e2e file that leaves an orchestrator task enabled blocks every arm for the whole suite. A 90s poll still expired in CI. bun test --max-concurrency=1 keeps that peer from overlapping the single-flight check. Merge main after the task-qc schema landed and keep both DDL blocks. Co-authored-by: Michael Smith <finedesignz@users.noreply.github.com>
…applied stage-presets left its orchestrator task enabled, so the hub-global proof arm waited out the whole quiet window and still refused. Delete that user when the test ends. A concurrent AAF duplicate can take the notify claim after the transition commits; the delivery that applied the transition still answers applied. Co-authored-by: Michael Smith <finedesignz@users.noreply.github.com>
|
Zero Exposure did not auto-merge this PR. Blocking reason: auto-merge blocked: not eligible -- diff classifier: .planning/phases/PTYCAP-05-throwaway-repo-due-to-pr-proof/05-01-PLAN.md: potential flag/safety change: identifier-shape heuristic matched "enabled" added with a falsy value and no removed counterpart to pair against; semantic effect unverified; requires review: (3) orchestrator-proof-dal.ts (tracer subset; task 2 completes it): claimActiveProofRunForTask(taskId) = ONE statement UPDATE ... SET status='running', started_at=COALESCE(started_at, now()), updated_at=now() WHERE task_id=$1 AND status IN ('armed','running') RETURNING * (the single atomic find-and-claim; returns null when none); taskHasAnyProofRun(taskId); terminalizeProofRun({runId, status, outcome, detail, prUrl?, prNumber?}) = conditional UPDATE ... WHERE id=$1 AND status IN ('armed','running') RETURNING * setting status, outcome, outcome_detail, pr_url, pr_number, terminal_at=now() (null result means the claim was lost); disableProofTask(userId, taskId) = UPDATE scheduled_tasks SET enabled=false WHERE id AND user_id AND task_type='orchestrator'. All via the shared sql from ../db/postgres.ts.; diff classifier: .planning/phases/PTYCAP-05-throwaway-repo-due-to-pr-proof/0 Reviewed commit: |
PTYCAP Phase 5 - Throwaway-Repo Due->PR Proof (code + owner runbook)
Status: DRAFT - /qc BLOCKED (2 open MAJOR after 2 fix rounds). Live proof is an OWNER GO/NO-GO gate and was NOT run.
What it builds (all flag-gated, default OFF = true no-op)
orchestrator_proof_runstable + DAL (additive idempotent DDL; frozen budgets with CHECKs; single-flight partial unique indexes; canonical lowercase repo identity).pr_open_green, else a typed "did nothing and why" terminal.GET /api/orchestrator/proof(OpenAPI + docs synced).docs/ptycap-05-proof-runbook.md(throwaway repo + ruleset, GitHub App permissions incl. the bypass_actors visibility decision, arming, rollback order: kill switch/disarm FIRST, then flags).QC
Seats: sonnet correctness + adversary lenses, Codex read-only (agy MISSING). Record:
.planning/phases/PTYCAP-05-throwaway-repo-due-to-pr-proof/QC/.Open MAJOR (see QC/VERDICT.md): (1) attribution check compares aggregate counts, not per-turn; (2) a manual session halt engaged AFTER arm is indistinguishable from the proof-owned row. Both reachable only after an owner arms the proof.
Verification
bun run check-baseline: OK (pass 3169, skip 259, fail 0). One known flaky timeout (transcript-adapter-codex, AgentAutofix 74489ffe) on a first run.Not done here (owner-gated)
SC-2/SC-3 live proof: arming
REMO_ORCHESTRATOR_AUTOSPAWN+ proof flag + allowlist in prod. Runbook steps indocs/ptycap-05-proof-runbook.md.🤖 Generated with Claude Code
Re-QC 2026-11-10: PASS
Codex (read-only) + Claude review; repaired findings verified fixed; one new real defect fixed (re-arm blocked by earlier-run proof PRs). Record: .planning/phases/PTYCAP-05-throwaway-repo-due-to-pr-proof/QC/REQC-2026-11-10.md. SC-2/SC-3 live due->PR proof is owner-gated (nothing armed). agy seat missing.