Repository navigation
Windows capture core: journal check-then-open race fails UiCaptureDriver tests intermittently on main #826
Description
Describe the bug
The capture-core supervisor merged in #824 (fde702be) reads the worker's process-journal.json with a check-then-open sequence (Test-Path, then Open(FileMode.Open, Read, ReadWrite|Delete)). Meanwhile the worker publishes each new journal by replacing the file (File.Replace on an existing file, File.Move for the first). Between the check and the open the file can be briefly absent, so the supervisor throws Could not find file …\process-journal.json.
That makes the UiCaptureDriver cases in ValidationRunner.Tests.ps1 fail intermittently on hosted runners: 49 run, 47 pass, 2 fail (an accepted-admission command, and the report guard that depends on it).
- Observed on run 38673179020 attempt 2, job 116225253342 (
windows-spikes Swift), on a branch whose diff doesn't touch the capture core. - The same suite passed on Bind capture admission to coordinator-leased job identity #824's own exact head.
So this flakes on main and can fail any Windows PR.
Expected behavior
The journal is published and read without any window where the reader can observe a missing or partial file, and nothing is weakened: guards, hash and parse checks, and cleanup proofs all stay.
Proposed direction
- Publish immutable numbered snapshots (
process-journal.<seq>.json). The worker writes each one to a temp file in the same directory, then does a non-overwritingFile.Moveto the next sequence number. - The reader lists the snapshots, opens the highest complete one, and treats
FileNotFoundExceptionon a just-listed name as "list again", within a bounded deadline, never as fatal. - The supervisor removes the snapshots at release, and that cleanup is proven.
- No check-then-open sequences.
- Tests: a deterministic regression where a held reader runs against a concurrent writer that publishes many snapshots.
Additional context
- Within the Windows parity lane G effort (Windows: Share the pinned local UI verifier and capture driver #822), this is the 5th failed fix iteration on this step: an earlier
File.Moveoverwrite attempt hit File.Move (overwrite) unreliable because FileShare.Delete access of target file ignored dotnet/runtime#114230. Under the escalation rule, it gets a fresh owner. - Until it's fixed, a rerun of the affected job is a legitimate retry for PRs whose diff doesn't touch the capture core, with this issue cited.
Activity
coneilen commentedon Oct 15, 2026
Coordinator implementation approval and sole-writer rule
Operator-delegate decision: this is the fifth same-step failure; R2's owner is read-only handoff only, with the count and all failures preserved. Start exactly one fresh owner on claude-sonnet-5.5/high/long_context, with independent gpt-6.1-sol/high review. Captures stay blocked until this issue's fix merges.
Exactly four existing files are leased to the fresh owner: Tools/windows/UiCaptureDriver.psm1, Tools/windows/Invoke-UiCapture.ps1, Tools/windows/Invoke-UiCaptureWorker.ps1, and Tools/windows/Tests/UiCaptureDriver.Tests.ps1. The scope is the shared journal publication/read/cleanup protocol and its real regressions, not job identity, scene adapters, product code, workflow/runner budgets, provider pins, WinApp, or a second harness. D's shared-core edits and local tests are paused and will rebase after this merges. Preserve D's local work; do not modify its worktree.
Mandatory protocol: worker writes complete, explicitly sequenced snapshots to a fresh temp file in the same owned directory, then publishes process-journal.NUMBER.json by non-overwriting File.Move. Never overwrite, replace or pre-delete a published snapshot. Reader enumerates the valid published names, selects the highest complete published sequence, and directly opens it without check-then-open. Only FileNotFoundException for a just-listed name causes a bounded re-list within the existing run/cleanup deadline. Do not catch parse, encoding, access-denied, hash, lease, process-generation or other I/O failures as missing; do not fall back from a corrupt highest published snapshot to an older version. A temporary filename is not a published snapshot. Validate sequence/name/body agreement and unchanged lease/job/control identity. At final quiescence require the completed final journal and complete arrays, not an old snapshot or empty enumeration.
The supervisor removes journal snapshots at release and proves scoped cleanup. The proof must say what was removed, verify a positive pre-cleanup inventory when publication occurred, and distinguish a no-worker refusal from cleanup of an executed run. Do not delete primary failure logs/receipts or another process's files. Preserve diagnostic hash/identity/version evidence before removing the exact owned snapshot files, so deletion does not destroy the evidence supporting the receipt. Any removal/verification failure stays nonzero/cleanup-unproven, not successful release.
Preserve all 49 current test names/assertions, real supervisor/lease/worker/process/release paths, explicit UTC/15-minute/90-second reserve, shared build/review/CI/equivalence/immutable JSON guards, held-kernel/generation/descendant/EOF proof and official finally release. Add deterministic real concurrent reader/writer coverage with many numbered publications, held old readers, correct highest/final reads, bounded disappearance/re-list and cleanup proof. No process/lease/supervisor/worker/release mocks. Add malformed highest, missing final, wrong binding/sequence and cleanup-failure negatives; those cannot pass through re-list. Test-first retained behavioral RED, GREEN and wider positive-count regression; a compile/setup error is not behavioral evidence.
Local execution is restricted to exact no-desktop child and body-check commands in the kickoff, plus a named scratch-only journal-concurrency driver if explicitly listed there. No local full ValidationRunner, validate.ps1 windows-shell unit/integration, hidden native HWND fixtures, GraphCode/graphcoded/zmx, GUI/input/screenshots/backend, WinApp or native compatibility work. CI owns broader runners. Source/packet review, positive hosted full-runner counts and exact-head required CI must pass before coordinator merge. No timeout increase or dropped/renamed cases to make a run green.
Observed failure: E's b1ed PR run 38673179020 attempt 2/job 116225253342 logged supervisor Open/FileNotFound for process-journal.json and 49/47/2; the blocking/writer/foreign mechanism was not measured and no worker journal artifact was uploaded. The old controlled Replace test passing did not establish collision stress reliability. Main manual run 38675369427 is attribution evidence only; if it fails the same way, retain exact main/run/job/log proof here, not as a substitute for PR checks.
Unrelated PRs with no four-core-file diff may use up to two failed-job reruns per head, preserving failure IDs and citing #826. E has one rerun left. That policy neither permits a fresh capture attempt nor clears a capture lease; no capture is admitted until #826 merges. Keep #822's broader verifier and all product/visual parity claims open. Use Refs #826 and Refs #822; do not auto-close the broader verifier.
Align supported Windows context menus and connector creation (#816)
coneilen commentedon Oct 15, 2026
Operator escalation: final fresh owner, CIM determinism first
J3 is stopped and #827 is closed unmerged as superseded. Its handoff (91D5E5600BF7F18CBC71AEA7288E31BE04071C965800A8A597725C7A1B9E694B), failed and invalid runs, uncommitted review-fix candidate and review rejection are preserved; none is accepted as green. No previous failure counter is reset or hidden.
The operator authorizes exactly one fresh claude-opus-5.5/high/long_context owner, reviewed by gpt-6.1-sol/high. This is the final automatic owner: five failed same-step attempts on this owner means stop and escalate to the operator, not start another owner. Any command outside its exact kickoff allow-list is an immediate stop, not after-the-fact approval. No mid-run source edit, overwritten log, root renaming to reset attempts, broad local runner, product/native UI or capture is permitted.
First deliverable is deterministic process/CIM attestation, before journal work. The preserved unmodified-main control has all four main source hashes (module D4579E92, supervisor 7C7DBE17, worker E6C5F1CE, tests 1638C788), 49 executed/48 passed/1 failed, with the short-lived scene target's incomplete CIM metadata followed by held-kernel exited and stdout/stderr EOF. This is real baseline attribution, not permission to weaken the live-process guard. Require a reproducible behavioral RED on main before the fix, then the same state/race control GREEN. Use retained kernel generation/image and observed exit plus bounded EOF to establish already-gone before interpreting missing CIM. Missing/incomplete CIM on a kernel-live child remains failure; do not infer death from an empty query, sleep until luck, add permissive retries, or make live application identity succeed without complete expected parent/session/image evidence. Explain the state transition explicitly so waiting for a long-lived app's exit is not accidentally made a prerequisite to attest it alive.
Only after that proof is accepted, implement the mandatory immutable-numbered-snapshot design from comment 6183609665: same-directory full temp then non-overwriting Move; numeric highest complete snapshot; no check-then-open; bounded re-list only for FileNotFound on a just-listed name; exact terminal version/hash/identity/typed arrays bound to manifest after worker quiescence; supervisor release deletion with positive scoped inventory and retained hash/identity evidence. Corrupt/missing terminal, vanished highest floor, wrong lease/version, sharing/ACL/parse errors and cleanup failure cannot fall back to old success. Preserve all original 49 cases and add actual deterministic held-reader/concurrent-many-publisher and terminal/cleanup negatives. No second harness, mutable replacement loop or delete-before-publish gap.
Exactly the same four existing files: Tools/windows/UiCaptureDriver.psm1, Tools/windows/Invoke-UiCapture.ps1, Tools/windows/Invoke-UiCaptureWorker.ps1, Tools/windows/Tests/UiCaptureDriver.Tests.ps1. Scope adds only tightly coupled deterministic CIM state handling plus the approved journal protocol and tests. No Runner/workflow/budget, provider/pin, product, scene adapter, schema, README, agent or WinApp changes. D adapter and capture executor remain parked. No other core writer is active.
Hard local validation commands and one-use fresh output paths are listed in the assigned kickoff; any additional command or consumed-root rerun requires prior coordinator approval. Required source/hash/report/count checks, original failure preservation, different-vendor exact-head review, actual positive hosted full-runner/G1 counts, current required CI and full-template body remain mandatory. This core-changing PR does not get the unrelated-PR retry exception. Captures stay blocked until the qualified #826 fix merges and a separate current-product admission/lease is granted. Broader #822 and product visual claims remain open.
Share compact loop-card presentation across project and overview (#814)
Bind Quick Chat workspace chrome to stable chat identity (#815)
[-]Windows capture core: journal check-then-open race fails UiCaptureDriver tests intermittently on main[/-][+][Blocked: needs operator decision] Windows capture core: journal check-then-open race fails UiCaptureDriver tests intermittently on main[/+]coneilen commentedon Oct 15, 2026
Status: blocked: needs operator decision
J4 has stopped permanently at its fifth carried failure, as directed by the operator. No further capture-core owner, small fixes or journal Phase B are authorized. The live-window capture pipeline and D capture adapter stay parked; this issue remains open and does not establish a capture grant.
Read-only handoff evidence is retained under C:\gcs\j4; the exact final report is cim-green3/report.json, SHA-256 CB509A282DFB8AE345879F15958DD9F93EEAFCDDA3C01FB604630AAA628DC124, with 5 executed, 3 passed and 2 failed. The two observed defects are WQL rejecting ProcessId = -1 as an invalid query in the classifier test, and a scalar filtered pending-launch result lacking Count under StrictMode. They are recorded as failed validation, not excused or fixed after the stop. The decision-seam matrix and one endpoint/cleanup control passed, but the full regression was not executed and the lifecycle fix is not accepted. Frozen module/test/driver hashes and prior review/failure history remain preserved; there is no commit, PR or merged fix from J4.
The offscreen renderer continues independently within its already approved seven-file scope after C merges and each image is bound to the exact rendered source. It is "real paint path, offscreen component render", not whole-window, popup, terminal, per-monitor-DPI or NativeForms proof. No manual screenshots are permitted. The existing allowance of at most two failed-job reruns per head for PRs that do not touch the four capture-core files is unchanged, with this issue and all failing IDs retained. No fix proposal or new execution authority is supplied by this status comment.
Frozen read-only handoff: C:\gcs\j4\handoff-j4-frozen.md, SHA-256 3EFF2F9BC8E5B431DCF78C85D6F9166D0F5E8CE0E48198B591253BCA5D2F0EFA. It preserves the actual source, command and failure history; any unimplemented suggestions inside that historical artifact confer no permission. The operator's final freeze supersedes its earlier resume-after-merge wording: even a later #826 merge is not automatic authority to resume live capture.
[-][Blocked: needs operator decision] Windows capture core: journal check-then-open race fails UiCaptureDriver tests intermittently on main[/-][+]Windows capture core: journal check-then-open race fails UiCaptureDriver tests intermittently on main[/+]coneilen commentedon Oct 15, 2026
User-authorized new owner and bounded continuation
The user explicitly decided on 2026-10-14 at 23:55 PDT: "real-window screenshots option 1, two small capture fixes - approve under a new owner". This supersedes the no-new-owner restriction in the preserved J4 stop; J4's failures and frozen handoff remain unchanged and J4 stays parked.
J5 is the new GPT-6.1 Sol/high owner, with Claude Sonnet 5.5/high reviewing before any run. Phase A is limited to the disclosed invalid negative-WQL absent fixture and StrictMode filtered-array Count defects, starting from the frozen J4 candidate. It has a fresh five-failure budget explicitly granted for this new owner. No other edits are authorized. The decision driver must pass positively before the direct 49-floor regression, followed by current-head CI. Runs will use fresh J5-owned roots, not J4's consumed cim-green3. Exact commands remain separately gated on the review and coordinator grant; no local full ValidationRunner or windows-shell is authorized.
Only after Phase A review plus green CI is accepted may J5 plan the immutable numbered non-overwriting journal repair. That Phase B plan must be reviewed before implementation. There is no journal or lifecycle acceptance from this decision alone.
The selected real-window route is pinned WinAppCli 0.5.0 for exact admitted owned HWNDs, ordinary window chrome and authentic native dialogs. One GPT-6.1 Sol/high read-only planner is preparing its scope on #822 using the 20-criterion/30-row research addendum. Implementation waits for this issue to merge and plan review; no visual CI/golden gate or manual screenshots. Menus retain semantic HMENU evidence; terminal and tray capture stay deferred. Cap and the Quick Chat adapter remain parked until the repaired core merges and explicit sequential resumption. The unrelated-product two-reruns-per-head rule is unchanged; it does not apply to the core-changing repair PR.
Align Windows sidebar navigation with active destinations (#817)
Document bounded parity evidence and execution admission (#828)
coneilen commentedon Oct 15, 2026
Operator exception: proceed to the Phase B plan, not a retry
The original required Windows 2022 hardening failure on draft #829, head
96950ce7184eb46854aac1c54b1fb502f6688a89, is the inherited journal sharing-violation variant: run 38699220915 / job 116303129929 reports 51 executed / 49 passed / 2 failed. The first is the admission-lane-d journal Open sharing violation; the second correctly rejects that failed rehearsal data. The CIM/mutant/incomplete/rollback/EOF cases passed, and the Windows 2025 job passed all 51. This is not an observed Phase A defect.The operator classifies that inherited failure as uncounted: J5 remains 0/5 and explicitly permits the same owner/branch/PR to plan Phase B now, overriding the normal Phase A green-CI prerequisite for this specific inherited journal blocker. No rerun of 38699220915 is authorized. The original failed job stays preserved as historical Phase B RED; full required green CI is still mandatory before merge. This exception is not the unrelated-product two-rerun allowance.
Next authority is only a short delta plan comment on #829 and a Claude Sonnet 5.5/high plan review before any journal/test edit. Scope stays the four existing capture-core files and their journal tests. A separately admitted deterministic real reader/concurrent writer RED must fail current/main through the actual sharing/missing-open behavior, not compilation or missing APIs; then implementation, GREEN, the full 51+ direct suite on fresh owned roots, a fresh push and fresh CI remain exact-command gated. No local full ValidationRunner or windows-shell, GUI, installs, capture, manual screenshots or new owner.
Design refinement: unique closed/flushed
.tmpbecomes immutableNNNNNN.jsonvia a non-overwriting move; target conflict fails and published snapshots are never deleted during the run. Only transient Open FileNotFound/DirectoryNotFound or sharing/lock HRESULTs 0x80070020/0x80070021 may retry, bounded by count and deadline, at the same immutable sequence or a newer one, never older. Exhaustion is JournalUnavailable with sequence/HRESULT. ACL, parse, hash/length, encoding, foreign identity, schema, sequence gaps/duplicates and temp/partial masquerading as published fail closed immediately with explicit classification. Release binds the exact final sequence and hash, not highest-seen-at-release. Quiescence, both EOF and exact owned cleanup remain required.The live executor/adapters remain parked. The accepted stock-only #822 design is not implementation, installation, native qualification or capture authority; those still wait for the repaired core merge and separate grants.
Export source-bound offscreen canvas paint evidence
Export source-bound offscreen canvas paint evidence (#830)
Pair combined macOS references without promoting Windows parity (#832)
coneilen commentedon Oct 15, 2026
Follow-up only: independent lease provenance for reused seed evidence
The journal repair in #829 must not introduce mandatory
actualSeedEvidenceReuse.sourceLeaseId. Read-only history confirms that field is absent from supervisor blobb15d099520b65bcd1abdba4fecb09c3d45aca653at frozen baseline96950ce7, its29160994parent line, and locally availableorigin/main661f2837. It was introduced by the uncommitted Phase B candidate and has no in-repository producer.Per the operator decision, only that newly introduced comparison is removed. Existing current-lease, source-run, phase-receipt and source/hash checks stay; the numbered journal still binds its actual stored lease/run/worker/source identity across exact payload, terminal, manifest and final-record byte references. No default, lease self-comparison or legacy-protocol acceptance is added.
An independently approved expected historical source-lease value would require a separately reviewed producer/consumer provenance contract and producer handoff. That work is outside this journal fix; no new schema, producer, issue, adapter lease or implementation is authorized by this note. The helper's file-level positive/negative proof is not execution of the non-rehearsal
Assert-CapPreflightReportpath, and no live preflight result is claimed.Repair Windows capture attestation and immutable journals (#829)