Skip to content

Validate offline UI scenario data against issue-owned authority - #820

Merged
coneilen merged 3 commits into
mainfrom
coneilen-microsoft-ui-claim-foundation-g1
Oct 14, 2026
Merged

coneilen merged 3 commits into
mainfrom
coneilen-microsoft-ui-claim-foundation-g1

Conversation

@coneilen

@coneilen coneilen commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Bind offline scenario data to an independently supplied issue-authority snapshot without letting scenarios redefine acceptance criteria. Validation reports valid data, never verified UI, human approval, parity, or issue closure.

Refs #818. Implements G1 only under the approved plan and binding scope correction.

Changes

  • Add three strict draft-07 schemas and the issue-owned claim/proof/state convention, with no real claim or production scenario file.
  • Add a read-only PowerShell validator for duplicate/case-confusable JSON, externally supplied authority/hash binding, exact scenario selection, fixture paths/revisions/hashes, typed selector/action declarations, coverage, deadlines and cleanup intent. It runs no UI, input, command, subprocess, fixture setup or network operation.
  • Add 166 synthetic functional contracts using actual bounded child validator processes, including non-execution/privacy canaries and reparse/device-path refusals. Register one child invocation with a positive structured count immediately before ValidationRunner's final PASS; add only the three approved classifier path cases.

Acceptance checklist: all five G1 implementation contracts are addressed: convention, external authority binding, strict data validation, behavioral RED plus executable registration, and explicit evidence limits. All Windows UI acceptance claims, G2/G3 work and runtime evidence remain outside this PR. No plan deviation; the bounded read and absolute-end identifier patterns enforce the approved limits rather than adding scope. PowerShell 7.5+ is documented for string-preserving JSON conversion.

Test plan

RED: pwsh -NoProfile -File Tools\windows\Tests\UiScenario.Tests.ps1 -> positive valid fixture passed, then RED: callable validator accepted wrong authority revision (incremental WIP, not a main-versus-head product regression)
GREEN: pwsh -NoProfile -File Tools\windows\Tests\UiScenario.Tests.ps1 -> All 166 offline contract tests passed; executed=166, passed=166, failed=0
REGRESSION: gh run list -R scgopi/GraphCode --commit d1abb6e --limit 20 --json databaseId,name,headSha,status,conclusion -> all 7 workflow types succeeded; recorded 24 checks: 22 success/2 deliberate skips; hardening run 38621462157 jobs 116049863649 and 116049863737 each executed UiScenario166/passed166/failed0

Local regressions also passed: pwsh -NoProfile -File Tools\windows\Tests\ValidationRunner.Tests.ps1 (actual new child 166/166/0, then PASS), bash Tools/ci/tests/classify-changes.test.sh (62 passed/0 failed), and unchanged pwsh -NoProfile -File Tools\tdd\Tests\TddEvidence.Tests.ps1 (PASS).

Exact-head workflow Recorded successful run / job Evidence boundary
Windows release hardening 38621462157; 116049863649, 116049863737 Both actual G1 children 166 executed/166 passed/0 failed, then ValidationRunner PASS. Full pinned/environment opt-in jobs deliberately skipped.
Windows shell validation 38621462239; integration 116049842161; aggregate 116058612422 Three unit shards, integration and both packaging jobs succeeded. Scrubbed startup executed 5; no new G1 UI claim.
Windows port validation 38621462819; aggregate 116061747422 Privacy, Swift, terminal/contracts and aggregate succeeded.
macOS shared Swift regression 38621462134, attempt 2; 116053157341 Same-head retry succeeded. Portable XCTest executed 2. The app XCTest wrapper reported 0; no app-test/runtime coverage is inferred from that empty wrapper. Build/lint success is separate evidence.
Linux 38621462683; 116049845375 Build/format/CLI smoke workflow succeeded.
TDD evidence 38621462094; 116049757715 Body contract succeeded; later body edits trigger their own validation.
DCO 38621462198; 116049758711 All three rebased commits retain sign-off and Copilot trailers.

The preserved RED (red-authority.log) is a behavioral failure of an otherwise callable parser/schema validator before authority binding was implemented. WIP validator/test copies and SHA-256 records are retained separately from GREEN. Additional retained iterations: the first GREEN attempt reported a tooling ValidationError on valid data; a later boundary test exposed final-newline acceptance by regex $, corrected with absolute-end assertions. Neither failure is silently substituted with a passing log.

The preliminary Sonnet 5.5/high review found a UNC-path guard precedence bug and two independent negative-test gaps. A mocked-filesystem probe reproduced the path defect without network I/O (red-review-path-guard.log); normalized paths now reject UNC/device/ADS before reads. The same no-network invariant also rejects mapped network drives using local drive-type information, with a controlled no-read RED retained as red-mapped-drive.log. Tests independently reject visual component scenarios and self-anchored absence assertions. One review-fix run hit a fixture file-sharing error during repeated deletion; the failure is retained (green-review-final.log), and fresh per-case scratch paths avoid that reset pattern without retrying or hiding failures.

Local evidence lives in worker-owned C:\gcs\ig\logs: red-authority.log, red-source-hashes.txt, red-validator.ps1, red-tests.ps1, green-attempt-1.log, green-boundaries-1.log, initial green-final.log and regression-runner-final.log (154 cases), intermediate green-review-attempt-2.log and regression-runner-review-attempt-2.log (165 cases), final green-final-166.log and regression-runner-final-166.log (166 cases), green-final-source-hashes.json, regression-classifier-final.log, and regression-tdd-final.log. Initial-head CI hardening job 116017091708 in run 38611878304 independently logged UiScenario executed=154/passed=154/failed=0; it is not final-head evidence.

Retained pre-rebase CI failure: on b8472c0058fdccb74ee3b9aeb8297ff4784bf838, Windows shell run 38616560122/job 116033394023 failed with Scrubbed production shell startup failed with exit code -1073741502 at 18:47:21Z. This is the wrapper's child pwsh -File ScrubbedShellStartup.Live.Tests.ps1 status, with no structured startup result; a product/DLL/infrastructure root cause is not established. Smoke/stress immediately before it passed; the diagnostic-upload/post-checkout steps also failed, and no UIA diagnostics artifact was retained by CI. The raw failed job log is preserved locally as ci-b847-integration-failed-job.log and is not replaced by a passing rerun. One retry of this job on the same SHA passed (job 116042866083, run attempt 2), including SCRUBBED_SHELL_STARTUP: PASS; executed=5; all seven workflows on that SHA then succeeded (22 successful checks, two deliberate skips). No local native/broad suite or gate/product change was made. Both pre-rebase hardening jobs 116033105535 and 116033105589 in run 38616560047 independently logged UiScenario executed=166/passed=166/failed=0. These remain historical, not rebased-head evidence.

Fresh-base rebase: after preserving the completed retry measurement, rebased onto accepted documentation-only main dd501e1befd3e00f9e31254a3c1c0c6595543c29 for strict required-check freshness. Current head is d1abb6ec0f9ca2e3820915426c367ddbd0188ae0. The full eight-file G1 diff and all validator/test/schema bytes are identical before and after rebase (rebase-final-diff.patch versus rebase-dd501e-diff.patch, and green-final-source-hashes.json). docs-rebase-range-diff.log shows all three commits unchanged (=); docs-rebase-no-source-delta.log confirms no product/tooling/scenario delta. Per the user-directed cheap-docs-rebase rule, existing source-review acceptance and 166-case evidence were preserved without another source-review pass or focused rerun solely for the rebase. The table above records fresh d1 CI, not substituted older green runs.

Retained rebased-head CI failure: macOS run 38621462134/job 116049841522 on the exact d1abb6ec head failed during mise exec -- make install-zmx, before app tests: pinned Zig 0.15.2 reported LibCInstallation.zig:174: return error.DarwinSdkNotFound, then make exited 2. This establishes a build-time SDK-discovery failure, not which lookup failed, an image root cause, a G1 behavioral failure, or product runtime behavior. The failed output is retained in ci-d1-macos-failed.log and setup context in ci-d1-macos-first-full.log. The one exact-job retry succeeded (job 116053157341; ci-d1-macos-retry-pass.log); no SDK, tool pin, workflow, product or G1 source change was made for it, and the original failure is not overwritten.

Fresh-head CI catalogs and logs are retained under the same owned log root: ci-rest-d1abb6-retry.json, ci-final-runs-d1.json, ci-final-checks-d1.json, ci-d1-hardening-jobs.json, ci-d1-hardening-2025.log, ci-d1-hardening-2022-rest.log, ci-d1-shell-jobs.json, ci-d1-integration-pass.log, ci-d1-port-jobs.json, ci-final-macos-d1-jobs.json, and the TDD/DCO/Linux job catalogs. The different-vendor Sonnet 5.5/high agent accepted the corrected source and final-166 local evidence; final published-body/CI evidence verification is reported separately to the coordinator. This is agent review, not issue-authority human approval.

Not verified: human approval/authentication of authority exports; live UIA uniqueness, ownership or patterns; input/physical keyboard; rendering/screenshots; fixture installation/executor-version compatibility; actual cleanup, persistence, parity or claim closure. Locally, no product executable, native suite, desktop, Sandbox or WinApp installation was used. Full product unit/integration validation and make test/make check were not run locally; exact-head hosted CI is the regression gate. There is no screenshot claim or desktop slot for G1.

Checklist

  • I have read the Contributing Guidelines
  • I have signed off my commits (git commit -s) per the DCO
  • Tests pass locally (make test)
  • Code follows the existing style (make check)
  • I added the test/contract before the implementation and observed the intended RED failure

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 017c733a-252b-471a-9c43-26a6a67554df
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Add independent visual-component and healthy-root negatives and isolate each synthetic case.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 017c733a-252b-471a-9c43-26a6a67554df
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Exercise the no-read refusal with a controlled drive-type probe.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 017c733a-252b-471a-9c43-26a6a67554df
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
@coneilen
coneilen force-pushed the coneilen-microsoft-ui-claim-foundation-g1 branch from b8472c0 to d1abb6e Compare October 14, 2026 19:10
@coneilen
coneilen marked this pull request as ready for review October 14, 2026 19:58
@coneilen
coneilen merged commit 6916bc2 into main Oct 14, 2026
27 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant