Skip to content

fix(jev): support secure scoped capture on native Windows - #8620

Open
xiaobaifly7 wants to merge 2 commits into
loopx-project:mainfrom
xiaobaifly7:fix/jev-windows-safe-capture
Open

xiaobaifly7 wants to merge 2 commits into
loopx-project:mainfrom
xiaobaifly7:fix/jev-windows-safe-capture

Conversation

@xiaobaifly7

@xiaobaifly7 xiaobaifly7 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

  • Outcome basis / optional anchor: Reproduced native Windows capture defect; no public issue is required for this bounded repair.
  • Goal/source and gap: loopx-jev drift init originally rejected every native Windows workspace because Windows does not expose os.O_NOFOLLOW. Review of the Windows implementation then found that a real 8.3 alias of .git bypassed the requested-path exclusion and reached the file read.
  • Observable before → after: Ordinary files remain capturable; Git metadata reached through long names, case variants, extended/device prefixes or real 8.3 aliases is rejected before reading scoped file bytes. The repaired source-checkout CLI still creates a baseline, captures a changed scoped file after a real refresh, and writes a local typed receipt with egress disabled.
  • Issue/task and intended base: Native Windows Jev drift/shadow capture compatibility and the latest CANONICAL-SCOPE review on PR fix(jev): support secure scoped capture on native Windows #8620. PR base is main; original branch base is 04958a89abbbf6b7d3f79327437e2d6deae97453. Fetched origin/main at 9ce3a46bfe525aa2167de6699fc676ce8ef60247 merges cleanly; no rebase or history rewrite was needed.

Author Declaration

  • Written by: model_agent — OpenAI Codex, GPT-6 family.

Implemented against

  • Specification: The bounded Windows capture repair request, the latest change request, and the existing scoped-capture contract in packages/loopx-jev/src/loopx_jev/drift_capture.py (validate_paths, capture, stable_capture).
  • Criteria:
Criterion (spec clause) Disposition Symbol / path Test or command
Capture ordinary scoped files without O_NOFOLLOW on native Windows implemented drift_capture.capture, _windows_capture test_windows_regular_file_without_nofollow; native CLI init/refresh/status
Reject symlinks, junctions and reparse points throughout workspace traversal implemented _windows_identity, _windows_descriptor, _windows_capture Real symlink/junction tests, simulated reparse attributes, root-swap and parent-pinning regressions
Read the opened file that was identity-checked; reject unavailable or changed identity implemented _windows_identity, _windows_capture Device, inode, size and mtime mismatch tests; zero-identity and mutation tests
Preserve final-component no-follow semantics across check/open races implemented _windows_descriptor uses FILE_FLAG_OPEN_REPARSE_POINT Raced-in symlink to the same original file must never be read
Apply scope exclusions to the canonical opened object before reading, failing closed when unavailable implemented _windows_final_path, _windows_capture, existing validate_paths test_windows_rejects_native_git_path_representations creates real GIT~1 aliases; test_windows_fails_closed_without_final_path injects Win32 API failure/insufficient-buffer results; every negative case traps any attempted read
Preserve stable regular-file capture despite lstat/fstat ctime skew implemented Existing _windows_identity fields remain unchanged test_windows_stable_capture_allows_ctime_skew, ordinary-file and directory-pinning positives
Keep POSIX capture behavior implemented Existing POSIX os.open(..., O_RDONLY | O_NOFOLLOW) branch test_posix_keeps_nofollow_open; branch simulated on this Windows host
Run package tests, relevant shared tests and Ruff implemented Package suite and existing progress-review consumers Validation below records passing and failing checks separately
Produce an offline local capture and receipt implemented Existing Jev CLI, File authority fixture and progress-review receipt owner Real loopx-jev console entry point; no provider dispatch or request reservation
  • Self-check before submission: Reviewed capture callers, scope validation and handle ownership. Reproduced the reviewer's short-name bypass on native Windows before changing production code, then ran the same package/shared suites before and after, the capture suite, Ruff, strict mypy and the real offline CLI journey. The canonical workspace and file paths come from their already-open, identity-checked handles using GetFinalPathNameByHandleW with normalized DOS names; the workspace-relative file name goes through the existing scope validator before read(). No runtime dependency, configuration, receipt schema or capability was added. Native Linux/macOS and Windows ACL qualification were not run.

Scope And Continuation

  • Completed scope and remaining work: Complete within the native Windows capture repair scope. Windows validates reparse attributes and nonzero file identities, pins directory traversal with native handles, opens the final component without following a reparse point, and checks file identity/size/mtime before and after reading. Scope validation additionally checks the canonical handle path, rejecting .git even through a real short-name alias. Failed or incomplete canonical-name queries stop capture. Both run guides now describe native Windows support and its qualification limits. Existing suite/typecheck limitations are disclosed below.
  • Slice boundary / successor: N/A for the requested capture repair. Remaining native suite failures are disclosed rather than absorbed into this change.
  • Bounded refactoring: Applied within the existing capture owner: native descriptor acquisition, identity validation and the Windows capture path remain private helpers. No shared control-plane or language migration is needed for this platform IO repair.

Validation

  • Run state: finished
  • Input classes: synthetic, public_fixture
  • Environment: Native Windows, CPython 3.12.13, Node.js 24.18.0; an editable source checkout in a disposable environment. Temporary state, Git fixtures, caches and disposable Effect runtimes stayed within the worktree.
Check kind Result Public-safe evidence / limitation
regression_parity passed Native Windows before production changes: selected counterexamples plus the ordinary-file positive gave 4 failed / 5 passed; both real 8.3 aliases reached the read tripwire. After: the full test_drift_capture.py suite passes 30/30, without skips. Long-name, case, extended/device-prefix, short-name, API failure/truncation and all prior race/reparse cases pass.
unit failed Identical combined package/shared command before: 141 passed, 10 failed, 4 errors; after: 146 passed, 6 failed, 4 errors. The four new negative failures are fixed, and the added ctime-skew positive passes. All previously passing cases remain passing. Five package failures listed below are unchanged.
integration failed The shared subset in both combined runs retains 28 passed, 1 failed, 4 setup errors. The failure asserts Unix mode 0600; the errors occur while initializing the SQLite fixture (unable to open database file), before capture. File-provider criterion-binding cases passed.
real_entrypoint passed Ran the generated loopx-jev console entry point against a disposable Git workspace: drift init → edit scoped text → drift refresh wrapping real refresh-state → drift status → drift drain → drift status. Shadow mode, allow_egress=false, explicit disposable registry/runtime and suppressed external sinks. Result: baseline created, event queued, capture failures 0, one receipt written.
real_backend passed Disposable File authority and the actual typed receipt writer. Receipt is not_evaluated, reason egress_denied, dispatch not_sent; provider-attempt manifest remains empty. This qualifies local capture/receipt plumbing, not model judgment or acceptance completion. Disposable Effect runtime was stopped afterward.
regression_parity passed POSIX routing still invokes O_RDONLY | O_NOFOLLOW, performs the existing bounded read and never calls the Windows opener. This is simulated branch coverage on Windows; no native POSIX run is claimed.
static partial Ruff 0.15.22 passes on both changed Python files; git diff --check passes. Native strict mypy reports the same existing os.O_NOFOLLOW attribute error in both the original module and the follow-up (line 196 before, line 227 after); no typecheck pass is claimed. The root mypy allowlist does not include this optional package.
  • Package failures: test_deletion_mode_and_no_final_newline_remain_visible expects Unix executable bits; test_private_file_permissions_and_secret_like_scope_rejection expects mode 0600; test_evidence_only_work_is_not_dropped_as_no_code_delta expects LF from a Windows text-written fixture. test_committed_constructed_snapshots_match_generator_byte_for_byte compares generated CRLF against committed LF, and test_replay_reproduces_the_committed_live_summary reports a recorded replay mismatch whose cause was not investigated in this capture repair.
  • Coverage and gaps: The regression suite checks regular/missing files, exact CRLF preservation, real 8.3 aliases, case and extended/device-prefix representations, file and directory symlinks, real junctions, reparse attributes, a same-file final-link race, directory pinning/release, a root-directory junction swap, identity mismatch before any read, zero identities and mutation during reading. This volume has 8.3 creation enabled; both short-name cases executed without skips. API failure/truncation is injected at the Win32 boundary, and ctime skew is forced at the stat boundary (this host's ordinary-file lstat/fstat ctimes matched). The existing package/shared failures and native mypy error remain unresolved. Native POSIX, other Python versions, Windows ACLs, network shares and filesystems without stable IDs were not qualified. Capture fails closed when canonical names or identity are unavailable, or sharing prevents secure capture.
  • Operational boundary: During the short capture window, held directory handles deny write/delete sharing to pin traversal. Concurrent directory mutation can conflict with those handles; the observer does not fall back to weaker checks.

Frontend / Visual Evidence

  • UI impact: none.
  • Before: N/A; existing optional provider CLI capture fails on Windows.
  • After: Existing CLI performs scoped local capture and exposes the existing status/receipt readback.
  • First viewport and preview: N/A; no visual surface changed.
  • States and viewports shown: N/A.
  • Source data: synthetic.
  • Attention review: N/A; no UI composition changed.
  • Craft review: N/A; no visual change.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring (no functional changes)
  • Documentation update
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)
  • Benchmark boundary (adapters, runners, verifiers, scoring, evidence)
  • Capability or extension (providers, adapters, skills)
  • Public docs or presentation surface (README, protocols, dashboard)
  • Build, packaging, installer, or CI
  • Host or runtime integration

Technical Direction

  • Direction / acceptance reference, when applicable: Existing optional Jev progress observer; bounded Windows IO compatibility and capture-boundary repair. No new roadmap claim or capability.

Shared-authority RFC fixture impact

  • Production-scale fixture schema: N/A.
  • Semantic dimensions changed, or reviewed no-impact rationale: No authority, lifecycle, receipt schema or provider-selection behavior changed.
  • Provider conformance arms run: Existing relevant shared File/SQLite fixture tests were attempted; results and limits are in Validation.
  • Read-only legacy/file/PostgreSQL three-arm rehearsal: N/A; no store, promotion or runtime-routing change.

Boundary Checklist

  • Neither the diff nor this PR body/comments/attachments disclose private state, credentials, raw traces or verifier output, internal links, or local machine paths.
  • I did not duplicate maintainer-owned benchmark work.
  • I kept the change scoped to the task.
  • No paired English/Chinese RFC changed.
  • UI impact is marked none.
  • Every commit includes a DCO Signed-off-by trailer.

🤖 Generated with Claude Code

Signed-off-by: xiaobaifly7 <196511160+xiaobaifly7@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · max

本次评审对应提交 042e315368b5e9b30b985132353daa450bfc234d。

[P2] 新 Windows 采集分支还需按规范路径核验范围。

动机 / Motivation

在 Windows 或 POSIX 上运行可选 Jev drift wrapper 的操作者。主干在没有 O_NOFOLLOW 的 Windows 上显式不可用;macOS/Linux 继续沿既有 O_NOFOLLOW。原 POSIX 路径继续可用,实际 wheel 与合成 File authority 验证本地 baseline、真实 refresh 和 egress_denied 回执。新 Windows 实现保留 reparse 与身份检查;范围规范化仍需要补足。独立运行环境为 macOS。Windows 路径对照模拟 OS 命名和属性边界,未运行 Win32/NTFS、ACL 或远端 CI;本机 Windows 正常和现有 race 结果引用作者声明,不当作独立验证。Windows 分支只按输入拼写检查被排除的路径,规范路径目前只用于判断是否仍在工作区内。用等价路径解析的隔离对照可到达实际读取;作者的本机用例覆盖 reparse、替换和权限冲突,尚未覆盖这个规范路径边界。

改动思路 / Approach

Windows 必须用现有 capture owner 内的平台 IO,保留现有有界读取和原 Git/receipt 逻辑。 分支补足原生打开能力,但 scope 必须约束已打开的对象,不应只验证输入字符;两种语言的运行说明也要跟上。 评审依据 packages/loopx-jev/DRIFT_SHADOW.md 的 scoped files 合同。继续用同一个可选 Jev owner 采集文件。Windows 的句柄校验要同时守住路径范围;微软的路径命名说明 明确一个对象可以有多种路径表示。

具体改动 / Changes

关键代码讲解

POSIX-OFF-PARITY:精确 04958a89abbbf6b7d3f79327437e2d6deae97453 与 042e315368b5e9b30b985132353daa450bfc234d 的 wheel 都在 macOS 实跑 init → refresh → drain → status,并写回本地 not_evaluated/egress_denied;原 refresh 记录不变,未启用 observer 时不产生观察目录。HANDLE-IDENTITY:新分支对目录句柄、最终文件和 reparse 属性做校验。作者的本机 Windows 验证覆盖了普通文件、链接、句柄冲突和置换;我单独复跑了 portable 用例和实际 POSIX 路径。

对主干的风险 / Risks

CANONICAL-SCOPE:validate_paths 仅对输入路径的文本组件执行排除规则;resolve() 的结果只用于确认仍在工作区内。文件句柄的身份一致也不能说明其规范路径符合范围。在模拟 Windows 名称解析、使用真实 capture 和 Git 的隔离对照中,规范对象相同的两种表示给出不同结果。这个观察没有执行 Win32 或 NTFS。请在读前把同一范围规则施加到已绑定身份的规范路径,补本机 Windows 负例和普通文件对照。两份运行指南还将 Windows 写作未验证,需要与实际支持范围同步。

语义与 CI 对齐

macOS 主干 126 项通过,新头 143 项通过、4 个 Windows-only 用例跳过。两个精确 wheel 的实际 CLI、POSIX 拒绝用例、Ruff 与标准预检通过。作者说明中的 Windows 包/共享用例失败仍保留,macOS 结果不能为它们归因或视作通过;没有查询远端 CI。

我的整体评价 / Verdict

请求修改。保留现有 handle identity 与 POSIX 边界,补规范路径的统一范围校验及 Windows 本机验收。这是已有可选 Jev CLI 的平台采集路径,未改变视觉界面。未验生产安装、原生 Windows ACL 或真实模型。

English verdict: REQUEST_CHANGES. Validate the canonical opened path against the same capture scope, then demonstrate the rejection on native Windows. The packaged POSIX path and default-off behavior passed independently.

or path.is_absolute()
or ".." in path.parts
or ".git" in path.parts
or ".git" in parts

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 请把这个排除规则用于实际打开对象的规范路径。当前 resolve() 只检查是否仍在工作区;两种 Windows 路径表示可能指向同一对象,但只有原始字符串受规则约束。隔离的名称解析对照得到不同采集结果。请加原生 Windows 负例,确认两种表示都在读取前拒绝,并保留普通文件正例。

Signed-off-by: xiaobaifly7 <196511160+xiaobaifly7@users.noreply.github.com>
@xiaobaifly7

Copy link
Copy Markdown
Contributor Author

Addressed CANONICAL-SCOPE in drift_capture.py. Before reading, Windows now obtains normalized paths from the identity-checked workspace and file handles with GetFinalPathNameByHandleW, makes the file path workspace-relative, and applies the existing scope validator. Failed or incomplete canonical-path queries stop capture. Existing no-follow, reparse, directory-pinning and identity checks remain in place.

test_windows_rejects_native_git_path_representations proves the finding on native Windows: the fixture creates a real GIT~1 alias for .git, verifies both spellings identify the same file, and traps any attempted read. Both short-name variants reached that trap before the fix and now reject with invalid_scope_path. Long names, mixed case and extended/device prefixes are also rejected before reading. 8.3 creation is enabled on the tested volume; neither alias case was skipped.

test_windows_fails_closed_without_final_path injects API failure and insufficient-buffer results at the Win32 boundary. Ordinary regular-file capture and the stat-boundary ctime-skew positive pass. The native capture suite is 30 passed, no skips; the isolated real CLI init/refresh/drain/status journey also passes with zero capture failures and no provider dispatch.

Both run guides now describe Windows support and its limits. The combined package/shared run changes from 141 passed / 10 failed / 4 errors to 146 passed / 6 failed / 4 errors: the four new negative failures are fixed and the added positive passes. The six existing failures, four SQLite setup errors and existing native mypy os.O_NOFOLLOW error remain disclosed in the PR draft.

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

复审提交:be915c2b2466a7a52c4b3ab91c0c04d3b3aad477。
[P2] 缺失文件的 Git 元数据别名仍可写入观察范围。 缺失分支

动机 / Motivation

在 Windows 上用 Jev 观察所选工作区文件的开发者。 上次发现的已有文件别名现在会在读取前拒绝。但如果经同一目录别名选择的文件尚未创建,初始化会接受这个范围,文件一出现,观察器就无法形成这一轮证据。 注入 Windows 的路径映射并执行真实 Jev 和 LoopX CLI 时,缺失元数据文件的别名会保存为有效范围。文件出现后,原刷新仍写入,观察器报 capture_failed、清空基线且没有事件。 独立验证在 macOS,注入 Windows 路径及文件身份观察。作者另外报告原生 NTFS 结果。本轮未独立运行原生 Windows、ACL 或网络共享,也未查询 CI。

让初始化和后续采集对同一 Git 元数据父目录使用同一范围规则。 范围合同 允许未来出现的普通文件。更新后的 Git 元数据排除 仍需覆盖已知父目录。 核验项:opened_alias_is_rejected_before_read, excluded_scope_cannot_be_initialized。

改动思路 / Approach

沿用现有 capture、范围校验和 Windows 句柄,不引入第二套范围或证据状态。现有元数据拒绝应同时覆盖已知父目录。 保留 POSIX 路径和可选观察器边界,把修复收在 Windows 路径校验。 GetFinalPathNameByHandleW 支持已打开目录和文件。既有代码已经取得父目录句柄,缺失分支可以先检查规范位置再记录缺失。 Microsoft 文档

具体改动 / Changes

  • Windows 捕获 已存在的对象在读取前核验规范路径。上一提交对已有别名返回 0 并读取了合成文件。当前返回 2,不写基线且未调用 read。
  • 初始化 已有的 Git 元数据目录通过短名选中时,目标尚未创建会得到 baseline_created。目标出现后,源刷新成功,Jev 报 capture_failed、清空基线且无事件。没有读取被排除的字节。
  • 英文指南、中文指南 已说明 Windows 的验证边界。真实 Jev CLI 接受一个尚未创建的普通文件。创建后产生一个本地事件,关闭外发时记录 not_evaluated / egress_denied。 PR 基点 macOS 包测试 93 项通过,当前提交 111 项通过及 12 个原生 Windows 项跳过;两边已有真实 refresh 和关闭状态用例都通过。 三个独立 CLI 用例还验证了当前已存在别名、缺失别名和正常缺失文件。

对主干的风险 / Risks

影响的是观察器证据。已知排除范围初始化成功后,常规刷新仍可能追加 Core 运行记录,而该观察器的基线被清空。原 POSIX 和关闭状态用例保持通过。Ruff、差异检查、签署、双语 RFC 和当前主干合入模拟通过。观察器初始化后范围固定,后续的 Core 刷新不会替换其中错误的文件选择。操作者需另建观察范围才能避免同一故障。

我的整体评价 / Verdict

请求修改。初始化应在已有父目录可核验时拒绝该范围,避免把不可用的选择保存到后续轮次。 在把文件记为缺失前,取最深已打开父目录句柄的规范相对路径,使用现有范围规则检查。保留普通未创建文件的正例。 建议在同一项原生 NTFS 验证中确认前述拒绝和普通路径恢复。

English verdict: REQUEST_CHANGES. Head be915c2. The existing alias is rejected before read. A missing leaf under an already excluded Git directory alias still initializes; when it appears, a real refresh succeeds but Jev loses its baseline and emits no event. Validate the canonical existing parent before accepting absence.

try:
observed = os.lstat(target)
except FileNotFoundError:
return None

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 已有的 Git 元数据目录通过短名选中时,目标尚未创建会得到 baseline_created。目标出现后,源刷新成功,Jev 报 capture_failed、清空基线且无事件。没有读取被排除的字节。 在把文件记为缺失前,取最深已打开父目录句柄的规范相对路径,使用现有范围规则检查。保留普通未创建文件的正例。 原生 NTFS 用真实短名选中一个不存在的 .git 子项,init 应失败且无状态。用相同方法选择普通目录的缺失文件,创建后仍产生事件。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants