Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions packages/loopx-jev/DRIFT_SHADOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,11 @@ baseline. Changes outside the list may still be valid Agent work. `no_delta`
means no change in the observed material, not no progress on the whole task.
Reading a test report does not run the test or independently certify its claim.

Git metadata (`.git` at any path component) is excluded. On Windows this rule
also applies to the normalized, workspace-relative path obtained from the opened
file handle, so case variants and 8.3 aliases cannot bypass it. Capture fails
closed if the canonical handle path cannot be obtained.

For example, selecting only a function's file may omit the helper it calls and
the test that exercises it. A judgment from that packet cannot certify the full
behavior. Include relevant tests, results and dependencies deliberately; if the
Expand Down Expand Up @@ -112,9 +117,13 @@ certify model correctness or grant mutation authority.

## Run it

Use Python 3.11+ and the Node runtime required by the LoopX checkout. This
scoped collector targets POSIX file handling on Linux/macOS; Windows capture is
not qualified and unsupported file primitives produce an unavailable observation.
Use Python 3.11+ and the Node runtime required by the LoopX checkout. The
scoped collector uses POSIX no-follow file handling on Linux/macOS and native
handle, reparse-point, identity and canonical-path checks on Windows. Native
Windows tests cover local NTFS capture; ACL-restricted and network-share
behavior is not qualified. Unavailable file primitives or canonical paths
produce an unavailable observation. Directory handles stay pinned during capture
and can conflict with concurrent write/rename operations.
The optional distribution is not part of the default LoopX wheel. Install from
the source root into a fresh environment, then run the no-key checks:

Expand All @@ -125,6 +134,9 @@ uv pip install --python .venv-jev/bin/python -e '.[test]' -e packages/loopx-jev
.venv-jev/bin/loopx-jev drift --help
```

On Windows, use `.venv-jev/Scripts/python.exe` and
`.venv-jev/Scripts/loopx-jev.exe` in place of the `bin` entrypoints above.

This implementation only observes task progress; other assessment directions
and ranking code are not provided. Use `loopx_jev_drift_config_v0` and `minimum_label_probability`; old
multi-direction pilot profiles are rejected rather than silently promoted.
Expand Down
6 changes: 5 additions & 1 deletion packages/loopx-jev/DRIFT_SHADOW.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@

路径必须是文件,不是目录或通配符;最多选 32 个,并受下文的字节上限约束。初始化后清单固定;要变更范围,需显式新建观察器/预算并建立新基线。Agent 在清单外做的工作可能完全有效,`no_delta` 只表示所观察材料没有变化,不表示整个任务没有进展。读取测试报告也不等于执行测试或独立认证其中的声明。

路径任何层级的 Git 元数据目录(`.git`)均被排除。在 Windows 上,同一规则也检查从已打开文件句柄取得的、相对工作区的规范路径,大小写变体和 8.3 别名不能绕过。无法取得规范句柄路径时,采集直接拒绝。

例如只选择函数所在文件,而漏掉被调用的 helper 和对应测试,评估就不能证明完整行为。应主动纳入相关测试、结果和依赖;必要材料装不下时,不能把部分材料包装成完整证据。

## 绑定规范任务 criterion
Expand Down Expand Up @@ -64,7 +66,7 @@ binding 固定 Goal/Agent/Todo 身份、contract 修订/摘要、任务语义

## 操作方法

使用 Python 3.11+ 和 LoopX 检出要求的 Node 运行时。采集器目前面向 Linux/macOS 的 POSIX 文件处理;Windows 采集未验证,缺少所需文件原语时记录观察不可用。此可选发行包不进入 LoopX 默认 wheel;从源码根目录在新环境中安装,再执行不需要 key 的集成测试:
使用 Python 3.11+ 和 LoopX 检出要求的 Node 运行时。采集器在 Linux/macOS 上沿用 POSIX no-follow 文件处理,在 Windows 上使用原生句柄、reparse point、身份和规范路径检查。原生 Windows 测试覆盖本地 NTFS 采集;受限 ACL 和网络共享场景尚未验证。缺少所需文件原语或无法取得规范路径时,记录观察不可用。采集期间目录句柄保持固定,可能与并发写入或重命名冲突。此可选发行包不进入 LoopX 默认 wheel;从源码根目录在新环境中安装,再执行不需要 key 的集成测试:

```bash
uv venv .venv-jev
Expand All @@ -73,6 +75,8 @@ uv pip install --python .venv-jev/bin/python -e '.[test]' -e packages/loopx-jev
.venv-jev/bin/loopx-jev drift --help
```

Windows 上将上述 `bin` 入口替换为 `.venv-jev/Scripts/python.exe` 和 `.venv-jev/Scripts/loopx-jev.exe`。

本切片只观察任务进展,不提供其他评估方向或排序代码。使用 `loopx_jev_drift_config_v0` 和 `minimum_label_probability`;旧多方向试点配置会被拒绝,不会静默提升。配置 key 本身不启用 shadow 或允许出站。失败时原 Agent 工作继续,不会自动启动独立 Agent 裁判。

为已有 Goal 创建被 Git 忽略的本地目录,复制 [config.shadow.json](examples/drift/config.shadow.json) 和 [basis.json](examples/drift/basis.json)。将示例 Goal id、目标、验收条件改为本次契约。可选 `evidence` 引用交付工作区中的常规文件,例如独立产生的测试报告;不要在配置或契约中填写密钥。默认 `allow_egress: false`,确认指定材料允许出站后再设为 true,并在**消费者的环境变量**中配置 `TYPESAFE_API_KEY`。
Expand Down
172 changes: 159 additions & 13 deletions packages/loopx-jev/src/loopx_jev/drift_capture.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@
import difflib
import hashlib
import os
from pathlib import Path
import stat
import subprocess
import tempfile
from typing import Any
from contextlib import ExitStack
from pathlib import Path
from typing import Any, BinaryIO

from .protocol import request_bytes

Expand Down Expand Up @@ -43,11 +45,16 @@ def validate_paths(repo: Path, paths: list[str]) -> list[str]:
raise ValueError("scope_requires_one_to_32_unique_files")
for name in paths:
path = Path(name)
parts = (
tuple(part.casefold() for part in path.parts)
if os.name == "nt"
else path.parts
)
if (
not name
or path.is_absolute()
or ".." in path.parts
or ".git" in path.parts
or ".git" in parts

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 请把这个排除规则用于实际打开对象的规范路径。当前 resolve() 只检查是否仍在工作区;两种 Windows 路径表示可能指向同一对象,但只有原始字符串受规则约束。隔离的名称解析对照得到不同采集结果。请加原生 Windows 负例,确认两种表示都在读取前拒绝,并保留普通文件正例。

or name != path.as_posix()
or any(ord(char) < 32 for char in name)
):
Expand All @@ -58,8 +65,138 @@ def validate_paths(repo: Path, paths: list[str]) -> list[str]:
return sorted(paths)


def _windows_identity(
info: os.stat_result,
*,
directory: bool = False,
) -> tuple[int, int, int, int]:
attributes = getattr(info, "st_file_attributes", None)
if attributes is None:
raise ValueError("unsupported_capture_platform")
if stat.S_ISLNK(info.st_mode) or attributes & stat.FILE_ATTRIBUTE_REPARSE_POINT:
raise ValueError("scope_escapes_workspace")
if not (stat.S_ISDIR if directory else stat.S_ISREG)(info.st_mode):
raise ValueError("scope_requires_regular_files")
if not info.st_dev or not info.st_ino:
raise ValueError("capture_identity_unavailable")
return info.st_dev, info.st_ino, info.st_size, info.st_mtime_ns


def _windows_descriptor(target: Path, *, directory: bool = False) -> int:
import ctypes
import msvcrt
from ctypes import wintypes

kernel = ctypes.WinDLL("kernel32", use_last_error=True)
create = kernel.CreateFileW
create.argtypes = (
wintypes.LPCWSTR,
wintypes.DWORD,
wintypes.DWORD,
wintypes.LPVOID,
wintypes.DWORD,
wintypes.DWORD,
wintypes.HANDLE,
)
create.restype = wintypes.HANDLE
kernel.CloseHandle.argtypes = (wintypes.HANDLE,)
kernel.CloseHandle.restype = wintypes.BOOL
filename = os.path.abspath(target)
if not filename.startswith("\\\\?\\"):
filename = (
"\\\\?\\UNC\\" + filename[2:]
if filename.startswith("\\\\")
else "\\\\?\\" + filename
)
# Directories deny write/delete sharing to pin traversal; OPEN_REPARSE_POINT
# opens a raced-in link itself. BACKUP_SEMANTICS permits directory handles.
flags = 0x00200000 | (0x02000000 if directory else 0)
handle = create(
filename, 0x80000000, 0x1 if directory else 0x7, None, 3, flags, None
)
if handle == wintypes.HANDLE(-1).value:
raise ctypes.WinError(ctypes.get_last_error())
try:
return msvcrt.open_osfhandle(handle, os.O_RDONLY | os.O_BINARY | os.O_NOINHERIT)
except BaseException:
kernel.CloseHandle(handle)
raise


def _windows_open(target: Path) -> BinaryIO:
descriptor = _windows_descriptor(target)
try:
return os.fdopen(descriptor, "rb")
except BaseException:
os.close(descriptor)
raise


def _windows_final_path(descriptor: int) -> Path:
import ctypes
import msvcrt
from ctypes import wintypes

kernel = ctypes.WinDLL("kernel32", use_last_error=True)
final_path = kernel.GetFinalPathNameByHandleW
final_path.argtypes = (
wintypes.HANDLE,
wintypes.LPWSTR,
wintypes.DWORD,
wintypes.DWORD,
)
final_path.restype = wintypes.DWORD
buffer = ctypes.create_unicode_buffer(32768)
# FILE_NAME_NORMALIZED | VOLUME_NAME_DOS keeps root and file names comparable.
length = final_path(msvcrt.get_osfhandle(descriptor), buffer, len(buffer), 0)
if not 0 < length < len(buffer):
raise ValueError("capture_path_unavailable")
return Path(buffer.value)


def _windows_capture(repo: Path, target: Path) -> tuple[bytes, int] | None:
with ExitStack() as parents:
current = repo
canonical_repo = repo
for part in ("", *target.relative_to(repo).parts[:-1]):
current = current / part
try:
expected = _windows_identity(os.lstat(current), directory=True)[:2]
except FileNotFoundError:
return None
descriptor = _windows_descriptor(current, directory=True)
parents.callback(os.close, descriptor)
if _windows_identity(os.fstat(descriptor), directory=True)[:2] != expected:
raise ValueError("workspace_changed_during_capture")
if not part:
canonical_repo = _windows_final_path(descriptor)
try:
observed = os.lstat(target)
except FileNotFoundError:
return None

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 已有的 Git 元数据目录通过短名选中时,目标尚未创建会得到 baseline_created。目标出现后,源刷新成功,Jev 报 capture_failed、清空基线且无事件。没有读取被排除的字节。 在把文件记为缺失前,取最深已打开父目录句柄的规范相对路径,使用现有范围规则检查。保留普通未创建文件的正例。 原生 NTFS 用真实短名选中一个不存在的 .git 子项,init 应失败且无状态。用相同方法选择普通目录的缺失文件,创建后仍产生事件。

expected_file = _windows_identity(observed)
with _windows_open(target) as stream:
if _windows_identity(os.fstat(stream.fileno())) != expected_file:
raise ValueError("workspace_changed_during_capture")
canonical = _windows_final_path(stream.fileno())
if not canonical.is_relative_to(canonical_repo):
raise ValueError("scope_escapes_workspace")
validate_paths(
canonical_repo, [canonical.relative_to(canonical_repo).as_posix()]
)
raw = stream.read(MAX_BYTES + 1)
opened = os.fstat(stream.fileno())
if (
_windows_identity(opened) != expected_file
or _windows_identity(os.lstat(target)) != expected_file
):
raise ValueError("workspace_changed_during_capture")
return raw, opened.st_mode & 0o111


def capture(repo: Path, paths: list[str]) -> dict[str, Any]:
if not hasattr(os, "O_NOFOLLOW"):
windows = os.name == "nt" and not hasattr(os, "O_NOFOLLOW")
if not windows and not hasattr(os, "O_NOFOLLOW"):
raise ValueError("unsupported_capture_platform")
repo = repo.resolve()
paths = validate_paths(repo, paths)
Expand All @@ -73,15 +210,24 @@ def capture(repo: Path, paths: list[str]) -> dict[str, Any]:
target = repo / name
if target.is_symlink() or not target.resolve().is_relative_to(repo):
raise ValueError("scope_escapes_workspace")
if not target.exists():
files[name] = None
continue
if not target.is_file():
raise ValueError("scope_requires_regular_files")
# O_NOFOLLOW also rejects a final-component symlink introduced after check.
with os.fdopen(os.open(target, os.O_RDONLY | os.O_NOFOLLOW), "rb") as stream:
raw = stream.read(MAX_BYTES + 1)
mode = os.fstat(stream.fileno()).st_mode & 0o111
if windows:
captured = _windows_capture(repo, target)
if captured is None:
files[name] = None
continue
raw, mode = captured
else:
if not target.exists():
files[name] = None
continue
if not target.is_file():
raise ValueError("scope_requires_regular_files")
# O_NOFOLLOW also rejects a final-component symlink introduced after check.
with os.fdopen(
os.open(target, os.O_RDONLY | os.O_NOFOLLOW), "rb"
) as stream:
raw = stream.read(MAX_BYTES + 1)
mode = os.fstat(stream.fileno()).st_mode & 0o111
size += len(raw)
if size > MAX_BYTES or b"\0" in raw:
raise ValueError("oversized_or_binary_scope")
Expand Down
Loading
Loading