Skip to content

fix(canary): preserve exact changed Git paths - #8585

Merged
huangruiteng merged 4 commits into
loopx-project:mainfrom
gcl-coder:fix/canary-exact-git-paths
Nov 11, 2026
Merged

huangruiteng merged 4 commits into
loopx-project:mainfrom
gcl-coder:fix/canary-exact-git-paths

Conversation

@gcl-coder

@gcl-coder gcl-coder commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The canary changed-file selector used Git's NUL framing but still decoded path captures through text mode. Python newline translation changed a literal carriage return in a valid pathname into LF, and path deduplication stripped valid leading and trailing spaces.

The first repair preserved raw UTF-8 paths, but displayed undecodable records with repr(). A real Git tree containing raw path bytes ff2e7079 and the literal ASCII pathname b'\\xff.py' then produced one display value for two distinct Git objects. The public selector silently dropped one path and still returned ok=true.

Premerge repeated the same loss after selection: its dedupe normalized literal backslashes to separators and stripped trailing spaces, so a five-path Git diff reached the gate and catalog as four rewritten paths.

Change

  • Read Git name-only output as bytes and decode valid UTF-8 pathnames without newline translation or trimming.
  • Reject undecodable Git pathnames instead of manufacturing a colliding display string. The selector reports ok=false, undecodable_path_count, an explicit UTF-8 error, and no partial changed-file list.
  • Stop plan, run, smoke-suite, and premerge before selection or check execution whenever Git path collection is incomplete.
  • Preserve exact Git path identities through premerge. Deduplication no longer rewrites values; normalization remains only for surface matching. Explicit --changed-file inputs keep the existing slash normalization at the premerge CLI boundary.

Validation

  • python3 -m pytest tests/canary/test_git_path_framing.py -q -> 7 passed.
  • The selector regression creates a real Git repository with both raw \\xff.py bytes and the literal ASCII name b'\\xff.py', proves Git emits two NUL records, and runs the public canary plan --from-git-diff entry. The command exits nonzero with the UTF-8 error and does not emit selection_inputs.
  • The premerge regression creates six real Git paths: a literal backslash name, a CR name, a leading space, fixtures/trailing.py, fixtures/trailing.py , and a docs/ boundary path. It asserts the selector, premerge top-level list, classification, catalog selection inputs, and boundary scan inputs keep all six exact names. The regression fails on the previous head with docs/back/slash.txt and a merged trailing-space path.
  • python3 -m pytest tests/canary/test_repository_root.py tests/canary/test_premerge_public_boundary_run.py -q -> 30 passed.
  • ruff check and git diff --check pass.
  • Remote CI is still queued; its final status will be reported separately.

The canary selector captured `--name-only -z` with text mode. Python's
universal-newline translation changed a literal CR in a Git pathname into
LF, while the common dedupe helper stripped valid leading and trailing
filename spaces. The selector then reported names that did not exist and
could miss the checks for the changed files.

Keep Git output framed as bytes, decode valid UTF-8 losslessly, and retain
raw display identities for invalid byte names. Deduplicate derived names
without rewriting them; explicit --changed-file input keeps its existing
normalisation.

Signed-off-by: ChengLong GU <88359731+gcl-coder@users.noreply.github.com>

@songoow songoow left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · GPT-6 Astra Pro · OpenAI · self_reported

评审 head:90ae3497331fa12d3e4fede8dd642dccf6798500;固定比较基线:beff0f3e55d1f70eceebbd8e78c11f1cff34643b。

动机 / Motivation

通过 canary plan/run/premerge --from-git-diff 选择检查范围的开发者与自动化。

原选择器虽用 NUL 分隔 Git 路径,但 text=True 会把文件名中的 CR 改成 LF,后续 strip 又丢掉首尾空格;替换解码还会合并不同的非法 UTF-8 字节路径。

在 Git 输出、去重和公开 selector 输入之间保留每个不同路径的身份,同时保留显式 --changed-file 的既有规范化。

本次范围是 Git 路径读取和去重,不增加新的检查执行权、catalog、状态存储或全仓扫描策略。

改动思路 / Approach

继续在现有 Git 输入边界使用字节模式,并为 Git 来源保留精确去重,这个修复位置合理。问题在于展示编码被当成身份,需在该局部边界完成无碰撞表示,无需另建 catalog 或状态 owner。 限定 Git pathname collection 的可逆身份语义;不以本轮路径检查宣称所有 canary 执行器已验收。

-z 继续定义记录边界;读取 bytes 避免 universal-newline translation;显式 --changed-file 先走旧规范化,Git 来源单独精确去重。共享 runner 既有字节路径读取也说明无需以展示值承担文件身份。

具体改动 / Changes

完整检查2个文件及 plan/run/premerge 消费链。_display_git_path 在 UTF-8 解码失败时使用 repr;收集器与 CLI 汇总改用精确字符串去重。新增真实 Git 测试覆盖 CR、前导与尾随空格,4项路径测试全部通过。

额外用真实临时 Git 仓库,经两个完整 checkout 的公开 canary plan 执行三组对照,共6次 CLI 调用:CR/空格在head保留;FF/FE两个非法UTF-8路径在head不再被替换字符合并;但非法字节路径与其普通ASCII repr字面名发生新的碰撞。没有 mock Git 输出。基线beff、PR实际base7382与本轮已读取主干cf807的两个差异文件逐字节相同,生产路径来源和哈希已核对。

接受依据为现有源码验证规范与既有one-path/one-record测试,spec_revision=beff0f3e55d1f70eceebbd8e78c11f1cff34643b,criterion_id=existing_boundary:真实 --from-git-diff 入口不得在选择前悄悄合并两个不同的Git对象。

对主干的风险 / Risks

[P2] 混合 UTF-8/repr 表示会静默丢掉一个合法的 Git 路径。

复现使用原始路径字节 ff2e7079 与 62275c7866662e707927。两次 Git 输出均有2条 NUL 记录;固定基线公开 CLI 保留2条,head 的 selector_sources.git_diff 和 selection_inputs.changed_files 只剩1条,仍返回 ok=true。这个现象直接破坏变更集合身份;本轮没有进一步断言某个具体 catalog 检查已经漏跑。

将 Git 路径身份与展示字符串分离,使用对所有字节路径都可逆且无碰撞的内部表示,去重后才进行展示;若下游不支持某类字节路径,应明确拒绝而不是返回 ok=true 的不完整集合。保留本PR已修复的CR、空格以及不同非法字节路径对照。

与已读取主干cf807的合并演练无文本冲突,主干未改动该选择器,不能靠当前合并消除碰撞。仅验证本地实际 Git、源码 CLI 和相关路径测试;没有运行完整CI、安装包或其他操作系统文件名边界。没有页面布局变化。

我的整体评价 / Verdict

建议 REQUEST_CHANGES。字节读取和 Git 来源不做 strip 的方向正确,正向修复已独立确认;但新增表示必须在这个PR内完成无碰撞身份传递,避免“保留精确路径”的改动引入新的静默去重。

English verdict: REQUEST_CHANGES. Preserve distinct byte-path identities before rendering; the mixed UTF-8/repr mapping currently collapses two real Git paths in the public selector.

Comment thread loopx/cli_commands/canary.py Outdated
return record.decode("utf-8")
except UnicodeDecodeError:
return repr(record)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Keep byte-path identity separate from its display representation

The UTF-8 and repr(record) branches can return the same string for different Git paths. I reproduced this with a real repository containing pathname bytes ff2e7079 (FF followed by .py) and 62275c7866662e707927 (the ASCII literal b'\xff.py'). Git emits two NUL records, but both now become the latter string and _dedupe_exact_paths drops one.

Through the actual canary plan --from-git-diff --git-diff-base HEAD~1 entrypoint, the baseline reports changed_file_count=2; this head reports ok=true, count1, and only one selection_inputs.changed_files entry. This is a new collision even though the CR/space fixes and the distinct-FF/FE-byte control work.

Please retain a lossless, non-colliding path identity for collection/deduplication and reserve repr for display, or explicitly reject unsupported byte paths. Add the mixed invalid-byte/literal-repr case through the public selector so a valid change cannot silently disappear.

@gcl-coder gcl-coder Nov 11, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修复并更新到 72e0bca。

采用明确拒绝不可表示字节路径的方案:Git 路径在 NUL 分帧后严格按 UTF-8 解码,任何不可解码记录都会让 selector 返回 ok=false、undecodable_path_count 和明确错误,不再用 repr() 参与去重或伪装成可查询路径;plan/run/smoke-suite/premerge 也会在构建选择或执行检查前停止。

新增真实 Git 反例同时包含原始 \xff.py 字节和 ASCII 字面名 b'\\xff.py',断言两条 Git 记录可区分、去重不再发生,并验证公开 canary plan --from-git-diff 返回非零且不输出 selection_inputs。tests/canary/test_git_path_framing.py 5 项通过,Ruff 和 git diff --check 通过。

Signed-off-by: ChengLong GU <88359731+gcl-coder@users.noreply.github.com>
@mergify

mergify Bot commented Nov 11, 2026

Copy link
Copy Markdown

Hi @gcl-coder, the DCO Sign-off check did not pass. Please inspect
its details first: checkout, fetch, timeout or infrastructure errors
need their own recovery, not a rewrite of otherwise signed commits.

If the log confirms a missing Signed-off-by trailer, amend the
affected commit with git commit --amend -s; for multiple commits,
use an interactive rebase against the current base from the correct
base-repository remote and sign off each affected commit. Push the
rewritten PR branch with git push --force-with-lease origin HEAD.

@gcl-coder
gcl-coder force-pushed the fix/canary-exact-git-paths branch from 9a3bd80 to 72e0bca Compare November 11, 2026 09:59
@gcl-coder
gcl-coder requested a review from songoow November 11, 2026 10:08

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

评审提交:72e0bca67d45873dfb1db7f5a493150bfda9dd1f。比较基线:c49d59b24d3f7745fccf3a1f2df1e35036fe517b。结论:REQUEST_CHANGES。

P2:premerge 仍会丢掉合法 Git 路径。 同一个 Git 夹具中,selector 报告 5 个合法路径,premerge --no-execute 却成功返回 4 个。fixtures/trailing.py 和尾部多一个空格的同名路径合并,且字面反斜杠被改成斜杠;下游 catalog 也收到改写后的集合。

动机 / Motivation

使用 canary --from-git-diff 选择本地变更检查的开发者。 旧 selector 使用 text mode 读取 Git,改写 CR,并把带空格的合法文件名去空格合并。 Git 的完整可用路径应以原身份进入每个检查消费者;无法安全表示的路径须明确失败。

精确基线的 canary plan 在同夹具中输出 4 个,改写 CR 并合并尾空格。新提交的 plan 保留全部 5 个;旧 premerge 的归一化仍会在进入验证门后再次丢失身份。 同一个 Git 夹具中,selector 报告 5 个合法路径,premerge --no-execute 却成功返回 4 个。fixtures/trailing.py 和尾部多一个空格的同名路径合并,且字面反斜杠被改成斜杠;下游 catalog 也收到改写后的集合。

不扩展 Canary 运行权限、调度或检查目录,不把手工参数兼容性当作 Git 文件身份。

改动思路 / Approach

Fix raw decode at the existing Git selector and retain identity at its current downstream consumer; extra selection policy is unnecessary. Keep current manual normalization for independent explicit --changed-file callers; preserve Git path identity separately at premerge.

评审边界:Git selector validation, exact filenames and truthful premerge readback

具体改动 / Changes

字节解码 现在保留 CR 和合法空格,也避免非法原始字节和字面转义名冲突。CLI 精确去重 后,premerge 的归一化 仍会重写原值。上轮提到的不可解码路径冲突已通过四个入口复验。

22 项定向原生 CLI 测试通过。独立 Git 仓库中,非法 UTF-8 与看似其转义表示的合法文件同时存在且另有已暂存合法路径;plan、premerge、run、smoke-suite 均退出 1,未开始选择。

依据 docs/development/testing-and-quality.md:

  • EXACT-GIT-PLAN:Preserve valid Git paths and reject undecodable records。22 项定向原生 CLI 测试通过。独立 Git 仓库中,非法 UTF-8 与看似其转义表示的合法文件同时存在且另有已暂存合法路径;plan、premerge、run、smoke-suite 均退出 1,未开始选择。
  • EXACT-PREMERGE-SELECTION:Pass the same exact changed Git paths through premerge planning。同一个 Git 夹具中,selector 报告 5 个合法路径,premerge --no-execute 却成功返回 4 个。fixtures/trailing.py 和尾部多一个空格的同名路径合并,且字面反斜杠被改成斜杠;下游 catalog 也收到改写后的集合。

对主干的风险 / Risks

精确基线的 canary plan 在同夹具中输出 4 个,改写 CR 并合并尾空格。新提交的 plan 保留全部 5 个;旧 premerge 的归一化仍会在进入验证门后再次丢失身份。 用真实 git diff --name-only -z 数据运行 loopx canary premerge --from-git-diff --no-execute,对照 selector、门禁和下游 catalog 的精确路径。 未执行整套 premerge 或查询托管 CI;负例在执行计划构建时就已丢失路径。Git 单个命令非解码失败的旧告警/回退不在本次证明范围。

我的整体评价 / Verdict

保留当前的严格解码修复,并补齐实际 premerge 路径。让 premerge 保留来自 Git 的原始身份,用原值做去重、读取和回显;将分类用的规范化表示留在匹配边界。给 premerge 增加两个合法路径只差尾空格的真 Git 回归用例,同时保留手工 --changed-file 的兼容行为。

English verdict: REQUEST_CHANGES at 72e0bca67d45873dfb1db7f5a493150bfda9dd1f. Invalid UTF-8 now fails closed, but premerge still rewrites valid Git path identities, drops a distinct trailing-space path and reports success. Preserve identity through the gate and catalog.

)
return _dedupe_preserving_order(changed_files), git_diff_selector
]
changed_files = _dedupe_exact_paths([*changed_files, *derived])

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 保留传入 premerge 的合法 Git 路径身份

这里的精确去重在 CLI 层是正确的,但这份列表进入 build_premerge_validation_gate 后会被 _dedupe / _norm_path 再次 strip 和替换反斜杠。真实 Git 对照中,同时存在 fixtures/trailing.py 和同名尾部多一个空格的文件时,selector 两个都列出,premerge --from-git-diff --no-execute 仍退出 0 且下游只收到一个;字面反斜杠也被当作目录分隔符。请让实际 gate/catalog 消费原始身份,单独保留分类的规范化,并让回归测试断言 premerge 自己消费的列表。

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已在 5d89a2431c7769cecb611aa07c71cf0d89e27c7a 修复。

premerge._dedupe 不再用 _norm_path 重写身份,原始 Git 路径直接进入 classification、catalog 选择、direct checks、risk profile 和 boundary 扫描;_path_matches 仍只把规范化用于分类匹配。显式 --changed-file 的反斜杠归一化保留在 premerge 的 CLI 入口,Git 来源路径不经过该步骤。

新增真实 Git 回归:同一夹具包含字面反斜杠、CR、前导空格、fixtures/trailing.py 与同名尾部多一个空格的文件,以及 docs/ 下的 boundary 路径。断言 selector、premerge 顶层 changed_files、classification、catalog_run.selection_inputs.changed_files、boundary_run.selection_inputs.changed_files 与 scan_paths 均为 6 条原名。该回归在旧 head 上会看到 docs/back/slash.txt 和尾部空格被合并,修复后通过。

本地验证:tests/canary/test_git_path_framing.py 7 passed,test_repository_root.py 与 test_premerge_public_boundary_run.py 30 passed,Ruff 与 git diff --check 通过。远端 CI 当前仍在排队,结果以实际检查状态为准。

Premerge deduplicated with _norm_path, which rewrote literal backslashes and stripped trailing spaces after selection. Keep Git-provided identities intact for classification, catalog selection, direct checks and boundary scans, while retaining path normalization only for surface matching.

Keep explicit --changed-file slash normalization for premerge callers. Add a real Git regression with six distinct paths, including a literal backslash, CR, leading space, and two names differing only by a trailing space.

Signed-off-by: ChengLong GU <88359731+gcl-coder@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

评审提交 5d89a24。

动机 / Motivation

使用 Canary 按 Git 变更选择验证的开发者。 同一个真实 Git 仓库有六个不同的合法路径。上一轮 premerge 只保留四个,还会把字面反斜杠名当成另一个目录内的文件。当前提交保留全部六个,边界扫描也读取了正确文件。 合法路径以原名送到 catalog 和扫描器,无法以 UTF-8 表示的路径在检查开始前得到明确错误。 本次只修复现有 Canary 的路径交接。 仍需在其他平台和完整发布验证流程观察,尤其是不同文件系统允许的文件名。

改动思路 / Approach

应在现有入口把文件名作为身份传递;只修正读取层会把同一错误留在下游。两个边界各处理自身职责即可。 现有 Canary 的 Git 路径采集、检查选择与实际文件读取。 现有 Git selector 读取 NUL 记录,premerge 的分类负责选检查,扫描器负责读取文件。

具体改动 / Changes

关键代码讲解

旧的 CR、空格与字节碰撞问题仍覆盖。相对上一次的 72e0bca,这次修复的是 premerge 的第二次归一化。全量比较基线为 c49d59b;当前目标分支 6c11d3e 在 Canary 相关代码和验证规范上与该基线相同。

37 项定向测试及 Ruff 通过。独立 Git 仓库在 premerge、catalog 与扫描输入均读回六个原名。 上一轮读取了同名目录里的正常文本,报告扫描通过。当前提交实际读到字面反斜杠文件,报告其无效 UTF-8。另一个夹具混合不可解码路径、合法转义形状名字和正常暂存项,四个入口都退出 1,且未返回检查计划。

依据 docs/development/testing-and-quality.md:

  • EXACT-GIT-PLAN 已实现。合法记录精确去重,无法解码时四个入口都停止。
  • EXACT-PREMERGE-SELECTION 已实现。catalog 与扫描器保留同一六条路径,实际读取命中此前漏掉的文件。

对主干的风险 / Risks

当前目标分支与候选在普通 Git 文件和只传 --changed-file 的输入上结果一致,手工反斜杠仍按历史规则处理。 原始路径的精确值与用于选择检查类型的规范化字符串各有一处用途。新增诊断和计数是现有 selector 的局部输出,不改变写入权限或引入共享状态。本地 POSIX Git 和源码 CLI。Git 单个来源失败时的旧告警路径、Windows 文件系统、整套 Canary 和托管 CI 未独立覆盖。

我的整体评价 / Verdict

我在 5d89a24 没有发现阻断项。既有路径修复和新增的下游读取证据相互吻合。先前两轮指出的路径碰撞与 premerge 路径丢失均已独立复验。对持续交付,预检会检查真实变更而不会误信别名;对开发者,回显及错误可用于定位问题。仍需在其他平台和完整发布验证流程观察,尤其是不同文件系统允许的文件名。

English verdict: APPROVE at 5d89a24. All six paths survive premerge and the real scan reads the previously missed file. All four entry points reject undecodable pathnames before planning.

…paths

Signed-off-by: ChengLong GU <88359731+gcl-coder@users.noreply.github.com>
@huangruiteng
huangruiteng merged commit 278f9b9 into loopx-project:main Nov 11, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants