Skip to content

fix(benchmark): identify missing checkpoint components - #8610

Open
loopx-agent wants to merge 1 commit into
mainfrom
codex/edgebench-checkpoint-missing-component-20261111
Open

loopx-agent wants to merge 1 commit into
mainfrom
codex/edgebench-checkpoint-missing-component-20261111

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

An EdgeBench checkpoint can lose a declared file or directory between worker-side preparation and Docker export. The supervisor reply and terminal preservation receipt reported only NotFound, leaving the original component ambiguous.

The existing capture path now carries the declared component ID into both failure receipts. The worker displays it only after binding the reply to the original request and matching the ID to a component in that request. The receipt carries the ID without the guest path or Docker response body.

  • Intended base is main.

Author Declaration

  • Written by: model_agent · OpenAI Codex.

Implemented against

Checkpoint capture and worker-barrier contract

Criterion Disposition Evidence
A missing declared path rejects that checkpoint Implemented Real Docker supervisor and terminal failure cases
Failure replies follow writer release and preserve the original capture identity Implemented Real Docker supervisor reply and direct capture tests
Worker replies contain bounded diagnostics Implemented Exact request readback accepts the declared ID and ignores an unrelated ID

Scope And Continuation

Complete within checkpoint failure diagnostics. The existing capture rejection, deadline, source-retention and scoring behavior applies. The terminal test confirms that failed sealing retains the stopped source and creates no extra grading opportunity.

Validation

Tested revision 6929a2447080be10ba33496f67fa2cf5a7040312. All executed checks below completed. Inputs were synthetic fixtures.

Check Result Evidence
Local regression suite Passed 101 tests across checkpoint handoff, diagnostics and terminal preservation. 16 backend-dependent cases were skipped by the local environment.
Real Docker and native HTTP Passed 6 selected cases using isolated containers and synthetic grading. Includes successful capture, missing source in supervisor and terminal paths, and preservation of a pre-existing pause.
Static and premerge checks Passed Ruff, Git diff hygiene, RFC pairing, Python compilation, three catalog canaries and public-boundary scan. The premerge gate reports the benchmark-sensitive manual review hold.

Delivery boundaries

  • UI impact is none.
  • Area is benchmark runner.
  • Shared-authority fixture impact is none.
  • Public-boundary scan passed. The change includes a DCO sign-off and targets the existing benchmark checkpoint contract.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes conclusion (author-owned PR; GitHub blocks formal self-review)

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · max

本次评审对应提交 6929a2447080be10ba33496f67fa2cf5a7040312。

[P2] 在 worker 端显示已核实的缺失组件 ID。 真实 Docker 归档一个路径名含 no such image 的缺失文件会返回 404。Docker SDK 将错误分类为 ImageNotFound。supervisor 和终态回执都带已声明的组件 ID,worker 只接受字面 NotFound,因此该次仍省略 ID。 条件分支

动机 / Motivation

排查 EdgeBench checkpoint 失败的操作员和在同一 native 边界等待回执的 worker。旧 supervisor 和终态只报 Docker 错误类别,操作员需要额外定位缺失项。常见 NotFound 的两个回执与 worker 消息均显示原请求中的组件 ID。原请求释放后回复,失败终态保留已停止的原容器。常见 NotFound 的两个回执与 worker 消息均显示原请求中的组件 ID。原请求释放后回复,失败终态保留已停止的原容器。隔离 Docker、实际 mailbox 和本地 native HTTP fixture 已验证。native 测试的评分为合成,正式 benchmark 收益和已安装环境待独立验收。真实 Docker 归档一个路径名含 no such image 的缺失文件会返回 404。Docker SDK 将错误分类为 ImageNotFound。supervisor 和终态回执都带已声明的组件 ID,worker 只接受字面 NotFound,因此该次仍省略 ID。

改动思路 / Approach

缺失组件语义属于归档动作的失败边界,worker 以现有 request digest 和成员证明关联性。保留 Python Docker adapter 和私有 mailbox 的当前责任,不扩大 native grading 或恢复权限。 组件 ID 来自已经校验的请求。两个宿主出口复用同一字段投影,worker 先核验 request_sha256,再核对组件成员。发送和读取对缺失错误的分类应覆盖同一组实际归档响应。

具体改动 / Changes

关键代码讲解

  • capture_worker 在 writer barrier 内对声明组件执行 Docker archive。缺失时记录对应 ID,并让原 cleanup 释放来源。两个宿主回执都通过 capture_failure_fields 序列化。 文件导出
  • 实际缺失路径的 HTTP 404 有两种可复现形状。普通路径得到 NotFound。路径含 no such image 时,docker-py 7.2.0 根据诊断正文将其归为 ImageNotFound。 SDK 分类
  • 我从同一个容器回读宿主与 guest 的相同回执。普通缺失时三处都有 declared_code。ImageNotFound 时两个宿主回执有 declared_code,worker 的异常文本只剩错误类别。 worker 回读

适用契约 benchmark/README.md,基线 43b4699。对应检查为 barrier_release_and_retention、bounded_failure_payload 和 same_request_component_visibility。 捕获契约

对主干的风险 / Risks

该缺口会让 worker 在真实归档缺失时仍无法确定要查哪个组件。两个容器回执验证了原摘要和预期 ID。真实 writer barrier 在回复前释放,终态仍停止并保留同一容器,组件路径和 Docker 响应正文没有进入回执。失败分支不会生成捕获 artifact。测试使用隔离 Docker 与合成评分,正式 benchmark 资格仍待验收。

语义与 CI 对齐

101 项本地测试通过,16 项因平台或依赖跳过。另跑了 7 项隔离 Docker 与 native HTTP 用例。独立前后对照经过真实容器、Docker SDK、宿主回执和 guest handoff,复现了子类漏报。Ruff、DCO、RFC 双语差异与 git diff --check 通过。

我的整体评价 / Verdict

回执字段和普通路径已有可核验证据。实际 SDK 子类仍使同一声明组件在 worker 端漏报,阻断诊断目标的完整交付。请统一回执语义并补上该真实归档用例,然后复审。本次评审覆盖现有宿主与 CLI 行为。隔离 Docker、实际 mailbox 和本地 native HTTP fixture 已验证。native 测试的评分为合成,正式 benchmark 收益和已安装环境待独立验收。

English verdict: REQUEST_CHANGES. A real Docker archive 404 can be classified as ImageNotFound. Both receipts carry the verified component ID. The worker discards it.

error_kind = result.get("error_kind")
message = f"controller checkpoint failed: {error_kind}"
missing = result.get("missing_component_id")
if (error_kind == "NotFound" and isinstance(missing, str) and

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 在 worker 端显示已核实的缺失组件 ID。 真实 Docker 对缺失路径返回 HTTP 404。路径中含 no such image 时,Docker SDK 7.2.0 将同类归档错误包装为 ImageNotFound。当前 supervisor 和 terminal receipt 均记录 declared_code,worker 从原请求摘要验证回执后仍只显示 ImageNotFound,因为这里仅接受字面 NotFound。请共用组件缺失的结构化标记并保留请求摘要与成员校验。用真实 Docker 覆盖两种 SDK 错误形状。 Docker SDK 的分类代码

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant