Skip to content

Keep checkpoint capture on the admitted continuation clock - #8621

Open
huangruiteng wants to merge 3 commits into
mainfrom
codex/edgebench-resumed-phase-clock-20261112
Open

huangruiteng wants to merge 3 commits into
mainfrom
codex/edgebench-resumed-phase-clock-20261112

Conversation

@huangruiteng

Copy link
Copy Markdown
Collaborator

After a budget continuation, the scheduler publishes its active phase deadline as a decimal epoch, including for whole seconds. The checkpoint controller rejected that file before contacting the native judge. A periodic review could proceed without a usable checkpoint, and failure preservation read the same clock through a different parser.

Both capture paths now use the scheduler's phase clock reader. It accepts the original integer format and the scheduler's decimal format, preserves the sealed clock bytes and remaining allowance, and rejects invalid or rounded input. Receiver activation and native scoring opportunities keep their existing rules.

Validation: exercised both worker capture routes with the real scheduler and Docker, then ran capture and restore against the native checkpoint HTTP API with a synthetic evaluator. Tested original and continued clocks, moved source rejection, invalid clocks, and continuation without resetting the phase allowance, submission count, or cooldown. The tests do not need a model or an account login.

Periodic and terminal checkpoint capture must read the same phase clock after scheduler continuation. Accept the decimal epoch published by the scheduler without rounding it or changing the receiver budget; preserve the legacy integer phase file and reject invalid clocks before native capture.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · max

本次评审对应提交 5ce63978b4cff5b6b75982bbcdc4301155b71e87。

[P2] 把现有的真实 HTTP 负例与新时钟格式对齐。 同一用例在 merge base 通过,在当前提交失败。

动机 / Motivation

开启自动 checkpoint 或终态故障保全的 EdgeBench 操作者。调度器即使续跑截止时间恰好是整秒也写出 .0;旧 controller 在访问 native judge 前拒绝所有小数,周期保存漏点,终态只保留已停源。对同一测试时钟,两个真实保存入口都封存 .0 与 .25 原字节、按原生捕获时间计算剩余时间,并保留终态前序链和原生机会。验证范围是真实 Docker、固定版本 native checkpoint HTTP 与合成评分;未运行生产 trial、模型、远端 CI 或完整研究态恢复。预合并人工门仍在。现有 native HTTP 回归测试仍要求 1.5 在 capture 前抛 ValueError;当前头接受该格式、写入原生点,精确头因此确定性失败。

改动思路 / Approach

在生产时钟格式的 scheduler owner 放共享 parser,同时复用现有封存、校验和原生恢复。本切片仅修复既有 benchmark 捕获路径的时钟兼容。 保留 producer 的 repr 字节可以维持阶段标识和精确剩余时间。两个调用方共用同一纯 reader,也能一致拒绝舍入、空值和越界。

具体改动 / Changes

关键代码讲解

对照 §7.7 Research-state branching and matched continuation:

  • ADMITTED-PHASE-CLOCK — implemented. read_phase_deadline 读取整数或能原样往返的小数。_phase_budget 用 native 捕获时刻计算余额,周期封存和直接恢复走原入口。
  • TERMINAL-LINEAGE — implemented. _scheduler_components_at_barrier 在原屏障内用同一时钟定位状态,再回读前序。没有活动时钟时保持原始路径。
  • INERT-RESTORE — implemented. _restore 从封存物理字节复算余量,零余额仍不能越过原 reserve。相同合成评分在两版均不因封存额外提交。
  • NEGATIVE-INPUT-CONTRACT — not_met. 现有真实 HTTP 用例 仍将 1.5 与 NaN 一起要求在捕获前拒绝。当前提交确实封存该规范小数,原测试报告 DID NOT RAISE ValueError。请改用 1.50 或需舍入的小数验证格式拒绝;单独断言过期规范小数余额为零且不能激活。

对主干的风险 / Risks

两条实际 Docker 路径在同一整数时钟上前后均通过;调度器写出的 .0 和 .25 以前都漏点,新版两条路径都封存原始字节与真实余量,终态留存活动前序。无效时钟仍在原生写入前失败。原 worker 可释放暂停,终态停源可保留。现有测试把过期和格式错误合并为一个拒绝条件,导致真实 API 验收失败。

语义与 CI 对齐

用同一冻结 native HTTP 版本、真实 Docker 及合成评分比较 merge base 和当前头。5 项直接检查、10 项 canary 通过。补齐隔离测试依赖后,定向用例中 10 项通过,包括真实 Host phase shell 和 native HTTP capture/restore:延后 8 小时仍按封存余额激活,旧提交和 cooldown 保留。1.5 是本提交新增失败;read_race 在两版均失败,列为基线问题。未读取 hosted CI 或运行实际研究 trial。

我的整体评价 / Verdict

请求修改。长期结果:保存同一续跑阶段可减少缺失 checkpoint,完整研究态恢复仍未验证。操作者结果:两条实际入口恢复了续跑封存,现有负例仍需修正并复跑。建议保留共享解析;这次只扩展现有 benchmark 时钟格式,没有新增通用 vocabulary、默认自动动作或恢复授权。改动只涉及 benchmark Python 封存路径及测试,没有前端、首屏或页面交互变更。无视觉验收项。

English verdict: REQUEST_CHANGES. The reviewed head fixes real periodic and terminal capture for scheduler-written whole-second and fractional clocks, but its existing native HTTP test still rejects canonical 1.5. Test a noncanonical input for format rejection and separately assert that an expired canonical clock retains zero allowance and cannot activate a receiver.


def read_phase_deadline(raw: bytes) -> int | float:
"""Read the initial integer epoch or the scheduler's continued phase clock."""
if len(raw) > 64 or not re.fullmatch(rb"[0-9]+(?:\.[0-9]+)?\n?", raw):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 扩大小数语法时请同步现有真实 HTTP 负例。

现有 test_invalid_host_epoch_cannot_create_native_point 的 1.5 用例在 merge base 通过,当前提交报 DID NOT RAISE ValueError。实际 capture 已写入原生点,余额为零且 receiver restore 拒绝。请改用 1.50 或需舍入的小数验证无原生点;另验规范过期值可保存证据但不激活。

@mergify

mergify Bot commented Nov 12, 2026

Copy link
Copy Markdown

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @huangruiteng.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase Mergify: the pull request has merge conflicts with its base branch label Nov 12, 2026
…-8621-review-repair-20261113

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

# Conflicts:
#	benchmark/edgebench/checkpoint_controller.py
#	benchmark/tests/test_checkpoint_controller.py
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
@mergify mergify Bot removed the needs-rebase Mergify: the pull request has merge conflicts with its base branch label Nov 12, 2026

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · max

本次评审对应提交 70f0930b1c9fcbe3980e1c29e52bf3c21ed858f2。

[P2] 自定义端点的阶段测试在执行前失败。 三个参数都在 assert readiness 失败。自定义端点正确跳过原生 codex_readiness,这里却仍用那个标记切分命令。 从生成命令的实际 dispatch 边界提取自定义路线的阶段前缀。保持实际 shell、剩余时间、提交计数和 cooldown 断言。复跑三组参数。

动机 / Motivation

在 EdgeBench 续跑阶段保存或恢复 checkpoint 的操作者。原问题是整数 reader 拒绝调度器发布的 .0 或小数。当前基准已经接收规范小数,但仍会把格式异常的小数舍入后封存。两个 capture 入口共用阶段时钟 reader,按原字节封存。真实 HTTP 上,1.5 可封存但零余额,1.50 和需舍入的小数会在创建原生点前停止。验证使用本地 Linux Docker、固定 native HTTP 源和合成评估。未验证生产 trial 或完整研究状态的采用。阶段 shell 测试改走自定义端点后仍等待只在原生登录路线出现的标记,三个用例在执行阶段命令前失败。

改动思路 / Approach

由发布 active phase deadline 的 scheduler 统一解析,周期预算与终态阶段查找复用。真实调用保留原来源和评分机会。已存在的阶段捕获与恢复。测试使用同一固定 native checkpoint API 和合成评估,调度、授权及机会来自原 owner。 由发布 active phase deadline 的 scheduler 统一解析,周期预算与终态阶段查找复用。真实调用保留原来源和评分机会。

具体改动 / Changes

关键代码讲解

保留规范阶段时钟和范围检查。 read_phase_deadline 用原捕获时刻计算余量。 _phase_budget 按同一时钟找到原前序。 _scheduler_components_at_barrier 生成自定义路由后错误查找原生 readiness 步骤。 host_phase_command

ADMITTED-PHASE-CLOCK 检查保持原阶段和余额,真实两条 capture 使用同一时钟。 TERMINAL-LINEAGE 检查按原阶段定位原前序,保留源证据。 INERT-RESTORE 检查过期规范时钟封存后不激活接收者。。 原验收约束

同样的两个共享失败出现在基准及当前头,保留为未通过证据。独立 witness 早先一次未进入小数阶段,串行重跑读取到 .25 并通过两条路径。已发布头的三组 readiness 失败可复现,属于本次改动。

对主干的风险 / Risks

规范 1.5 零余额不能激活接收者,非规范 1.50 和长小数不能建点。原 head 三个续跑测试失败,独立提取当前真实命令后同样三例通过。 原 native 捕获时刻、封存字节和既有余额控制继续管理激活。两条保存路径不增加评分或许可。 原整数和原生接收者要求保持。仅当已有捕获路径读取阶段时钟时应用同一解析器,不启用新的自动捕获。

语义与 CI 对齐

5 项直接、10 项选择性预合并检查均通过,人工合并门仍在。两版均有 read_race 和一个 scheduler 路径用例失败,未把它们计为通过。 三组 fresh/resumed/subsecond 在 assert readiness 失败。换用独立测试提取实际阶段 shell 后三组通过。独立真实 Docker 核两条捕获路径的整数、.0 与 .25 全通过。 托管 CI 未查询。

我的整体评价 / Verdict

请求修改。保留共用 reader 与过期保护,修正三组阶段用例的命令切分后复评。改动在 benchmark CLI、Docker 封存与测试。未修改页面呈现。验证使用本地 Linux Docker、固定 native HTTP 源和合成评估。未验证生产 trial 或完整研究状态的采用。

English verdict: REQUEST_CHANGES. The canonical deadline and expired-restore cases work. The reviewed tests switch to a custom provider but still require a native readiness marker, preventing all three real phase-shell assertions from running.

worker = SForgeWorker(SForgeConfig(agent_model="fixture", agent_effort="xhigh"),
# The model-free prefix must not try to inspect a desktop login. The
# custom endpoint uses the same phase shell and is never contacted.
config = SForgeConfig(agent_model="fixture", agent_effort="xhigh",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 切分实际生成的阶段命令

三个参数都在 assert readiness 失败。自定义端点正确跳过原生 codex_readiness,这里却仍用那个标记切分命令。 基准三例通过。用同一候选源码、在实际 timeout 派发处切分的独立复验三例都通过。从生成命令的实际 dispatch 边界提取自定义路线的阶段前缀。保持实际 shell、剩余时间、提交计数和 cooldown 断言。复跑三组参数。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants