Skip to content

Allow bounded native solver histories during benchmark closeout - #8564

Merged
huangruiteng merged 1 commit into
mainfrom
codex/managed-terminal-large-evidence-20261111
Nov 11, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/managed-terminal-large-evidence-20261111

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Outcome

A long benchmark attempt can finish with all grades drained while its native solver history exceeds the limit for compact control receipts. The managed runner then fails to retain the terminal solver snapshot. Replaying a completed attempt also fails when its final result embeds the same history, leaving the verified attempt's reservation occupied.

Use a bounded 64 MiB read for the two native evidence artifacts. The managed binding and receipts keep their compact bound. Closeout still verifies the exact solver evidence, launch identity, and grading drain before it can publish a terminal proof or release the original reservation.

Verification

  • 37 tests passed on the Linux source checkout with the native SDK and Node 24. Coverage includes a history above the former limit at normal closeout and during recovery when the terminal receipt write fails.
  • A completed managed attempt with both evidence files above the former bound was replayed against the real controller. The terminal row and released reservation read back, and all six original evidence records retained their digests.
  • Pending grading and changed native history refused closeout in isolated replay. Ruff, staged RFC pairing, and git diff --check passed.

The change affects runner and recovery entrypoints. No visual surface changes.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · max

评审提交:8c3677661ec2b1f6c24786018d680ec76ad68c75

动机 / Motivation

使用原生评分的 Benchmark 托管尝试已结束,长时间运行或多次恢复累积了完整的求解记录。求解记录和最终结果超过紧凑回执的 1 MiB 上限时,runner 无法保留终态,控制器也无法核对并释放原预留。两个原生证据文件使用各自的 64 MiB 读取上限。正常结束与丢失终态回执后的重放都能核对同一次尝试,并由控制器释放原预留。两个原生证据文件使用各自的 64 MiB 读取上限。正常结束与丢失终态回执后的重放都能核对同一次尝试,并由控制器释放原预留。待评分、修改后的求解记录和越界文件均阻止发布终态。控制记录保留原有上限。本轮使用本地隔离 controller 和固定版本的真实 SDK 写入函数,求解输出为合成记录。Linux 容器内的完整长跑及高密度近上限记录的内存占用未测。本轮使用本地隔离 controller 和固定版本的真实 SDK 写入函数,求解输出为合成记录。Linux 容器内的完整长跑及高密度近上限记录的内存占用未测。

改动思路 / Approach

沿用原 bounded reader、终态证明和 controller。对原生长证据使用独立的文件大小上限。覆盖正常保存和终态回执丢失后的重放。收尾仍验证同一次尝试的评分、求解历史和来源。 三处读取沿用已有文件校验。runner 保存求解记录时读取 solver_execution,回执写入失败后的重放还读取 final_result。只有原来源、结束记录和已排空的评分通过,控制器才会收到终态证明。

具体改动 / Changes

关键代码讲解

完整差异两个文件,+27/−14。按基线核对 same-attempt-proof、retained-terminal-recovery 和 controller-slot-release。64 MiB 专用于 solver_execution 和 final_result。绑定、运行回执及评分历史仍各按原限制读取。基线收尾契约 保存来源 原次重放

两边用同一已固定的原生 SDK 和真实 controller。主干两条大历史路径失败,当前全部 20 项通过。正常及丢回执后均读回终态和原槽位释放,重放保留同一份证明。

对主干的风险 / Risks

真实 replay CLI 接受两份填充至 64 MiB 的有效 JSON。六种超限文件都拒绝,修改一个求解段或留下待评分状态同样不能发布。恢复原证据后读回同一证明,原始记录字节未改。

语义与 CI 对齐

主干其余 18 项通过,普通入口行为一致。候选标准 canary 的 5 项直接和 6 项选中检查通过,另有 Benchmark 人工验收门槛。边界测试使用 JSON 空白达到大小上限。

我的整体评价 / Verdict

通过当前提交的独立评审。正常路径与回执丢失后的路径都可核对长证据并释放原预留。评分未完成、篡改和超限依旧被拒绝。验证覆盖实际 CLI 与控制器回读。本轮使用本地隔离 controller 和固定版本的真实 SDK 写入函数,求解输出为合成记录。Linux 容器内的完整长跑及高密度近上限记录的内存占用未测。

English verdict: APPROVE. Valid large native evidence closes and replays the same attempt. Oversized, mismatched or pending evidence still withholds terminal proof.

@huangruiteng
huangruiteng merged commit 5249cb3 into main Nov 11, 2026
28 of 36 checks passed
@huangruiteng
huangruiteng deleted the codex/managed-terminal-large-evidence-20261111 branch November 11, 2026 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants