Skip to content

fix(qualification): preflight source-bound model behavior pairs - #8468

Merged
huangruiteng merged 1 commit into
mainfrom
codex/model-behavior-source-preflight-20261109T205723
Nov 10, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/model-behavior-source-preflight-20261109T205723

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Paired model-behavior qualification assumes that every candidate is a v0 TurnEnvelope. An explicitly supplied complete v1 source is rejected or receives instructions for the wrong field layout. Validation of a private second arm can also happen after a first provider call.

This proposal extracts one complete qualification slice from #8266:

  • Validate complete sources with the existing TypeScript version/integrity owner, and compare the whole paired source plus its action semantics before execution.
  • Finish schema and confidentiality checks for both arms before either actor call, then reuse the existing actor/result/receipt path.
  • Select provider field instructions from the validated schema rather than the arm label. Retained complete captures preserve original host/settlement bindings and exact numeric values through storage and corpus loading.
  • Preserve the mainline raw-quota/v0 corpus control, actual-default portfolio, frozen matrix, scenario scoring, no-write sandbox and live-execution permissions.

The Python changes are provider/qualification adapters; admission, source integrity and execution authority remain with their existing typed owners. The bounded refactor reuses already validated actor requests rather than introducing a second result runner. No production output or context-delivery default changes.

Validation at f1d79a02f1284f5ee11278ff76b8a1b730d6a180, based on main 417e9fd41609eb553111d677f9635d3ee3160735:

  • Untouched baseline with the same new targeted cases: 19 failed, 5 passed, demonstrating the format/instruction/preflight defects.
  • Final source suite: 295 passed, including existing portfolio/retention regressions and real File/SQLite CLI private-source rejection before provider IO.
  • Default configured mypy: all 17 roots passed; five touched production modules also passed explicit mypy. Ruff, diff checks and the diff-scoped semantic advisory passed.
  • Exact wheel: 234 passed, with two source-CLI cases intentionally excluded from the wheel suite (both passed in the source suite). Six imported product modules were checked against the wheel and source for every case.
  • Standard premerge: six direct checks and fourteen selected checks passed; public-safety scan and the bilingual-diff check passed. The exact-scope author change-quality receipt verifies valid. Author checks do not grant merge authority.

Initial local validation caught a missing version export and a missing test import; both were repaired and the final clean suite rerun. The first wheel build lacked the required chat bundle; it was rebuilt from this worktree's source before the successful wheel build.

This is an author proposal, not independent C1 acceptance. Live-model understanding/adoption, original-quota single-session outcomes and complete cost remain unqualified; no paid model or benchmark run, installed rollout, release or merge was performed. Early C1 experiment records remain together in #8266. A merge of this slice lets C1 reuse this qualification support; it does not complete C1's production cutover.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=max

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Exact head: f1d79a02f1284f5ee11278ff76b8a1b730d6a180;对照基线:417e9fd41609eb553111d677f9635d3ee3160735。

动机 / Motivation

维护模型行为资格检查的开发者,需要安全地比较同一决策的两个输入,并判断当前完整源能否原样到达模型适配器。
旧版拒绝合法完整v1源;若第二个输入带隐私内容,还可能先调用一次模拟provider。当前head接受匹配的完整源,并在任一输入不安全时、两种执行顺序下都于首次调用前拒绝。
独立基线对照确认完整源、原始host/settlement绑定、未知字段和精确数值得以保留;不匹配源、坏版本/哈希及编码transport中的隐私内容被拒绝,历史正常控制仍可运行。
本PR不切换生产默认packet、actual-default portfolio或冻结评分,不自动调用模型,不授予工具或外部写权限,也不证明模型理解、在线效果或完整成本收益。

改动思路 / Approach

生产完整源的版本、内容及源完整性仍由现有TypeScript owner判定;Python只负责资格输入和provider适配,配对预检复用既有执行阶段,没有另造调度、授权或完整源决策器。
当前PR完整交付显式完整源的离线比较、双输入保密预检与保留样例读回;生产默认切换、在线模型资格和promotion仍在原边界之外。
配对先确认完整来源和语义一致,再构建两个canonical no-write request,最后按实验顺序执行已有actor/receipt阶段。只放宽版本会留下错误字段读取和迟到的隐私拒绝,重建旧capture会改掉原始绑定。现有共享执行函数的抽取已完成同域、可回退的简化;无须独立v1 actor、corpus或第二个Python决策owner。raw/v0 corpus仍是原默认,但full-source的peer_route读取说明也得到明确修正,不能把它称为所有共享prompt完全不变。

具体改动 / Changes

完整10文件+354/-53,覆盖5个生产Python适配文件、4个测试及一个协议;无生成物、临时探针或默认生产路径迁移。先于实现阅读的规范为 docs/reference/protocols/model-behavior-qualification-v0.md,固定在 417e9fd41609eb553111d677f9635d3ee3160735。PAIR-PROVENANCE(配对来源及独立语义重算)、PAIR-NOWRITE(私密输入及工具/写权限限制)、RETAIN-BOUNDARY(显式、本地、有界的保留样例)均implemented。LIVE-PROMOTION(deterministic fixture不能证明模型行为或授权promotion)属于原有后续运行边界,本PRout_of_scope且未放宽。ID对应原文未编号条款,新增协议文字不充当自身验收依据。

关键代码讲解

  1. _packet_schema(loopx/control_plane/testing/model_behavior_qualification.py:254)遇到v1或reserved contract_binding时调用已有完整源reader;未知版本、缺失或坏绑定/哈希不能退回raw reader。Python常量复用已有TS版本值,不产生新协议或状态分类词表。
  2. run_model_behavior_qualification_pair(loopx/control_plane/testing/model_behavior_qualification.py:895)对raw源使用原composer,对完整capture保留原绑定,比较整个源而非只比较已知字段;随后比较action semantics。两个request完成安全预检后才进入actor顺序循环。单臂和双臂共同使用既有结果规范化/receipt执行逻辑。
  3. _decision_instruction(loopx/control_plane/testing/doubao_model_behavior_actor.py:297)以validated schema选择source_arm;完整源candidate读取interaction_contract等实际字段,raw full的peer_route读取agent_identity/selected_todo,实验arm身份仍保留。实际fake transport看到原packet、没有tools定义,没有构造答案key。
  4. _case(loopx/control_plane/testing/model_behavior_corpus.py:59)对完整capture深拷贝保留,对raw输入继续构造历史v0。实际临时保留存储写入、0600权限、重新加载及两臂读回均检查了绑定和精确数值;输出receipt没有packet或path。
  5. scheduler_transport_validation_view(loopx/control_plane/testing/model_behavior_packet_safety.py:145)按完整源scheduler_hint路径解码真实transport字段,扫描仅用副本,原packet原样送到适配器;编码输入不获得scheduler执行权限。
独立检查 结果与边界
四个改动测试文件,source-bound Python3.13.16 236 PASS,含隔离File/SQLite真实quota CLI隐私拒绝;未调用模型
同脚本、同夹具、Python3.11.17的完整基线/当前检出 当前17/17;基线9/17,独立暴露完整源支持、绑定重读、字段指引和两种私密第二输入顺序的旧缺陷
完整owner及actual-default扩展测试 127 PASS、1 FAIL;既有失败详见下段,不写成整套通过
标准premerge恢复正常Python导入环境后 6 direct+14 selected PASS,exit0,含公开边界、词表、双语、compile和架构检查

最初继承宿主PYTHONSAFEPATH的premerge因quota-plan夹具import失败;基线和head同命令均复现,恢复正常导入后两者该项通过,完整premerge复跑通过,原失败保留。初版独立脚本把历史v0无法接受的大整数/浮点源样本当作正常控制,两版都拒绝;最终矩阵保留该断言作为已知限制的负向兼容性检查,并增加普通legacy正向控制,没有删除失败证据或宣称修复v0数值限制。

对主干的风险 / Risks

最强风险是两臂都读错相同字段却“等价”,或先调用一臂再发现另一臂不安全。对照明确检查原packet、完整未知字段/大整数、原始host/settlement绑定及真实provider指引;私密内容位于任一输入时,两种顺序都为零调用。real File/SQLite CLI材料仅在隔离fixture中生成,并因私密路由在provider前被拒绝;没有借测试改动活动Goal。这里的fake actor/transport证明源和路由,不证明模型理解或在线收益。显式v1支持不是feature toggle,也没有自动启用模型;历史默认、冻结矩阵及评分仍保留,raw full指引修复是披露的行为差异。

扩展portfolio的real-tool测试在基线和head、同Python3.11命令下均失败于turn_required_vision_replan/provider_transport_failed;独立pytest tracing捕获完全相同的未修改测试transport vision_patch_action -> _packet 的“quota was not observed”断言。该fixture/transport/replan路径未被本PR改动,改动的packet-extraction指引分支不是此tool-provider指令路径。因此将其归为pre_existing_unrelated,保留失败及后续场景abort,不能声称portfolio完整资格通过;为何夹具未观察到quota的更深层原因仍未完整追溯,也没有宣称已修复。当前改动另有236项定向、17例独立配对和完整源owner的通过证据,既有问题由该fixture owner处理,无需把修复无关fixture强塞入本PR。

语义与 CI 对齐

版本、内容hash及source hash继续由同一TS owner校验;Python adapter只选择该owner,不以字符串/prose猜测工作状态。新增协议文字说明了输入扩展及首次调用前的双输入保密义务,没有把模拟行为变成执行授权,没有放宽live/promotion条件。局部复用是reuse_existing;scope与现存资格caller相符。未查询、轮询或等待CI,代码结论依据当前本地证据;合并仍需独立权限与最新readiness,不能由此评审推得。

我的整体评价 / Verdict

有用的限定目标已达到,problem_context为goal_achieved;long_horizon为improved,依据是避免无效配对浪费首次provider调用并保存原始绑定,未冒称长程模型效果改善。user_experience为preserved,原流程新增交互轮次为0,早期拒绝后可用正确public-safe输入继续。已检查资格检查、corpus和CLI拒绝调用链;本PR不改变渲染页面、布局或产品编辑器。
未验证范围:未进行付费/在线模型运行、理解度、采用、推广或效果成本比较;扩展portfolio的既有失败保持披露。
历史输入兼容性应retain;预检与执行拆分已是有界简化,范围与问题相称。未发现此head的阻塞回归。APPROVE保留上述既有失败、v0数值限制和在线证据缺口,不授予merge、安装或付费运行权限。

English verdict: APPROVE — complete-source provenance and pair-wide preflight are validated; live model claims remain unproven, and the independently reproduced pre-existing portfolio fixture failure remains disclosed.

@loopx-agent

loopx-agent commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

已合并版本的独立复核:f1d79a02f1284f5ee11278ff76b8a1b730d6a180,目标 main,本地集成基线 d42388a178838bd9b883b973e0f7aa93452dbb30。已完整读回现有准确版本批准评审,approval closeout 为 clear,无阻塞评审或 inline findings。该 COMMENTED 结论不等于 GitHub 正式自批准;远端已由 maintainer 合并为 5e782fb3bcfc20d968a4836a55b6fa0dcf443852。

本轮独立验证:

  • CPython 3.14.8,四个变更测试文件与完整源 owner:准确 head 314 PASS;与上述 main 的真实合并预览树同组 314 PASS。两次均固定各自检出、解释器与工作区外临时目录,包含隔离 File/SQLite CLI 负例,没有调用在线模型。
  • Ruff、配置的 17 个 MyPy 根及 5 个变更模块、双语 diff、DCO 与 whitespace 检查通过。标准 premerge 为 6 direct + 14 selected 全通过,0 failure / 0 manual hold;准确范围 change-quality receipt cqr_bd05d6bdcda9485c53bb 已 verified,未修改作者 head,未应用额外 safe-fix。
  • 首轮测试因临时目录误放 Git 工作区内,得到 311 PASS / 3 个存储边界拒绝;修正执行环境后完整重跑通过,原失败记录保留。最初 premerge 预览要求补齐当前范围质量回执,现已补齐并通过实际 gate。

变更复用现有 TypeScript 来源及完整性 owner,在首次 actor 调用前检查两个输入,保留完整 capture 的绑定和数值;历史控制继续覆盖。现有评审披露的无关 portfolio 夹具失败、历史 v0 数值限制及在线理解/采用/完整成本缺口仍保留,未宣称它们已修复。遵循当前 CI policy,未查询、轮询或等待 CI。实际合并树 8e314fadf9abc5a9fdddc0729909cc6a940bd089 与最新集成测试的完整源码树相同;贡献提交有 sign-off,GitHub 生成的二父合并提交签名 verified。最终读取已为 MERGED,未重复发起合并。以上是源码及确定性资格,仍不证明安装采用或在线收益。

@huangruiteng
huangruiteng merged commit 5e782fb into main Nov 10, 2026
28 of 40 checks passed
@huangruiteng
huangruiteng deleted the codex/model-behavior-source-preflight-20261109T205723 branch November 10, 2026 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants