Skip to content

docs(research): connect decision feedback to action-value evidence - #8594

Merged
huangruiteng merged 1 commit into
mainfrom
codex/research-action-value-evidence-20261111
Nov 11, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/research-action-value-evidence-20261111

Conversation

@huangruiteng

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

A score increase after switch records progress on the executed route; it does not identify the gain over continuing grow. The research plan already calls for forecasts and followthrough, but needs a concrete bridge from those records to action-value evidence.

This extends #8461 in the existing English/Chinese research rationale, joint Target/Yardstick/Stride RFC, and EdgeBench pilot:

  • Define action value conditional on state, remaining budget and continuation, and distinguish it from an alternative-action contrast.
  • Explain the decision episode: visible state → prior forecast/choice → actual work/resources → evidence and next decision → terminal outcome. Preserve forecast revisions, availability times, delayed credit and missing outcomes.
  • Separate forecast calibration, Proxy–Native calibration, checkpoint contrasts and frozen-policy new-task evaluation. Add C17–C18 and a concrete pilot readback requirement without extra per-Turn fields or evaluation calls.

Intended base: loopx-project/loopx:main. This is the requested documentation refinement, not a runtime implementation or a measured benchmark improvement.

Author Declaration

  • Written by: model_agent — OpenAI Codex (GPT-6).
  • Specification and revision: existing budgeted research rationale §9, joint decision RFC §§5.8–5.9, and research-program RFC §8.7 at 0a654c6223784daeec92009f3e7361c26d8b6079, refined by this proposal.
Criterion Disposition Path / section Validation
Link decision-time evidence to later outcomes without fabricating counterfactual labels implemented in docs docs/research/budgeted-research-planning.{md,zh-CN.md}, §§6–7, 9 Bilingual semantic review and equation parity
Reuse decision/evidence owners and preserve missing, failed and delayed results implemented in docs docs/architecture/rfcs/target-yardstick-stride-decisions-v0.{md,zh-CN.md}, §5.9, C17–C18 Paired RFC diff check and manual contract review
Qualify a useful decision-to-work slice before claiming policy benefit implemented in docs docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.{md,zh-CN.md}, §8.7 Rendered cross-links and pilot consistency review
  • Self-check: reviewed the complete staged diff, preserved prior requirements while restructuring §9, checked public boundaries and verified the rendered anchors. No runtime, training or benchmark execution is claimed.

Scope And Continuation

Complete within this documentation scope. The existing §8.7 pilot remains the next implementation/experimental step: first establish actual feedback adoption, then matched-budget outcomes. A learned Meta Model, new scheduler, schema, training corpus and production promotion remain outside this PR. C17–C18 are proposed acceptance scenarios, not newly executed tests.

Validation

  • Tested revision: 1aed67b8f95ee8d8d6fa8069e7b22b25c0af838a (same document contents as the local build).
  • Run state: finished.
  • Input classes: none (documentation validation).
Check kind Result Public-safe evidence / limitation
static passed uv run --extra test --with-requirements docs/requirements-docs.txt mkdocs build --strict --site-dir .local/validation/docs-site
static passed scripts/check_rfc_bilingual.py --base origin/main --staged, then --base origin/main --head HEAD
static passed scripts/generate_rfc_status_index.py --check; git diff --cached --check
manual passed English/Chinese semantics and block equations checked; all eight added local links/anchors resolve in rendered HTML
real_entrypoint not_applicable No runtime behavior changed; the EdgeBench experiment and learned policy remain proposed

Frontend / Visual Evidence

Not applicable: body content on existing documentation pages only; no documentation chrome, navigation or product UI changes.

Signed-off-by: huangruiteng <huangrt01@163.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · max

评审提交 1aed67b8f95ee8d8d6fa8069e7b22b25c0af838a。无阻塞问题。 [P3] 建议把检查点上的两种对照分别命名:如果两臂都先选 grow,只在后续决策规则上不同,配对差仍可能非零;它估计的是该检查点上的策略差,不能成为 grow 相对另一首步的因果标签。 在证据表把“整个后续策略不同的条件政策对照”与“只干预首步、后续规则相同的动作对照”分开标注。

动机 / Motivation

准备 EdgeBench 接续试验的人,需要知道一次观测能说明什么,以及如何保留能检验下一次决定的证据。原文要求保留预测和执行,却没有给出如何用这些记录判断策略的完整样例;在一次 switch 后涨 1 分时,读者可能把涨分误记成相对 grow 的收益。现在两种语言都把当时可见的状态、事前预测、实际工作、返回反馈和最终成果串起来:只有执行路线的进展已观察,没执行的替代路线仍为未知。现有决策、测量和后续行动记录足以描述首个可核查片段;校准、实际采用和终局收益各有独立验收。本次交付是研究和验收文档。首次真实采用、对照试验和训练都仍需另外执行和验证。

改动思路 / Approach

用既有的计划、执行、测量与接续记录,还原“当时状态 → 预测和选择 → 实际工作与消耗 → 返回证据和下一步 → 冻结终局”。既保留预测原文,也保留结果可见前、有新依据的修订。反馈支持原路线时,能在实际接续中看见有据继续;收到了提示或换了路线,本身不证明有收益。

具体改动 / Changes

关键内容讲解

以合并基点 0a654c6223784daeec92009f3e7361c26d8b6079 的 docs/architecture/rfcs/target-yardstick-stride-decisions-v0.md 为准:联合 RFC 的 §5.8、§5.9 继续由既有决策机制负责;C14 的剩余时限、C15 的前置工作、C16 的待确认状态都保留。新增 C17/C18 把未选路线、迟到反馈和同源片段拆分列为反例。研究说明 与中文版 用同一组公式区分所选路线的进展、给定后续规则的价值、相对替代路线的差。若 switch 从 60 到 61、同条件的 grow 到 62,所见进展是 +1,配对差才是 −1;没有对照时,后者未知。

试验 §8.7 要求第一段真实接续可读回;若十八小时终局未到,两小时的观测只检验对应时间窗。保留失败、取消、未涨分和无法判断的片段,分开验证预测校准、实际采用、同状态对照和新任务收益。三个双语文档各保留现有职责,无需增加记录格式或额外评测。

对主干的风险 / Risks

PASS:完整远端 diff 对照、MkDocs 严格构建、三对文档的六个锚点及八个新增同语种链接、英中公式和 C17/C18 反例、双语 RFC 校验及状态索引。实际恢复是否等价、评测何时可见和全部成本,仍须由试验观测;不能用本次构建或合成推演代替。托管 CI 未查询。

我的整体评价 / Verdict

这次交付完成了文档层的证据链,并让读者能从说明追到具体试验验收。范围集中,不需要再加抽象或合并三个文档。只改既有文档正文和链接;已核对构建结果,无产品页面布局改动。下一步仍是现有 pilot 的真实接续和完整预算对照。

English verdict: APPROVE. At 1aed67b, the bilingual design separates observed progress, action contrasts and policy benefit. The strict docs build and added links pass; real adoption and efficacy remain untested.

| --- | --- | --- |
| Pre-action forecasts and corresponding realized outcomes | Probability, gain-range and duration calibration on covered states; report missing/cancelled outcomes separately | Unchosen outcomes, or that accurate prediction makes allocation better |
| Same-candidate Proxy–Native pairs | Measurement bias, ordering and scope | Proxy improvement as a causal policy gain |
| Different continuations from the same checkpoint | Local contrasts and uncertainty under equivalent restoration, isolation and aligned budgets | One pair as an exact expectation, or the policy's from-start effect |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] 如果两臂都先选 grow,只在后续决策规则上不同,配对差仍可能非零;它估计的是该检查点上的策略差,不能成为 grow 相对另一首步的因果标签。 在证据表把“整个后续策略不同的条件政策对照”与“只干预首步、后续规则相同的动作对照”分开标注。 可用“两臂都先 grow,但后续规则不同”作为区分用例。

@huangruiteng
huangruiteng merged commit 6fdc007 into main Nov 11, 2026
26 of 27 checks passed
@huangruiteng
huangruiteng deleted the codex/research-action-value-evidence-20261111 branch November 11, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants