Skip to content

perf(todos): preserve summary identity with linear deduplication - #8465

Merged
loopx-agent merged 2 commits into
mainfrom
codex/quota-summary-linear-dedup-20261109t1900
Nov 9, 2026
Merged

loopx-agent merged 2 commits into
mainfrom
codex/quota-summary-linear-dedup-20261109t1900

Conversation

@loopx-agent

@loopx-agent loopx-agent commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

大规模 Todo 的普通摘要读取原先为每个候选行扫描全部已选行,重复查找随行数平方增长。本 PR 在现有 TypeScript projectTodoSummaryDisplay owner 中用局部索引完成同一选择,保留原始 ID 的严格相等、匿名行的坐标/文本冲突规则、首次出现顺序及 source callback 血缘。规划/终态视图与 flat-items 路径沿用已有语义。

验收针对完整原生读取结果和独立身份语义:保持全部 summary/planning/terminal 视图、一次原生请求、零快照写入与存储内容不变。相应测试先在原实现通过;将候选实现故意改成归一化 raw ID 后,实际原生 API 的声明式结果 oracle 失败,恢复后通过。没有新增公共状态、配置或权限规则;同域简化就是删除重复前缀扫描,避免引入额外 helper 或 provider。

完整 TypeScript 回归首先发现基线已存在的静态登记遗漏:yardstick_decision.ts 使用并实际调用统一的 BARE_SHA256_PATTERN,但尚未列入 CANONICAL_CONSUMERS。原始基线和最初候选运行相同隔离命令均失败于这个唯一缺项。第三处文件只补登这一现有消费者;精确集合、AST 导入/使用检查和禁止私有重复校验器的断言全部保留。相关 27 项检查在补登后通过。

Validation for final head 4f2f6469667c1865eee8b1867c843be9fd9d9a2e (the production source is byte-identical to the initial candidate; the final commit changes only the static consumer roster):

  • TypeScript focused tests: 41 passed; control-plane typecheck passed.
  • Native public todo.context.page base/head parity passed, including source identities, flat duplicates and planning selection; inputs preserved.
  • The unchanged 8,000-Todo File/SQLite regression passed in all 28 matched base/head runs: three serial samples and four concurrent processes for each backend/revision. Original timeout and expected complete views were retained; all 28 owned native workers exited successfully.
  • Source provenance was asserted for each paired native run: matching Python/dependencies, actual checkout package and actual checkout native server, distinct disposable temporary roots. A separate interleaved same-process projection comparison retained byte-identical complete output at 1k/2k/4k/8k rows.
  • Initial full control-plane TypeScript run: 6,301 passed, 1 failed, 78 skipped (the preserved consumer-registration failure above). Corrected-head full rerun: 6,302 passed, 78 skipped, zero failures. The skipped PostgreSQL-dependent cases remain unverified. Adjacent source-qualified Python regression: 208 passed, no skips.
  • Final exact-scope CQR cqr_52dfb22ee4bbcf6a781d was recorded and verified against main snapshot 4995f606b74d9512a43a8008ae81ec7cad4339f2 and unchanged final head. Standard premerge passed 4 direct + 16 selected checks in 118.34s, with zero failures, warnings or manual holds. Final independent exact-head review and merge readiness are checked separately before merging.

Observed native test wall medians, seconds (includes fixture, startup and cleanup):

Provider / cohort Base Head
File / serial, n=3 10.417 8.663
File / four concurrent processes, n=4 22.086 13.752
SQLite / serial, n=3 10.909 8.968
SQLite / four concurrent processes, n=4 19.642 14.576

The indexes add O(n) request-local references; no memory reduction is claimed. These are bounded samples under varying host load. The current immutable base also passed every run: historical timeouts were not reproduced, and this evidence does not establish their root cause, sustained installed-host latency, or long-horizon outcome benefit. No timeout, test assertion or qualification criterion was relaxed.

English: Replace quadratic summary identity scans with local indexes while preserving raw-ID and legacy-coordinate semantics. Register the already canonical yardstick digest consumer without weakening the static owner checks. Required author validation passed; independent exact-head review and merge readiness remain separate gates; the paired runs demonstrate a bounded latency improvement, not historical timeout elimination.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

精确 head 4f2f6469667c1865eee8b1867c843be9fd9d9a2e 未发现阻断问题。审批仅适用于这个提交;没有合并、安装或升级运行时。

动机 / Motivation

已有大量 Todo 的 Agent 和操作人,通过原生 quota 或摘要读取当前工作时,会反复合并多个来源的同一批任务。旧实现为每个候选行回扫已选行,任务越积越多,普通摘要读取的重复比较随之增长;新实现索引本次已经选中的行,同时保持原始身份、顺序和匿名行规则。

本轮独立验证真实 native 入口在同一输入上的完整输出不变,File/SQLite 的八千行读取仍完整且没有 snapshot 写入,并用错误身份归一化的反例确认测试能检出漏行。交付边界是当前共享摘要路径的行为保持与重复比较消除,不证明历史超时已修复、已安装宿主采用或持续净收益,也不承诺内存下降。

改动思路 / Approach

真实入口仍是 native todo.context.page 和既有 quota/Python 摘要适配器。唯一决策 owner 仍为 TypeScript projectTodoSummaryDisplay;本次改为在请求内记录已经选中的行,没有增加数据库字段、缓存失效协议或第二份 Python 规则。两个有效 raw ID 只比原始值;至少一侧无有效 ID 时才使用原坐标和去空格正文匹配。只有真正入选后才登记索引,因此保留 first-wins、原 source 次序和 callback lineage。

这一边界适合放在当前 projector:正确性关系由同一 owner 管,provider IO、选取权限和持久化仍由原 owner 管。flat-items、planning、terminal 是既有不同语义的分支,不能为了复用索引强行合并。本次没有新增开关;对共享路径的兼容性由真实 base/head 对照证明。代价是 O(n) 请求内集合引用,不能由局部 CPU 改善推导内存也下降。

具体改动 / Changes

关键代码讲解

完整范围三条路径 +63/−2。context_projection.ts:254 的 projectTodoSummaryDisplay 用 raw-ID 集合及坐标下的 all/anonymous 正文集合替代 selected.some。其中 coordinate === coordinate 保护不可省略:Map 的 SameValueZero 会把 NaN 与 NaN 视为相等,旧严格相等不会。±0、truthy/falsy ID、带空格但原值不同的 ID、匿名/有 ID 混合及对象身份都应沿原关系处理,当前 direct TS suite 已覆盖。

同文件 projectTodoContextPage:26 继续返回完整 summary/planning/terminal 三视图。work_items/yardstick_decision.ts:96 的 normalizeWorkYardstickSourceSnapshot 本身未改,它已从 canonical digest owner 导入并实际使用 BARE_SHA256_PATTERN;第三条路径只把这个既有消费者加入 exact-set roster,没有删除 AST 或 private-matcher 负例。独立 immutable-base 执行确实得到 16 PASS/1 roster FAIL,当前相关三文件共39项全部 PASS。

规范依据为 既有 Todo Field Roles,spec_revision 8fbfe41229b7d86a9dbdfe0a35cdf26c046e8a22。本评审映射 TODO-PROJECTION(canonical身份与投影边界)、TODO-COMPAT(完整视图和顺序兼容)、TODO-NO-WRITE(读取不写canonical/snapshot)三项均已实现;原文没有细化 raw-ID 混合关系,该兼容 oracle 来自 immutable base 的真实调用,未把新实现自证为规范。

对主干的风险 / Risks

最大风险是索引把旧关系简化成错误等价类,静默隐藏合法任务。为避开测试 expected 与新 helper 同错,本轮另写旧严格关系的独立 oracle,经实际 Python→Node24 native 入口执行35次 public call。base、当前 head 与恢复后 head 的完整结果均相等,fixture SHA aebce7bf0fbf05fd53e5ea240bd11da71cfe48640af8685f83c20f96737afd72,完整输出 SHA d5beb5d2bd575312e19565f2049168b2e9bc1362b1a5ae31d7b591f767e36fa3。故意把 raw ID 改为 trim 后,首个 public 场景就实际丢行并 FAIL;恢复原字节、更换独立 namespace 后同组35次全部 PASS。

本轮独立的四组相邻 Python 测试52 PASS,覆盖 File/SQLite 八千行、完整三视图、单次native quota RPC与没有snapshot写入;该组在38cb执行,而38cb→4f只补一行测试消费者登记,生产SHA 939598ce…及这些测试不变已核验。immutable base 的两后端大状态例也 PASS。当前4f另独立完成39项相关TS、完整typecheck及标准premerge(4 direct+16 selected全PASS,0 warning)。初始38cb标准premerge的semantic scan曾在120.723s超时,此记录仍保留;新head在同一原阈值下完成全部检查,不能因此宣称历史根因被修复。远端CI未查询。

作者完整终端记录另有当前4f TS 6302 PASS/78 SKIP/0 FAIL及生产相同38cb的208项相邻Python PASS;这些明确是作者证据。其 matched native 28例全部通过,串行File中位10.42→8.66s、SQLite10.91→8.97s,投影局部数据也支持少做重复比较;整例含启动/fixture且宿主负载未控制,rusage并未证明所有内存指标下降。这里不将作者数据改称本轮独立性能复现,也不外推历史timeout、p95或安装后的持续收益。

语义与 CI 对齐

复用既有Todo身份/终态与canonical digest词汇,无新增Enum、公开协议或持久state。最终标准semantic scan已执行通过,未放宽预算、删掉断言或豁免必要检查。raw匹配不是正文关键字分类,索引不能授予admission、quota、claim或写入authority。

我的整体评价 / Verdict

APPROVE。长程接续与用户体验均为 preserved:积累状态仍返回同样完整工作,既有读取零额外交互,身份、终态证据与 no-write 边界经真实入口验证。请求内索引是同域、有用且可回滚的简化;保留 legacy 身份分支有实际调用和负例依据,不需要额外兼容版本或缓存。已检查CLI/摘要payload;没有渲染呈现变化,因此无需截图,渲染变化的未验证范围为 none。installed-host采用、持续净成本和历史超时根因仍未证明。后续新head或merge readiness须单独核验。

English verdict: APPROVE 4f2f646. Real native base/head/restore outputs match, the deliberate raw-ID mutation is detected, and exact-head TypeScript, typecheck and standard premerge pass. Historical scan timeouts and installed-host performance remain separate, unproven claims.

@loopx-agent
loopx-agent merged commit 67dcbde into main Nov 9, 2026
33 of 39 checks passed
@loopx-agent
loopx-agent deleted the codex/quota-summary-linear-dedup-20261109t1900 branch November 9, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant