Repository navigation
feat(eval): enforce product parity and runtime binary identity gates - #6853
Closed
Colafornia wants to merge 4 commits into
Closed
Colafornia wants to merge 4 commits into
Colafornia wants to merge 4 commits into
Conversation
…tore Add the maka-eval foundation modules: - strict maka.eval.v1 projection and native maka.eval.v2 orthogonal attempt contracts, condition-only RunFingerprint, and semantic difference registry - exact 6-counter token accounting, non-mutating local metering gateway, admission cutoffs, HMAC-authenticated checkpoints, and receipt-vs-proxy reconciliation - single-writer append-only attempt journal, SQLite runtime artifact capture, critical artifact verification, and deterministic state reconstruction Generated-by: Codex Generated-by: Google Antigravity
…nical reporting Add execution orchestration, Harbor isolation, and deterministic projections: - version-pinned Harbor / Pier trial environment bridge, loopback TCP + HMAC relay agent, and DNS/IP egress filtering proxy - task-group admission concurrency, shared preparation cache lock, phase-scoped deadlines, bounded retries, and crash recovery - external CLI subject adapter (exit-code and protocol-v1 result frames) and canonical v1/v2 journal projections with 3-verdict differential oracle evaluation Generated-by: Codex Generated-by: Google Antigravity
Add real Maka Host subject execution and CLI commands: - resolve outbound model routes through Maka provider policy before attaching the non-mutating local metering gateway - initialize isolated Host roots via maka host init --settings-stdin without secret leakage, execute through public Settings/Session and composer draft/receipt APIs, and record executed Maka binary digests - preserve unattended approvals and questions as explicit settlement outcomes and report environment limitations without silent bypass - add maka-eval run/report/host-subject CLI commands and wire CI runtime helper consumers Generated-by: Codex Generated-by: Google Antigravity
Add end-to-end Product Parity verification and M3/M5 milestone acceptance gates: - add maka-eval parity to verify product composition, effective instructions, tool exposure, execution policies, and resolved outbound routes between normal Maka execution and maka-eval - enforce passing parity reports and confirmed runtime Maka binary SHA-256 identity from settled execution artifacts in M3 acceptance and M5 production delivery gates Generated-by: Google Antigravity
Member
Author
|
Parked until the preceding layer merges into rust-preview so each PR shows a single-layer diff; will reopen sequentially. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
English
Summary
Condition fingerprints alone cannot prove that
maka-evaldrives Maka identically to normal user execution or that settled results came from the parity-verifiedmakabinary. This final PR adds end-to-end Product Parity verification (maka-eval parity) and enforces parity and runtime binary identity inM3andM5milestone gates.Refs #6667. Builds on #6852 (until it merges, review commit
08b1ca4a2for this layer alone).parity&maka-eval parity: Preserve declared endpoint semantics on the reference path while drivingmaka-evalagainst a local probe under identical inputs, verifying committedRunSpec/RequestSpecfacts, resolved upstream routes, effective instructions, tool exposure (Code Mode, skills, tool schemas), and execution policies.report&M3/M5milestone gates: Require passing parity reports matching each Maka subject's public configuration digest, verify confirmed runtimemakabinary SHA-256 identity from settled execution evidence across local and container paths (failing closed when missing or mismatched), and require complete non-indeterminate settlement form5.Verification
Validated at
08b1ca4a2(git diffagainst the fullfeat/maka-evalimplementation at4eddc488cis empty):cargo test --locked -p maka-eval— 25 passed (2 unit tests and 23 integration tests covering liveM3/M5product parity, instruction/tool/policy drift detection, end-to-end Zenmux route parity, and CLIM3binary-identity enforcement across local and container paths).node --test scripts/ci.test.mjs— 18 passed.cargo clippy --locked -p maka-eval --no-deps --all-targets -- -D warnings— passed.RUSTDOCFLAGS="-D warnings" cargo doc --locked -p maka-eval --no-deps— passed.node scripts/asf-license-headers.mjs check,node scripts/ci.mjs check format, andgit diff --check— passed.Dependency-inclusive Clippy hits the existing
clippy::needless_updateinplugins/agent/provider-protocol/src/chat_completions/response.rs:87outside this PR's diff. Live Docker Harbor cohorts, frozen legacy oracle cohorts (#6667), paid model endpoints, and fulljust checkremain before marking ready for final acceptance.AI use
Tool(s) and scope: Google Antigravity authored the Product Parity verifier,
M3/M5milestone and binary-identity gates, CLI wiring, documentation updates, and integration tests, and verified the stack. Commit carriesGenerated-by: Google Antigravity.Checklist
Does this PR entail a change in behavior?
中文
概述
仅靠运行条件指纹无法证明
maka-eval与常规用户路径以完全一致的语义驱动 Maka,也无法确保已结算结果来自通过等价性验证的同一份maka二进制。本 PR 新增端到端产品等价性验证(maka-eval parity),并在M3与M5里程碑门禁中强制校验等价性报告与运行时二进制身份。关联 #6667。基于 #6852(合入前可直接查看提交
08b1ca4a2以仅审阅本层改动)。parity与maka-eval parity:保留参考路径的原始端点语义并在相同输入下驱动maka-eval连接本地探针,比对落盘的RunSpec/RequestSpec事实、解析后的上游路由、生效指令、工具暴露(Code Mode、内置技能、工具模式定义)及执行策略。report与M3/M5里程碑门禁:强制校验匹配公开配置摘要的等价性通过报告,依据已结算执行产物核验实际maka二进制 SHA-256 身份(支持本地与容器路径,缺失或摘要不匹配时显式失败),并在m5下要求全量完成确定性结算。验证
在提交
08b1ca4a2上验证(与完整分支feat/maka-eval的4eddc488c对比差异为空):cargo test --locked -p maka-eval— 25 项通过(2 项单测与 23 项集成测试,新增M3/M5产品等价性校验、指令/工具/策略漂移检测、Zenmux 路由端到端等价性及本地与容器路径的M3二进制身份门禁测试)。node --test scripts/ci.test.mjs— 18 项通过。cargo clippy --locked -p maka-eval --no-deps --all-targets -- -D warnings— 通过。RUSTDOCFLAGS="-D warnings" cargo doc --locked -p maka-eval --no-deps— 通过。node scripts/asf-license-headers.mjs check、node scripts/ci.mjs check format与git diff --check— 通过。包含依赖的 Clippy 受基线已有的
plugins/agent/provider-protocol/src/chat_completions/response.rs:87(clippy::needless_update,不在本 PR 改动范围内)阻断。真实 Docker Harbor 队列、#6667 历史参考实现队列比对、付费模型端点及完整just check仍需在转为正式验收前完成。AI 使用
工具及范围:Google Antigravity 编写产品等价性校验器、
M3/M5里程碑与二进制身份门禁、CLI 接线、文档更新及集成测试,并完成整组分支栈验证。提交包含Generated-by: Google Antigravity。检查清单
本 PR 是否改变行为?