feat(eval): add experiment contracts and usage pricing semantics

Cargo.toml

@@ -34,6 +34,7 @@ members = [
     "plugins/workspace/workspace-files/native",
     "plugins/workspace/support",
     "products/maka/acp",
+    "products/eval",
     "plugins/agent/engine",
     "plugins/agent/native",
     "plugins/workspace/workspace-files/patch",
@@ -149,6 +150,7 @@ imagesize = { version = "=0.15.0", default-features = false, features = ["png",
 jsonschema = { version = "0.60.1", default-features = false }
 libc = "0.2.191"
 maka-acp = { path = "products/maka/acp" }
+maka-eval = { path = "products/eval" }
 maka-agent-engine = { path = "plugins/agent/engine" }
 maka-agent-native = { path = "plugins/agent/native" }
 maka-assistant = { path = "plugins/agent/assistant" }

products/eval/Cargo.toml

@@ -0,0 +1,29 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+[package]
+name = "maka-eval"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+publish.workspace = true
+
+[dependencies]
+hex.workspace = true
+serde.workspace = true
+serde_json.workspace = true
+sha2.workspace = true

products/eval/README.md

@@ -0,0 +1,41 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+
+# maka-eval
+
+[简体中文](README.zh-CN.md)
+
+Measurement and control plane contracts for multi-arm benchmark experiments (`maka.eval.v1` and `maka.eval.v2`).
+
+## Architecture
+
+- [`contract`](src/contract.rs): Strict `maka.eval.v1` spec validation and cell expansion, immutable `V2AttemptRecord` with 3-gate causal adjudication (`BlockedIndeterminate` -> `RetryableInfra` -> `Selected`), leaf `v2 -> v1` projection, and spec-only `SpecFingerprint`.
+- [`measurement`](src/measurement.rs): Exact 6-counter token accounting (`inputTokens` normalized inclusive of cache reads/writes; `totalTokens` preserved without summation) and `PricingRates` (`DEEPSEEK_V4_FLASH_COST`) as the sole USD pricing authority.
+
+## Adjudicated `v2 -> v1` Rules
+
+| Scenario | `v2` Disposition | `v1` Projection & Selection |
+| --- | --- | --- |
+| Timeout + reward + `LowerBound` | `Selected(SubjectFailed)` | `subject_failed`, raw `score`, `costUsd: null`; selected & never resampled |
+| Audit truncated / policy error | `Selected(Completed)` | `completed` with `egress-audit` metadata (`bytes`, `sha256`, `malformedLineCount`) |
+| Missing audit / critical artifacts | `RetryableInfra` (unless Gate 1 indeterminate) | `infra_failed`, `score: null` (stricter than legacy TS when score existed) |
+| Late cancellation | Before verify: `BlockedIndeterminate`; After: `Selected` | `indeterminate` (`score: null`) vs preserved score + marker |
+| Post-hoc evidence recovery | `PostHocRecovery` appended monotonically | Derived view updates status/usage/cost without mutating the base attempt |
+
+[Source](src/lib.rs) · [Cargo manifest](Cargo.toml)

products/eval/README.zh-CN.md

@@ -0,0 +1,41 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+
+# maka-eval
+
+[English](README.md)
+
+多臂基准评测实验的测量与控制平面契约底座（支持 `maka.eval.v1` 与 `maka.eval.v2`）。
+
+## 模块结构
+
+- [`contract`](src/contract.rs)：严格的 `maka.eval.v1` 实验规格校验与单元展开、不可变 `V2AttemptRecord` 及三关因果裁定（`BlockedIndeterminate` -> `RetryableInfra` -> `Selected`）、末端只读 `v2 -> v1` 投影，以及仅覆盖声明规格的 `SpecFingerprint`。
+- [`measurement`](src/measurement.rs)：精确的 6 计数器 Token 核算（`inputTokens` 归一化后包含读写缓存；`totalTokens` 原样保留不重算求和），并以 `PricingRates`（`DEEPSEEK_V4_FLASH_COST`）作为唯一 USD 计价权威。
+
+## 已裁定 `v2 -> v1` 语义规则
+
+| 场景 | `v2` 裁定 | `v1` 投影与选样 |
+| --- | --- | --- |
+| 超时 + 有 reward + `LowerBound` | `Selected(SubjectFailed)` | `subject_failed`、保留 `score`、`costUsd: null`；直接入选且不重跑 |
+| 审计日志截断 / 策略错误 | `Selected(Completed)` | `completed` 并输出完整 `egress-audit` 产物（含 `bytes`、`sha256`、`malformedLineCount`） |
+| 缺失审计日志 / 关键产物 | `RetryableInfra`（若已被第一关判未决则保持未决） | `infra_failed`、`score: null`（比旧版 TS 保留分数更严格） |
+| 迟到取消 | 验证前 `BlockedIndeterminate`；验证后 `Selected` | `indeterminate`（`score: null`）vs 保留分数并附加标记 |
+| 事后证据恢复 | 通过 `PostHocRecovery` 单调叠加 | 派生视图更新状态与计量，不原地改写原始 attempt |
+
+[源码](src/lib.rs) · [Cargo 配置](Cargo.toml)