Repository navigation
Conversation
frf12
left a comment
frf12
left a comment
There was a problem hiding this comment.
Reviewed only the three commits in #2435, from 82cde710d9244d8c16d349fa23e3bce88eef8b38 to c1f9e3dd9eb08020d5535758b68d6c52cd7582a6. Two issues introduced by the prefixed-extraction path are described inline; the existing #2429 findings are not repeated. @jasper-yl could you share how you would handle these two cases?
| conversation_id: Some(prefix.conversation_id), | ||
| parent_conversation_id: None, | ||
| sediment: false, | ||
| prepared_contexts: Vec::new(), |
There was a problem hiding this comment.
[P2] Prefix extraction drops the original admission context
The stored prefix contains the actual sent body, including any injected prepared-context block, but this call clears prepared_contexts. Core then rejects that block with context-binding-required, even while its original binding is valid. Similarly, setting parent_conversation_id to None above prevents a subagent prefix from presenting the parent's receipt for its initial encrypted agent_message, producing encrypted-origin-unverified. Both normal cases fall back instead of using the prefix. With the current Python Core, the original requests and prefixed requests retaining their admission context allow; clearing these fields reproduces the failures. The parent case uses one child response and unexpired receipts, independently of #2429's receipt-count issue. Rust save/replay paths were inspected, not run end to end.
There was a problem hiding this comment.
@jasper-yl This admission-context issue is reachable again on 67fcfdaa: injected turns now retain the sent prefix, but PrefixEntry no longer stores the original prepared-context locations and the extract call still supplies an empty list. I replayed the same Responses payload against Core's actual checker in disabled, shadow, and advisory modes: with the original valid binding and location it allows; without them it raises context-binding-required in all three modes. Thus this also affects the non-enforcing deployment where the updated PR says prefix extraction works, even when the prefix has no encrypted reasoning. The Core check was executed with synthetic payload/binding; I inspected the gateway capture/replay path rather than running a live provider. Could the retained prefix preserve and present its original prepared locations and bindings while valid, and fall back when they expire?
|
|
||
|
|
||
| def _refusal_reason(payload: dict[str, Any]) -> str: | ||
| reason = _GATEWAY_REASONS.get(payload.get("error")) # type: ignore[arg-type] |
There was a problem hiding this comment.
[P2] A standard provider error object prevents fallback
A normal upstream 429 body such as {"error":{"message":"Rate limit reached","type":"rate_limit_error","code":"rate_limit_exceeded"}} reaches this lookup with a dict as the key, raising TypeError: unhashable type: 'dict'. The pipeline catches neither this exception nor invokes its normal extractor: a reproduction through AtomicMemoryGenerationPipeline.extract ended with fallback_calls=0. This response shape is reachable through the new route: Core accepts a complete provider error response, and the gateway's non-2xx Released branch passes its status/body through respond. Consequently, a routine provider error aborts this extraction instead of taking the advertised fallback. The Python reproduction used a simulated HTTP response; Core error admission and the Rust forwarding path were checked separately.
|
@frf12 Thanks, both are bugs in this path. Fixed in 1. Admission context.
2. Provider error objects.
Checks: gateway 142 tests and clippy; |
52b6616 to
b517e56
Compare
frf12
left a comment
frf12
left a comment
There was a problem hiding this comment.
Reviewed b517e5652272d4a9260bb0ba8ca21b4e10af0b24 with the receipt-consolidation flow from #2429. One interaction specific to the background prefix-extraction path remains, described inline. @jasper-yl could you take a look and share how you would handle it?
| degrade_policy: pc_gateway_core::DegradePolicy::Pause, | ||
| inject_context: false, | ||
| capture_prefix: false, | ||
| conversation_id: Some(prefix.conversation_id.clone()), |
There was a problem hiding this comment.
[P2] Extracting an older prefix drops the live conversation's newer receipts
This extraction call uses the original conversation ID and runs through the same executor, including receipt consolidation. Consider two completed turns: turn 1 saves a prefix containing reasoning ciphertext R1; turn 2 adds R2, and the conversation's current receipts prove both. If background extraction then processes turn 1's saved prefix, its request contains R1 but presents the current conversation's receipts. Core returns a consolidated receipt covering only R1, and ReceiptStore::store removes all of that conversation's receipts presented by this check, including the proof for R2. The next normal request still replays R2 and is withheld with encrypted-origin-unverified. This needs only two turns and no expiry or capacity limit. I reproduced the Core decisions through the public /v1/context/check route with simulated provider payloads and the receipt replacement sequence: the live request allows before extraction and is withheld after replacement. The Rust extraction/replacement paths were inspected, not run end to end. Background extraction of an older snapshot should not invalidate a continuing conversation.
There was a problem hiding this comment.
@frf12 Right — the extraction call borrows the conversation's receipts but must not trade them in. Fixed in 92f558f00: ManagedCall gained a background flag, set only by the extract endpoint. A background call still presents the live receipts so the older prefix's admitted values verify, but the executor skips both receipt writes: the send-time consolidation and the release receipt. The conversation's own receipts stay authoritative, so the turn-2 proof survives a delayed extraction of turn 1.
The regression test runs your sequence: two live turns leave release-2/merged-2 in the store, a background extraction returns merged-bg/release-bg from the gate, and the next normal request still presents release-2/merged-2 — without the fix it presented the background run's receipts and the R2 replay lost its proof. Gateway suite: 146 tests, clippy clean, make check passes, and the docs now state the receipts are read-only for this path.
21a9168 to
abbe692
Compare
hidb4ai
left a comment
hidb4ai
left a comment
There was a problem hiding this comment.
The prefixed path can attribute previously injected memory to a new Source. Please preserve the Source evidence boundary before using this path.
| return "\n\n".join(( | ||
| instructions, | ||
| "The conversation above is the Source evidence for this extraction. Only its most recent user turn is " | ||
| "evidence: the last user message before this one and everything after it. Earlier turns are context only; " |
There was a problem hiding this comment.
[P2] Keep injected context outside the new turn's evidence
The gateway prepends the prepared context to the last user message before saving the sent prefix, but builds the captured Source from the pristine request specifically to exclude that context. This instruction now tells the model that the entire last user message is evidence, so an old injected memory can be extracted again and cited as if the new turn supported it. The same mismatch occurs for tool output omitted or truncated in the captured transcript. validated_candidates only checks that the cited ID exists; it cannot establish that the fact appears in that Source. In a capture-and-flush probe, a simulated gateway reply citing source:1 persisted a fact absent from the Source without invoking fallback. The provider reply was simulated; the prefix construction and instruction were inspected separately. Please ensure prefixed extraction only uses content actually represented by the cited Source, or fall back when those boundaries cannot be preserved.
There was a problem hiding this comment.
@hidb4ai Thanks, confirmed: the instruction cannot keep the model from using what it sees, and validated_candidates checks only the citation. Fixed in 56cd73193 at the gateway, so the prefixed path is used only when the Source represents the whole turn the model reads.
Which turns keep a prefix. The gateway keeps a prefix only when both hold:
- no prepared block was injected into the request;
- the new
represents_turncheck accepts the captured record. It rejects a record with an omitted step, a truncated field, or omitted binary data. It also rejects a turn whose user message, assistant messages or tool outputs contain non-text parts (images, files), provider-hosted tool calls, or unrecognized items.
Otherwise no handle is recorded, and Core extracts that turn from its Source the normal way.
Consequence. A kept prefix never contains an injected block, so it no longer carries prepared-context bindings.
Tests:
- An executor test runs an injected turn, a turn with an image part, and a turn with a truncated field. None keeps a prefix, and the test fails if the check is bypassed.
- A transcript unit test covers binary tool output, image tool output, hosted calls, unknown items and truncated output.
What the instruction still relies on. The kept prefix also shows the model earlier turns and its own reasoning. The instruction limits evidence to the latest user turn. Earlier turns are real conversation captured as their own Sources, so a fact taken from one would be true but cited to the wrong turn. That rests on the instruction, as before.
Cost to the feature. This narrows where the path applies. In my earlier measurement, the turn that saved about 22% read an 18 KB file, which the Source truncates to 8 KB. Under this rule that turn now falls back. I've noted this in the PR description.
There was a problem hiding this comment.
@jasper-yl Thanks for the fix. I rechecked 56cd73193 and the evidence boundary is still incomplete in two cases:
- A Responses
apply_patch_callhas its patch inoperation.textual_itemaccepts everyClientCall, butitem_stepsdoes not copyoperationinto the transcript. With anupdate_filepatch containingDeploys require two approvalsfollowed by a successful tool output, the gateway captures a Source without that text while retaining a prefix whose request contains it. represents_turnchecks the request but not the terminal response. A terminalmcp_callwith anerroris therefore still retained even though the transcript drops the error. This also bypasses the intended exclusion of provider-hosted calls.
I reproduced both through the Rust executor: each captured Source omitted the relevant text and nevertheless received a usable prefix handle. The gate and upstream responses were simulated; I did not test a live provider or claim it produced a bad memory. For the first case the omitted patch is in the replayed request, so a valid model response can still cite the Source for content it does not contain. Could the eligibility check cover the actual fields represented by each supported tool call and the terminal response as well?
There was a problem hiding this comment.
@hidb4ai Thanks. Both cases reproduce as you describe. Before I extend the eligibility check further, I'd like to question the approach, because measured on real sessions it disables the feature.
Data. I ran the current branch end to end: three SWE-bench Pro tasks with an unmodified Codex 0.145 against OpenAI gpt-5.6-sol, through the gateway and an enforcing Server.
- Every turn fell back. The eligibility check rejected all three turns, so all three were extracted the normal way. The causes were ordinary coding output, not edge cases: the transcript caps truncated tool output (8 KB per field) and omitted early steps (64 KB total). qutebrowser lost 11 steps and 1 field, ansible 9 steps and 2 fields, openlibrary 2 fields.
- The prefix path itself works. On the same Sources, the prefixed extraction call cost 7.5–8.7k units against 18.0–19.5k for normal extraction (cached input at 0.1, output at 8).
Requiring the Source to contain everything the model reads means the path almost never runs. Adding apply_patch_call operations and terminal responses to the check would narrow it further.
I think two different things are mixed together here.
- Content that is not evidence: injected memory. I agree this must not be attributed to the new turn. But it is identifiable: it sits between
BEGIN_POWERCONTEXT_PREPARED_*markers at a position the gateway records. The instruction can exclude it explicitly. Any re-extracted duplicate is then compared with the same existing memory during reconciliation, so it becomes a noop or a merge, not a new memory. We can also refuse to add the new Source as supporting evidence when a candidate matches an injected item. - Evidence the Source records only partially: truncated tool output, omitted steps, an
apply_patchoperation, a hosted call's result, images.- This is real content of that turn. The Source is a bounded normalized record of it, not a complete copy.
- Falling back does not make the memory more correct. Normal extraction simply cannot see this evidence, so facts that only appear there are lost.
- The real gap is auditability: a reader of
source:Nmay not find the text. That can be addressed without discarding the evidence:- the gateway records in the Source metadata what the record omits (truncated fields, omitted steps, non-text items);
- Core records on the extracted memory that it came from the full turn over the prefix, so a reviewer knows the stored Source is a partial record.
Proposal. Keep the prefix for these turns. Exclude injected blocks explicitly. Make the incompleteness explicit in the Source and in the memory's provenance. Keep the cost check, and fall back only for unrecognized items.
If you still consider the Source the only acceptable evidence boundary, I'd like to discuss how to handle that before changing more code. One alternative would be a lossless record for prefix-eligible turns, at the cost of larger Sources. Which direction would you accept?
There was a problem hiding this comment.
@jasper-yl Thanks for measuring this and laying out the tradeoff. I agree that a guard which falls back on all three representative turns does not deliver the intended benefit, and that the omitted tool output can contain useful evidence.
My recommendation is to keep the cached-prefix path, but retain independently resolvable evidence for the content used to support a Memory, with its Scope/access and retention behavior defined. That could be a bounded evidence record or a durable reference to the original material; it does not require putting the entire provider request into a Source. A partial-record marker is useful disclosure, but by itself it cannot let a reader check the cited claim after the 30-minute prefix expires or the gateway restarts. The current Source documentation says each Memory revision retains the exact Source materialization used as evidence.
For injected Memory, an instruction and later reconciliation alone do not establish the boundary: the model may extract a paraphrase or combine old and new facts, and the current noop path combines proposal evidence with the target in the working set. Please make sure injected content cannot acquire the new turn as supporting evidence, including for those cases.
I reran both executor counterexamples on ce0e256f5; they still reproduce. These use a simulated gate and upstream, not a live provider. If the intended product contract is instead to allow irretrievable partial evidence, I think that should be an explicit contract decision rather than silently treating the partial Source as complete. Whichever approach is chosen, the three measured coding sessions would be valuable acceptance cases alongside restart/readback, access and attribution checks.
abbe692 to
56cd731
Compare
hireview001
left a comment
hireview001
left a comment
There was a problem hiding this comment.
- Resolve merge conflicts with the target branch.
- CI failed: CI result (failure).
- CI failed: managed-resource-candidates / Candidate windows-x86_64 (failure).
- CI did not complete: code-seekdb (3.11, 0) (cancelled).
- CI did not complete: code-seekdb (3.14, 0) (cancelled).
- CI failed: Main (failure). Failed steps: code-seekdb (3.14, 0) / Test unified migrations with embedded seekdb; code-seekdb (3.11, 0) / Test unified migrations with embedded seekdb; managed-resource-candidates / Candidate windows-x86_64 / Produce twice, compare bytes, consume catalog, and run bundled launcher/Ops.
| proofs[digest] = _Proof(renewed, digest in own and own[digest].handoff) | ||
| for value in replayed_argument_values(parsed, declared): | ||
| digest = value_digest(value) | ||
| if digest in own or digest in inherited: |
There was a problem hiding this comment.
[P1] Preserve handover permissions when renewing receipts
A before_send request can replay a synthetic call whose declared encrypted argument contains a ciphertext proven only as reasoning; this branch promotes that proof to handover permission, allowing a child to submit the reasoning as an agent_message without an intervening release. Preserve existing handover permissions during renewal and grant new permissions only from released encrypted arguments.
| fn textual_item(item: &Value) -> bool { | ||
| match classify(item) { | ||
| ItemKind::Ignored | ItemKind::ClientCall => true, | ||
| ItemKind::Message => item.get("content").is_none_or(|content| match content { |
There was a problem hiding this comment.
[P2] Reject messages omitted from the captured evidence
A text-only developer or system message after the last user message passes this branch even though item_steps() excludes non-assistant messages from the transcript, so prefixed extraction receives text absent from its cited Source as evidence. Reject these unrepresented messages before issuing a prefix handle.
There was a problem hiding this comment.
复核 head fed3a4d9：该缺口仍在，补两处源码级定位。
1. 记录侧丢内容但不记 loss。 item_steps() 的 ItemKind::Message 分支调用 assistant_text(item)，而 assistant_text 要求 role == "assistant"；developer / system 消息返回 None 后走 unwrap_or_default()，这段文本既不进 steps，也不置任何 CaptureLosses 标志（同为丢内容的 ItemKind::Unknown 分支会置 unmeasured_projection）。
2. 守卫侧只看首个 content 元素是否为文本。 represents_turn() 对 user 消息之后的 item 只检查 message.content.first().is_some_and(textual_item)；textual_item 的 ItemKind::Message 分支只校验 content 是文本，不校验 role。于是上面那段被记录丢弃的消息仍被判为「完整呈现」，keep_prefix() 照常写入 prefix_handle。
3. 同根因的另一个维度：守卫不读已算好的 coverage。 represents_turn() 只读 omitted_steps 与 has_elided_text()，没有读 turn_transcript() 已经算出并写入 Source 元数据的 powercontext_capture_coverage。同一文件里另有 5 处（多部件 message、未测量的调用载荷、未知 output 等）只置 unmeasured_projection；这些形态目前只是被 textual_item 的类型分支偶然挡住，任何将来只置标志的 lossy 分支都会漏过。
建议：
represents_turn()增加「coverage 声明无任何 loss」的前提（把CaptureLosses/declaration()传入，unmeasured_projection、field_byte_limit、transcript_byte_limit任一为真即返回 false）；- user 消息之后的 message item 改为按
assistant_text同一 role 过滤器判定； - 补一个用例：最后一条 user 消息后跟
{"type":"message","role":"developer","content":[{"type":"input_text","text":"…"}]}，断言represents_turn为 false 且该回合不产生prefix_handle。
验证口径：以上为源码级判断，基于 head fed3a4d9 镜像内 gateway/crates/pc-gateway-core/src/transcript.rs 的 item_steps / assistant_text / represents_turn 三处逐行核对；本轮未在本机执行 cargo test，没有运行时反证，这一项记为验证缺口。
There was a problem hiding this comment.
Addressed in 74fbedca6. A message of another role after the last user message (developer, system) is now a message step in the captured Source, and the eligibility check is gone. See responses_items_keep_every_field_the_model_read.
56cd731 to
ce0e256
Compare
|
|
|
@jasper-yl Thanks for the measurements and for explaining the tradeoff. Could we combine your cached-prefix approach with complete capture of the current turn? The direction I would suggest is:
Could you consider this together with your proposal and share your view on the implementation effort and any practical constraints, including the existing capture-size limit? We would like to hear what you think before settling the implementation details. |
|
@frf12 Agreed, this combines the two well. Effort and constraints for each point: 1. Complete capture of the current turn. Gateway-only and moderate.
2. Injected memory.
3. Use the prefix whenever available and cheaper. Removes the representation check ( 4. Fallback. Unchanged: normal extraction, with the existing reason codes. If this matches what you had in mind, I'll implement it on this PR after #2429 merges, with tests for an oversized turn split across Sources, a noop and a merge against an injected item, and a prefixed extraction over a turn with injected memory. |
| generation_stream_idle_timeout_seconds: float = Field(default=60.0, gt=0) | ||
| generation_model_context_window_tokens: int = Field(default=125_000, ge=1) | ||
| # Opt-in: extract Atomic Memory over the agent's cached conversation prefix through a gateway on the same host. | ||
| prefix_extraction_gateway_url: AnyHttpUrl | None = None |
There was a problem hiding this comment.
[P2] 新增的三个 Server 开关没有登记到 .env.example
InferenceConfig 新增 prefix_extraction_gateway_url、prefix_extraction_gateway_headers、prefix_extraction_cached_input_ratio；docs/{en,zh}/docs/operate/configuration.md 已同步记录 POWERCONTEXT_SERVER_INFERENCE_PREFIX_EXTRACTION_* 三项，但仓库根的 .env.example 仍是改动前的状态（该文件列到了 ..._GENERATION_MODEL_CONTEXT_WINDOW_TOKENS 这种同层级开关，并给 base_url/headers 留了注释示例）。
影响：按 .env.example 作为模板部署的运维看不到这个 opt-in 开关，也无法从模板得知 headers 需要 secret 处理；prefix_extraction 默认关闭，缺登记时该功能在实践中等于不可发现。这属于仓库约定的新开关登记面缺口（env 模板与两份配置文档需要对齐），不是运行时缺陷。
建议：在 .env.example 的 inference 段补三行，..._GATEWAY_URL 与 ..._GATEWAY_HEADERS 按既有 ..._GENERATION_HEADERS 的形式给注释示例，..._CACHED_INPUT_RATIO=0.1 给默认值；同时确认 headers 的 secret 处理与既有 generation/embedding headers 一致。
There was a problem hiding this comment.
Fixed in 74fbedca6. .env.example now lists the three settings in the inference section, commented out like the other optional endpoint overrides:
POWERCONTEXT_SERVER_INFERENCE_PREFIX_EXTRACTION_GATEWAY_URL, with the gateway's default listen address;POWERCONTEXT_SERVER_INFERENCE_PREFIX_EXTRACTION_GATEWAY_HEADERS, in the same form as..._GENERATION_HEADERS;POWERCONTEXT_SERVER_INFERENCE_PREFIX_EXTRACTION_CACHED_INPUT_RATIO=0.1.
A note above them says the feature is off unless the URL is set, needs GENERATION_MODEL for the fallback, and that the headers are secrets. They are handled like the generation and embedding headers (SecretStr values). test_env_example_loads_server_settings passes.
|
@jasper-yl Thanks for combining the complete-capture, marker-exclusion, cost-selection and normal-fallback directions. I do not think we should adopt the proposed general rule that a Please keep the normal reconciliation semantics. The requirement for this PR is that the injected block is explicitly marked and excluded as supporting evidence for the new turn; it should not automatically change the action selected for a genuine new update. If you see a narrower provenance-aware rule that distinguishes an echo from a new user-supported fact, please propose it separately before changing that behavior. For an over-limit turn, splitting the current turn into consecutive Sources seems reasonable to me. Please make sure the resulting Source window remains usable by normal extraction and that the prefix handle/provenance clearly identifies the complete turn. |
hireview001
left a comment
hireview001
left a comment
There was a problem hiding this comment.
- Resolve merge conflicts with the target branch.
| // 只有 Source 完整呈现了模型将看到的这一回合时才保留前缀：注入的记忆块、截断或省略的内容 | ||
| // 都不在 Source 里，不能被当作这一回合的证据。 | ||
| let represented = call.prepared_contexts.is_empty() | ||
| && represents_turn( |
There was a problem hiding this comment.
[P2] Check terminal response items before retaining a prefix
When terminal Responses output contains a hosted tool call such as mcp_call, this eligibility check examines only the request, retaining a prefix even though turn_transcript omits fields such as mcp_call.error. Validate the terminal response too and fall back for unrepresented output, otherwise extraction can cite a Source that lacks the supporting content.
There was a problem hiding this comment.
Addressed in 74fbedca6 by the complete-capture direction agreed above, which also removes this eligibility check. The captured Source now records the whole turn, including hosted calls in the terminal response: every argument field, plus output, outputs, results and error. An mcp_call with an error is recorded with that error as its tool output (responses_items_keep_every_field_the_model_read). So the Source holds everything the extraction reads as evidence, and no request- or response-side eligibility check is needed.
| conversation.receipts.pop_front(); | ||
| } | ||
| conversation.receipts.push_back(Receipt { | ||
| id, |
There was a problem hiding this comment.
[P2] Retain receipts added after an in-flight snapshot
With 33 concurrent requests sharing a presented snapshot, completed releases can fill the store before later consolidations finish. This cap then evicts an unexpired release receipt added after the snapshot, even though the replacement cannot contain its proof. A store-level concurrency reproduction loses that proof; a subsequent request needing it can be withheld. Preserve newly added proofs across concurrent consolidation rather than silently evicting them.
PC_GATEWAY_PREFIX_EXTRACTION=on keeps, for a Responses conversation, the exact request and the response output of each turn's terminal response in a bounded, expiring store, and records an opaque handle and the prefix's token counts in the captured Source's metadata. POST /gateway/v1/extract appends Core's extraction instruction to a kept prefix and sends it through the normal context checks. A prefix is kept only for a turn its Source fully represents. Core tries extraction over the prefix for a window holding exactly that one turn, when it is estimated to cost less than normal extraction, and falls back to normal extraction with a bounded reason otherwise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ce0e256 to
fed3a4d
Compare
|
Rebased onto master The extraction request now goes through the plain The direction agreed above (complete capture of the current turn, marker exclusion, split Sources for an oversized turn, and the |
…heaper The gateway now records the complete turn the model read, as projection gateway-turn-text.v2: no field is truncated and no step is omitted, and every argument field of a tool call, tool outputs and errors, other messages, visible reasoning summaries and unrecognized items are included. A turn over the 200,000-character Source limit is captured as consecutive parts; the prefix handle is on the last part. A prefix is kept for every captured turn, including turns with injected memory, since the Source comes from the pre-injection request. The extraction instruction states that content between PowerContext markers is not evidence. Core accepts a window holding exactly one whole turn, in one Source or all of its parts. Register the extraction settings in .env.example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@frf12 The agreed direction is implemented in
One consequence to note: gateway capture records whole turns whether or not prefix extraction is enabled, so gateway Sources are larger than before. Checks. Gateway: 213 tests passed, clippy clean. Core: the related Python suites passed (303 passed, 4 skipped), and |
…n input A complete turn of a real coding session often exceeds the 24,000-token default Atomic Memory input, and one oversized Source fails the Scope's memory flush. Parts are now sized by Core's character estimate of the escaped text, at most 12,000 tokens, instead of the 200,000-character content limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hidb4ai
left a comment
hidb4ai
left a comment
There was a problem hiding this comment.
@jasper-yl Thanks for the update. I found two issues with split-turn handling, described below. I also replied to the existing prepared-context binding thread with a separate regression in this head.
| def prefix_handle(value: AtomicMemoryExtractionInput) -> str | None: | ||
| """The gateway prefix handle of a window that holds exactly one gateway turn, which its last Source carries.""" | ||
|
|
||
| if not value.evidence or not _one_turn(value.evidence): |
There was a problem hiding this comment.
[P2] Keep split turns together through the extraction budget check. _prepare applies the ordinary extraction input budget before this code can select the prefixed path. With three 45,000-character pieces, each below the 12,000-token capture limit, a real SQLite-backed Server accepts all three Sources but the first /v1/memory/flush returns 500 because their combined input exceeds the default budget. Subsequent flushes process the pieces separately through normal extraction, so _one_turn never sees the complete turn and the prefix is never used. I reproduced this with a scripted model gateway and extractor. Please account for the actual prefixed request budget and preserve complete-turn handling when recovering from an oversized window.
| Err(GateError::GateRejected(409)) => { | ||
| evidence.record_not_sent(0, "sediment_duplicate") | ||
| } | ||
| Err(_) => evidence.record_not_sent(0, "sediment_failed:gate_error"), |
There was a problem hiding this comment.
[P2] Avoid publishing a partially captured turn as complete Sources. Capture failures are logged and the loop continues, so if an early part is rejected but later parts succeed, those later parts are stored independently (with the final part carrying the prefix handle), while the missing part is never retried here. Core cannot recognize a complete multipart turn and can consume the surviving fragments through normal extraction, advancing past an incomplete record of the interaction. A Rust executor probe with a scripted gate that rejects the first part and accepts the rest reproduced this flow. Please ensure partial captures are completed or explicitly prevented from being processed as a complete turn; the user-facing response can still remain fail-open.
Which issue or RFC does this PR close?
Closes #2413, as agreed there for a gateway and Core deployed on the same host.
The extraction request replays the conversation, including the provider's encrypted reasoning. Since #2457 a non-enforcing Server admits it, so this path works there. An enforcing Server withholds it, and extraction then falls back to normal extraction (
check_rejected).Rationale for this change
Atomic Memory extraction for a gateway turn sends Core's instructions plus the turn's normalized transcript as uncached input. The agent's provider already holds the whole conversation in its prompt cache. Appending the extraction instruction to that exact prefix lets the provider read the conversation from the cache.
Cached input is still billed, at about 10% on OpenAI. So reusing the prefix is cheaper only when the turn's transcript is large relative to that share of the prefix. That holds for turns with substantial tool output, and stops holding as a long conversation's prefix grows. The Server therefore estimates both requests and reuses the prefix only when that is cheaper (measurements below).
What changes are included in this PR?
Gateway
PC_GATEWAY_PREFIX_EXTRACTION=on(off by default; requiresPC_GATEWAY_SEDIMENT=on). For a Responses conversation, at the terminal response of each turn, the gateway keeps in memory:Limits are 64 entries and 30 minutes. The new
PrefixStoreholds them.The Source records the whole turn (new projection
gateway-turn-text.v2), so it holds everything the model reads as evidence and stays checkable after the prefix expires:mcp_call.error), other messages after the user's, visible reasoning summaries and unrecognized items;data:binary content becomes a placeholder with its media type and size.The capture-coverage contract, the shared fixtures and the Hermes helper accept the new profile, and
gateway-turn-text.v1stays readable for existing Sources.Oversized turns. Each Source must fit Core's default Atomic Memory input (24,000 estimated tokens) on its own: one oversized Source fails the Scope's memory flush. So a turn over 12,000 tokens, by Core's character estimate of the escaped text, is captured as consecutive Sources
<source_id>-<i>of<n>. Their metadataturnnames the turn and the part. A single step that does not fit has its longest text field split into consecutive steps marked"continued": true. Normal extraction reads the parts like any other Sources.Injected memory. A prefix is kept for every captured turn, including one with injected memory. The Source comes from the pre-injection request, so the injected text never becomes evidence. The kept prefix is the request actually sent, injected blocks included.
The captured Source's metadata (the last part of a split turn) gets:
prefix_handle;prefix_input_tokensandprefix_output_tokens, from that response's usage.The request and response never enter the Source, evidence, or logs.
POST /gateway/v1/extract(gateway key required when configured; 404 when disabled) takes{handle, instruction, max_output_tokens?}:id/statusremoved as clients replay them, then the instruction as one user message.tool_choice: "none"andstream: false. Every other field stays unchanged, including tools,prompt_cache_key, reasoning settings and instructions.before_send/before_releasechecks.{text, usage, model}.Core
inference.prefix_extraction_gateway_urlandprefix_extraction_gateway_headers(secret);prefix_extraction_cached_input_ratio, default 0.1, in (0, 1].AtomicMemoryGenerationPipeline.prefixedis tried only for a window that holds exactly one whole turn: one Source, or every part of one turn in order, with the handle on the last.Instruction: the active extraction instructions (built-in or custom Prompt), plus statements that:
BEGIN_POWERCONTEXT_<NAME>andEND_POWERCONTEXT_<NAME>markers is not evidence.It adds the Source records (evidence ID, position, provenance, without the transcript or gateway metadata) and the output JSON Schema. The schema goes in the message, because
text.formatwould change the cached request.Reconciliation is unchanged.
Cost check: the prefixed request is estimated as the instruction, plus the final response's output tokens at full rate, plus
ratio ×the cached input tokens. Normal extraction is the instructions plus the evidence, estimated like the extraction budget. The prefix is used only when the first is lower.Validation: the reply is validated by the same schema, content and citation rules as normal extraction, now shared as
validated_candidates.Fallbacks to normal extraction, each with a
fallback_reason:not_cheaper,prefix_size_unknown,prefix_unavailable,target_mismatch,disabled,invalid_instruction,check_rejected,gateway_unavailable,provider_error(a provider error object passed through),gateway_error,invalid_output.Usage: reported through the existing generation usage reporting, including cached input tokens.
atomic_memory.prefixed_extractionevent withoutcomesuccess/fallbackandfallback_reason. It uses already-allowlisted fields and carries no handle or content.Config template: the three Server settings in
.env.example.Docs: in English and Chinese:
deploy-server.md,configuration.md,observability.mdand the capture-coverage contract (development/capture-coverage.md). Alsogateway/README.md.Are there any user-facing changes?
gateway-turn-text.v2) whether or not prefix extraction is on. Sources can therefore be larger, and a long turn becomes several Sources.turnare ordinary Source metadata.How was this change tested?
Gateway:
cargo test: 213 passed;cargo clippy --all-targets -D warningsclean.gateway-turn-text.v2complete;apply_patch_calloperations, MCP approval answers, hosted call errors, unknown items, and image placeholders;turnand only the last has the handle;PrefixStorerandomness, Scope binding, expiry and bound.Core:
tests/builtin/artifacts/atomic_memory/test_prefixed_extraction.py: 16 tests through the real Server capture-and-flush path.not_cheaperbefore any gateway call; a missing size falls back withprefix_size_unknown.make checkpasses.End to end, real API. Setup: an unmodified Codex CLI 0.145.0 against OpenAI
gpt-5.6-sol, through a gateway and Server built from this branch, enforcing mode. A recording pass-through between both and OpenAI captured each response'susage. Normal extraction ran on the identical Source in a second Scope. Cost is input-equivalent, with cached tokens at 10%.not_cheaper, normal extraction (2,063)openai-responsesgeneration is withheld in enforcing mode for OpenAI reasoning models (encrypted reasoning in the reply), which is unrelated to this change.AI usage statement
Built with Claude Code (Claude Opus 5.5), which wrote the change, the tests and the docs, and ran the checks and measurements above.
🤖 Generated with Claude Code