[Bug]: the local context footprint over-estimates provider-reported usage by 4-28% on every turn, which makes compaction fire early #600

Activity

1. UncleMoc commented on Nov 11, 2026

   

   UncleMoc

   on Nov 11, 2026

   Author

   More actions

   Two more findings that make this look like a wiring gap rather than a design choice.

   The provider's own number is already available on another path. ContextManager in packages/agent-modules/context-manager/src/manager.ts resolves its count through the injected token counter first:

   count = this.tokenCounter
     ? await this.tokenCounter.countContextTokens({
         messages, model, apiKey, headers, systemPrompt, tools, thinkingLevel, signal,
       })
     : { tokens: this.tokenEstimator.estimateContextTokens(messages).tokens, source: 'local_estimate' };

   and packages/local-runtime/src/context/remote-token-counter.ts documents that endpoint as ground truth, with the local estimate only as the fallback for a transient failure:

   The provider token counter endpoint is ground truth; failures fall back to the shared BPE estimate so a transient network/provider issue does not block the user turn.

   It returns a source-tagged result (source: 'remote' | 'estimate') so the caller can tell ground truth from an estimate, and packages/local-runtime/src/context/context-usage-calibration.ts already calibrates the tool portion against it.

   The comment above that call site also records a trap worth repeating here — headers and tool declarations have to be forwarded unchanged, because "tool declarations are part of the real LLM payload (tens of thousands of tokens in MCP-heavy sessions). Omitting them systematically undercounts and delays compaction triggers."

   The automatic compactor does not use it. packages/local-runtime-v2/src/service/turn-system/compaction/automatic-context-compactor.ts decides through createLocalContextFootprintMeasurer instead, and that measurer is local-estimate only — we found no reference to countContextTokens or to the remote counter anywhere under packages/local-runtime-v2/src/.

   So the two paths disagree about where the number comes from: one asks the provider, the other estimates. That is consistent with what we measured (the estimate runs above the provider's number on the same turn), and it also means the fix we are asking for already exists in the codebase — just not on the path that decides when to compact.

   Question: can the v2 compaction trigger reuse ContextManager's counting, or is the separation deliberate? If it is deliberate, we would like to understand why, because the effect is visible to users as compaction firing while the context ring still shows room.
2. UncleMoc commented on Nov 11, 2026

   

   UncleMoc

   on Nov 11, 2026

   Author

   More actions

   The numbers in this issue reproduce exactly — and the thinking correction does not apply to the hosted path

   The reporter in #598 §9 re-ran our measurement and confirmed the direction and magnitude. We re-ran it ourselves after reading that, so here is the reproduction with the arithmetic spelled out.

   Same six sessions, same formula: estimate(systemPrompt) + estimate(JSON.stringify(tools)) + Σ estimate(messages before this turn) on the local side, usage.input + usage.cacheRead + usage.cacheWrite on the provider side for the same assistant message.

   session	turns	ratio (local / provider), median	tool-result tokens
   A	447	1.164	246,567
   B	258	1.207	190,497
   C	207	1.278	139,677
   D	183	1.076	109,119
   E	30	1.102	19,228
   F	395	1.044	200,515

   Every row matches the published figure.

   One thing to include when re-running this. The accumulated message count must include the compactionSummary message itself. All six sessions start with one — the summary is the context root — and omitting its tokens shifts each ratio down by 0.02-0.05, which looks like a different measurement entirely.

   On the thinking question, the answer depends on the provider, and for the hosted path it is the opposite of what #598 §9 concluded. The drop is shouldDropPortableThinking in outbound-message-normalizer.ts:

   function isSourceQualifiedByokProvider(provider: string): boolean {
     return provider === 'minimax_api' || provider.startsWith('custom_provider:');
   }

   All of our sessions are provider: "minimax", api: "anthropic-messages" — the hosted path, not the BYOK tree — so shouldDropPortableThinking is false and the thinking block is kept (same-model messages are returned untouched; a sibling anthropic-messages model replays the native block).

   Measured, not inferred. Same arithmetic, counting vs. excluding the thinking blocks:

   session	ratio with thinking	ratio without thinking
   A	1.164	1.031
   B	1.207	1.098
   C	1.278	1.106
   D	1.076	0.854
   E	1.102	0.928
   F	1.044	0.850

   Excluding thinking moves every session below 1 — the provider counts those tokens and the local estimate should too. On custom_provider:* / minimax_api the reporter's reading is the correct one and their numbers are the ones to trust; on minimax + anthropic-messages the published numbers are the right ones, and the local estimate really is above what the provider reports. The adjacent-turn difference form (which cancels the fixed part) agrees: median 0.99-1.08, in the same range as their 1.038.

   The ask is unchanged. The gap is real on the hosted path, it is not an artefact of the counting convention, and the fix is the same: take the provider's own number as the primary source for the compaction decision, keep the local estimate as the fallback when the provider cannot be asked. ContextManager in packages/agent-modules/context-manager/src/manager.ts already resolves through the injected counter first, and packages/local-runtime/src/context/remote-token-counter.ts treats the provider endpoint as ground truth with the BPE estimate only as the degradation path — while the v2 compaction path under packages/local-runtime-v2/src/service/turn-system/compaction/ still recomputes the whole history locally. If that split is deliberate, we would like to understand the reasoning before assuming it is an oversight.