[Oscar-Williams]

Summary

The desktop context usage indicator asks the CLI for get_context_usage with estimateOnly: true. That path serialized the whole message list as JSON and ran the rough character estimator on it, so every screenshot's base64 counted as text (about one token per four characters). A session with dozens of browser screenshots could show Messages near the full window while the provider still reported a modest input size.

The Messages category breakdown had the same problem: each content block was stringified before estimating.

Fix

• When estimateOnly is true, count messages with roughTokenCountEstimationForMessages (mapping API role/content into the shape that helper already accepts) and add a rough JSON estimate for tool schemas only.
• In the Messages breakdown, score each block with roughTokenCountEstimationForBlock, which already returns 2000 for image and document.
• Export that helper so the breakdown can call it. Cover a top level image, a tool_result that nests an image, a document block, and a user message that mixes text with an image.

How to see the bug on base

On 26cb655f2f26412d6792e20d3c6483a4d4bb3d55:

bun -e 'import { roughTokenCountEstimation } from "./src/services/tokenEstimation.ts"; import { jsonStringify } from "./src/utils/slowOperations.ts"; const data="A".repeat(800_000); const messages=[{role:"user",content:[{type:"image",source:{type:"base64",media_type:"image/png",data}}]}]; console.log(roughTokenCountEstimation(jsonStringify({messages,tools:[]})))'

That prints about 200033. After this change, estimateTokensForContextMessages on the same payload returns 2000. Nested tool_result images and document blocks show the same tip versus fix gap (~200050 / ~200035 tip, 2000 after).

Test plan

• bun test ./src/utils/__tests__/analyzeContext.test.ts: fails on the tip stringify path (~200033 / ~200050 / ~200035), passes after the fix (2000 / 2000 / text tokens + 2000)
• bun test: 2251 pass, 2 skip, 0 fail
• cd desktop && bun run lint: tsc --noEmit clean

Closes #230

[GoDiao]

Thanks @Oscar-Williams, and welcome to DreamCoder! This fixes the main case in #230: screenshots inside tool_result blocks are now scored as 2000 tokens in both the estimate-only path and the Messages breakdown, and the new estimate-only tests use hardcoded numbers. CI is green. Two things before we can merge it.

1. The processAssistantMessage / processUserMessage change has no test.
These two edits decide toolResultTokens, toolResultsByType and toolCallTokens, which is the second half of the issue. If I put them back to roughTokenCountEstimation(jsonStringify(block)), all 7 tests in analyzeContext.test.ts still pass. The three roughTokenCountEstimationForBlock tests only fail without the PR because of the new export; they assert behaviour that already existed.

Could you export processUserMessage and processAssistantMessage, and add a test that builds an empty MessageBreakdown, passes a user message with a tool_result holding a large base64 image, and expects toolResultTokens and the entry in toolResultsByType to be 2000? A similar case for processAssistantMessage would be enough on the assistant side. Please check that the test fails without the change. The three block tests overlap with the estimateTokensForContextMessages ones, so you can drop them if you like.

2. processAttachment has the same bug.
Sorry, issue #230 did not mention this place, so it was not in your brief. processAttachment in analyzeContext.ts still does roughTokenCountEstimation(jsonStringify(msg.attachment)). An @-mentioned image or PDF becomes a file attachment whose content.file.base64 holds the whole payload (attachments.ts around L3133-3155, FileReadTool.ts), so attachmentTokens is still counted as text. The headline total is not affected, because totalTokens goes through normalizeMessagesForAPI and your estimator. The per-category numbers then no longer add up to the total (visible in the CLI /context view).

This is the same cause as #230 in a third place, so it fits in this PR. roughTokenCountEstimationForMessage is already exported from tokenEstimation.ts, and it normalises the attachment before estimating, so something like this should do:

const tokens = roughTokenCountEstimationForMessage({ type: 'attachment', attachment: msg.attachment })

Please add a test with a file attachment that carries a large base64 image. Note that this also changes the numbers for other attachment types slightly, because the wrapper text is now counted; that is closer to what the API sees.

Also:

• main has moved on by one commit, so please merge it into your branch. The CHANGELOG entry should stay under ## [Unreleased], not under the 0.8.0 section.
• Please run cd desktop && bun run lint and bun test --randomize --seed=1 src/utils/__tests__/analyzeContext.test.ts before pushing.

Thanks again!