Skip to content

[Bug]: /btw side questions bypass pruning — session.generate is never pruned, so they fail at the provider limit #685

Open

Description

@Ariche2

Summary

/btw ("Ask a side question") sends the session's unpruned transcript. DCP's V2 setup registers its transform on context and compaction only, so the generate flow — which is what /btw uses — gets no pruning, no block substitution and no tool-output cleanup. On any session whose stored history exceeds the model window (but whose pruned context fits comfortably), the side question dies with:

Upstream request failed: [invalid_request_error] This model's maximum context length is
1048576 tokens; the prompt has 1056995 tokens (max_model_len)

The sidebar meanwhile reads 39%, because that number is the last request's usage, not the transcript.

Environment

  • OpenCode 2.2.7, web app
  • @tarquinen/opencode-dcp 3.2.0 (npm latest)
  • Model: opencode-go/deepseek-v4.1-flash, limit.context = 1,048,576
  • Config: maxContextLimit: "75%", minContextLimit: "50%", compress.mode: "range", compress.permission: "allow", debug: false

Why this does not need a failing session to demonstrate

prepare in packages/core/src/session/model-request.ts only adjusts maxTokens (outputLimit); nothing in the generate path trims messages. DCP's transform is registered on context and compaction only. Together that means a /btw request cannot contain any pruning — it follows from the hook list alone, no long session required.

A session that overflows the window is only what turns that from waste into a hard failure:

  1. Work in a session until the stored transcript since the last completed compaction exceeds the model window, while the DCP-pruned view stays well under it (any long session with heavy compress use gets there).
  2. Run /btw and ask anything.
  3. The request is rejected by the provider as over max_model_len.

Nothing is required from the model side — it fails before inference.

What it costs on a session that fits

On every session with an active compressed block, /btw sends the pre-compression text instead of the summary. Two consequences: the side answer is generated from a different view of the conversation than the main agent ever sees, and the request pays for tokens DCP already decided were reclaimable. The overflow in this report is the extreme case of the same thing.

Measurements

Session: 1,223 messages, last completed compaction at message 284 (summary, not native).

Stored transcript since that compaction 4,170,146 characters
Same, at V2's own 4 chars/token estimate (packages/core/src/session/model-request.ts:58) ~1,043,000 tokens
Plus system prompt, instructions, tool definitions and the question 1,056,995 tokens (the number in the error)
Last normal turn (input + cache.read + output + reasoning) 389,084 tokens = 37%
Sidebar 39% — packages/gui-plugins/src/context/indicator.tsx uses lastAssistantWithUsage, i.e. the last request, not the history

So the two flows are reading the same history and sending wildly different prompts, which is what points at the hook below.

Root cause

/btw is the built-in session.aside command; it calls client.session.generate → POST /api/session/:id/generate → SessionGenerate.generate (packages/core/src/session/generate.ts), which builds the transcript with SessionHistory.preview(...) — the full stored history since the compaction boundary, not a pruned view.

Every flow then goes through SessionModelRequest.prepare, which fires a different session hook per kind (packages/core/src/session/model-request.ts, the Service.of({...}) at the end):

primary: (input) => prepare("primary", input, agentHook("context", input.agent)),
generate: (input) => prepare("generate", input, agentHook("generate", input.agent)),

DCP's V2 setup (lib/v2/index.ts:181 on dev) registers only two of those:

for (const kind of ["context", "compaction"] as const)

generate is therefore the one model request kind that never reaches project / prune / view.restore(). Nothing on the OpenCode side trims messages to the window (prepare only adjusts maxTokens via outputLimit), so the raw history goes to the provider unchanged.

Suggested fix

Add "generate" to that loop, and gate the parts that should not run for a transient, tool-free call. Inside the handler, injectCompressNudges is already gated (kind === "context", lib/v2/index.ts:218); injectMessageIds (227), applyPendingManualTrigger (228) and createSystemPromptHandler (231) are not.

/btw's own system prompt is "Do not call any tools and do not take any actions", so injecting the compress-tool protocol and the <system-reminder> block into it is the same shape as #218 (small_model / title calls), and consuming a pending manual trigger on a throwaway request is a side effect the user cannot see.

Roughly:

for (const kind of ["context", "compaction", "generate"] as const)
    await ctx.session.hook(kind, (event) => serial(event.sessionID, async () => {
        // ... unchanged: load, project, stripHallucinations, sync*, prune, restore
        event.messages = view.restore()

        if (kind !== "context") return

        injectMessageIds(state, config, view.messages, priorities)
        applyPendingManualTrigger(state, view.messages, logger)
        // ... and the createSystemPromptHandler block
    }))

With that, /btw on the session above would send ~389k tokens instead of ~1.06M.

The delete event.tools?.compress lines (189/192) need no change: generate passes a real tools snapshot, and the ?. added by #678 already covers hosts that omit it.

Verifying a fix

No overflow needed, and no second large session. DCP already writes the transformed context per session when debug is on (lib/logger.ts → saveContext → ~/.config/opencode/logs/dcp/context/<sessionID>/<timestamp>.json):

  1. set "debug": true
  2. in any session that currently has an active compressed block, turn once normally, then run /btw on the same session
  3. compare the two newest files in that directory. The normal turn's contains the block summary; after the fix the /btw one should too. Before the fix it contains the original, uncompressed messages.

A unit test is cheaper and matches the existing tests/v2-*.test.ts shape: register the plugin against a fake ctx, fire generate with a synthetic event whose messages span a compressed range, and assert event.messages comes back with the block in place and event.system untouched.

Workaround

/compact the session first (creates a new boundary, so both flows read less), or ask the question as a normal prompt instead of /btw.


Happy to send this as a PR against dev if you'd rather review a diff than write it — the change is lib/v2/index.ts only, plus a test asserting that a generate event comes back with pruned messages and an untouched system array.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions