Budget the base prompt by section (and fix the custom-rules framing)#498
Conversation
Adds PromptSectionBudget, a pure character-budget allocator, and rewrites BaseCompletionPromptRenderer to build prioritized sections (persona, style, language, notes, clipboard, screen, and the caret prefix) allocated within a total budget. The caret prefix gets top priority, a guaranteed minimum, and preserve-end truncation, so a large glossary/clipboard/screen capture can never crowd it out. Also fixes a framing bug: custom rules now render as their own 'Writing style: ...' line instead of being jammed into 'in a <rules> style', which produced broken prose like 'in a Use British spelling style' for sentence-shaped rules. Sets up the OCR and writing-history context sections.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| /// Which end of the content to keep when it must be shortened. `beforeCursor` keeps its END | ||
| /// (the text nearest the caret); `afterCursor` keeps its START (the text nearest the caret). |
There was a problem hiding this comment.
Truncation names two cases (beforeCursor, afterCursor) that don't exist in the enum — the actual cases are preserveStart and preserveEnd. A reader following the comment to understand which case to use will be confused, and any future tooling that cross-references doc names will silently mis-match.
| /// Which end of the content to keep when it must be shortened. `beforeCursor` keeps its END | |
| /// (the text nearest the caret); `afterCursor` keeps its START (the text nearest the caret). | |
| /// Which end of the content to keep when it must be shortened. `preserveEnd` keeps the END | |
| /// of the string (the text nearest the caret); `preserveStart` keeps the HEAD. |
| if cap < section.minChars { | ||
| continue | ||
| } | ||
| let truncated = truncate(section.content, toChars: cap, mode: section.truncation) | ||
| .trimmingCharacters(in: .whitespacesAndNewlines) | ||
| guard !truncated.isEmpty else { | ||
| continue | ||
| } | ||
| var copy = section | ||
| copy.content = truncated | ||
| kept[index] = copy | ||
| remaining -= truncated.count |
There was a problem hiding this comment.
minChars guard checked against pre-trim cap, not post-trim length
The contract in the doc comment says a section is dropped when "the remaining budget can't fit at least this many characters," but cap is computed from the raw (untrimmed) content. After truncate + trimmingCharacters, truncated.count can be smaller than cap — so a section that passes cap >= minChars may end up included with fewer real characters than the minimum. In practice only the prefix uses minChars: 1 and its content is already trailing-trimmed, but if context sections are ever given a non-zero minChars, this mismatch silently violates the contract.
| sections.append( | ||
| PromptSection( | ||
| name: "prefix", | ||
| content: trimmedPrefix, | ||
| priority: 100, | ||
| minChars: 1, | ||
| maxChars: max(1, trimmedPrefix.count), | ||
| truncation: .preserveEnd | ||
| ) | ||
| ) | ||
|
|
||
| // Trailing whitespace is trimmed so the model continues from a clean word boundary instead of | ||
| // being asked to emit a leading-space token (which base models do poorly). A prefix ending | ||
| // mid-word keeps its final partial word, so mid-word continuation still works. Output spacing | ||
| // is reconciled downstream by `SuggestionTextNormalizer`. | ||
| let trimmedPrefix = Self.trimmingTrailingWhitespace(prefixText) | ||
| let kept = PromptSectionBudget.allocate(sections, totalChars: contextBudget) | ||
| let prefix = kept.first { $0.name == "prefix" }?.content ?? trimmedPrefix |
There was a problem hiding this comment.
When the total budget is smaller than trimmedPrefix.count, the allocator calls truncate(trimmedPrefix, toChars: cap, mode: .preserveEnd) and then trimmingCharacters(in: .whitespacesAndNewlines) on the result. A suffix of the prefix can start with a space or newline (e.g. "hello world".suffix(7) = " world" → trimmed to "world"), so the model receives a prompt that ends mid-word-boundary, defeating the invariant described in the doc comment. The original code always passed the full trimmedPrefix. This only fires when contextBudget is smaller than the prefix length (unlikely with the 2400-char default and pre-windowed input), but as a code path it silently breaks the caret-continuation contract.
1 participant
Replaces the flat-prose base prompt with a character-budgeted sectioned builder (
PromptSectionBudget). Each section (persona, style, language, notes, clipboard, screen, caret prefix) declares a priority + min/max budget; the allocator fills highest-priority-first within a total budget, so a large glossary/clipboard/screen capture can't starve the caret text (top priority, guaranteed minimum, kept by its end). Also fixes the custom-rules framing bug: rules render as their ownWriting style: ...line instead ofin a <rules> style(which broke for sentence-shaped rules). Validation: build-for-testing BUILD SUCCEEDED, swiftlint clean, 9 new PromptSectionBudget tests + updated renderer test. Foundation for the OCR + writing-history context sections.Greptile Summary
Replaces the flat-prose base-prompt builder with a character-budgeted section allocator (
PromptSectionBudget) and fixes the custom-rules conditioning line so sentence-shaped rules render correctly. Sections (persona, style, language, notes, clipboard, screen, prefix) each carry a priority and min/max char budget;allocatefills highest-priority-first so the caret prefix is never starved.PromptSectionBudget.swiftis a new pure-function allocator; well-tested across 9 cases covering priority fill, budget cap, render-order preservation, minChars drop, and truncation modes.BaseCompletionPromptRenderer.swiftwires the allocator in and fixes the\"in a X style\"framing bug by rendering custom rules as a standalone\"Writing style: \u2026\"line.minCharsguard fires against the pre-trim character count rather than the post-trim length.Confidence Score: 4/5
Safe to merge; the new allocator is well-isolated, well-tested, and the default budget makes the identified edge cases practically unreachable in normal use.
The refactor is clean and the core priority-fill logic is correct. The two findings are both edge cases: a stale doc comment and a whitespace-trimming quirk on the prefix that only fires when the total budget is smaller than the already-windowed prefix.
Both PromptSectionBudget.swift and BaseCompletionPromptRenderer.swift carry minor issues worth a second look before extending the allocator with new sections.
Important Files Changed
minCharsbeing checked against the pre-whitespace-trim cap rather than the final character count.trimmingCharacterson the truncated prefix end could strip a meaningful leading space when the budget is tighter than the prefix length.Sequence Diagram
sequenceDiagram participant Caller participant Renderer as BaseCompletionPromptRenderer participant Budget as PromptSectionBudget Caller->>Renderer: prompt(prefixText, userName, customRules, ...) Renderer->>Renderer: trimmingTrailingWhitespace(prefixText) Renderer->>Renderer: Build PromptSection list Renderer->>Budget: allocate(sections, totalChars: 2400) Budget->>Budget: Sort by priority descending loop each section in priority order Budget->>Budget: "cap = min(maxChars, content.count, remaining)" alt "cap < minChars" Budget->>Budget: drop section else Budget->>Budget: truncate then trim then guard non-empty Budget->>Budget: "remaining -= truncated.count" end end Budget-->>Renderer: kept sections in original order Renderer->>Renderer: Extract prefix and preface alt preface is empty Renderer-->>Caller: prefix only else Renderer-->>Caller: preface joined + blank line + prefix endReviews (1): Last reviewed commit: "Budget the base prompt by section so con..." | Re-trigger Greptile