fix(classifier): feature verb wins over debug keyword (part of #196)#289
Merged
fix(classifier): feature verb wins over debug keyword (part of #196)#289
Conversation
Messages like "add error handling", "create an issue tracker", or "implement the 404 page" were landing in the Debugging bucket because the classifier checked DEBUG_KEYWORDS (which matches `error`, `issue`, `404`) before FEATURE_KEYWORDS in both `refineByKeywords` (tool-bearing turns) and `classifyConversation` (chat-only turns). The position of the matched word in the sentence is a much stronger intent signal than the order of the checks in code, so we now pick whichever pattern matches earliest. The new helper `firstMatchingCategory` runs each candidate regex once with `RegExp.exec` and keeps the match with the lowest `index`. Ties (rare in practice — same start position) break by the order the candidates were listed, which is `refactoring > feature > debugging` for coding turns. That ordering preserves existing behavior for plain bug reports (e.g. "login is broken, traceback below") while flipping mislabeled feature work to its correct category. 8 regression tests in `tests/classifier.test.ts` cover the mislabel cases from #196 plus tie-break / chat-only cases. Full suite: 45 files / 609 tests, all green. Closes the activity-misattribution half of #196. The Cursor provider attribution half (single 'cursor' project for all sessions) is addressed in a separate PR.
This was referenced May 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of three PRs addressing the issues flagged in #196 and #159. Independent of the others; can land alone.
The bug
Messages like `add error handling`, `create an issue tracker`, or `implement the 404 page` were landing in the Debugging bucket because the classifier checked the debug regex before the feature regex. Real users from #196 saw their feature work mislabeled.
```ts
// Before: order of checks decided the result
if (DEBUG_KEYWORDS.test(userMessage)) return 'debugging' // wins on "add error handling"
if (REFACTOR_KEYWORDS.test(userMessage)) return 'refactoring'
if (FEATURE_KEYWORDS.test(userMessage)) return 'feature'
```
The fix
Pick whichever keyword pattern matches earliest in the user message. The position of the leading verb is a much stronger intent signal than the order of the checks in code.
```ts
// After: position decides
return firstMatchingCategory(userMessage, [
{ regex: REFACTOR_KEYWORDS, category: 'refactoring' },
{ regex: FEATURE_KEYWORDS, category: 'feature' },
{ regex: DEBUG_KEYWORDS, category: 'debugging' },
]) ?? 'coding'
```
Tie-break (same start index) goes to the candidate listed first. Ordering is `refactoring > feature > debugging` so existing behavior for plain bug reports (`login is broken, traceback below`) is preserved — DEBUG word leads in those cases.
Same logic applied to `classifyConversation` for chat-only turns.
Examples
Tests
8 new regression tests under `describe('classifyTurn — feature vs debugging precedence (#196)')`. Full suite: 45 files / 609 tests, all pass.
Out of scope