Repository navigation
feat(common-utils): share the Event Patterns drill-down predicate - #3662
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 0f71583 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Duplicate code analysisNo new duplicate code introduced by this PR. jscpd compared this branch against |
🔵 Tier 2 — Low RiskSmall, isolated change with no API route or data model modifications. Why this tier:
Review process: AI review + quick human skim (target: 5–15 min). Reviewer validates AI assessment and checks for domain-specific concerns. Stats
|
PR Review
2 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 2 minor 1 posted as inline comment(s) on the changed lines. 1 listed below. Findings outside the changed lines1 minor
Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
Deep ReviewScope: ✅ No critical issues found. The regex construction, the 🟡 P2 -- recommended
🔵 P3 nitpicks (2)
Reviewers (8): correctness, adversarial, security, testing, api-contract, maintainability, project-standards, previous-comments. Testing gaps: Unit tests validate generated regexes only through the No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
E2E Test Results✅ All tests passed • 470 passed • 1 skipped • 1045s
Tests ran across 4 shards in parallel. |
|
Review findings outside the inline threads, fixed in 04c8a76:
|
|
Second-round findings outside the inline threads:
That's the second round of automated findings, so per AGENTS.md I'm leaving any further ones for review. |
| '^[\\s\\x{0B}\\pZ\\x{FEFF}[:punct:]]*a[\\s\\x{0B}\\pZ\\x{FEFF}[:punct:]]*$', | ||
| ); | ||
| const partial = { extraDelimiters: ['/', ':', '-', ']', '\\', '^'] }; | ||
| const regex = toJsRegex(templateToRegex('a <*> c d e f g', partial)); |
There was a problem hiding this comment.
🔵 minor — New RE2 syntax is never run by RE2 or ClickHouse; toJsRegex rewrites it before testing
Every assertion runs a JS translation of the regex. The translation replaces [:punct:] with literal ranges, \x{..} with \u{..} and \pZ with \p{Z}. So a ClickHouse/RE2 problem with the new constructs would still pass: [:punct:] nested inside a larger bracket class, the \x{1C}-\x{1F} range, or \-/\[/\^ escapes in a class. Add a ClickHouse integration test (the package already has *.int.test.ts suites, e.g. src/__tests__/queryChartConfig.int.test.ts). It should run SELECT match(...) on a few bodies, with and without punctuation and Python whitespace.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
0361b05 to
94697eb
Compare
|
Rebuilt this as a GitHub stack with one commit per layer; the content is unchanged, and the force-push is in the timeline. The "Fixed in" commits in the replies above are from before the rebuild. |
94697eb to
331bc7c
Compare
templateToRegex and buildPatternPredicate build the whereSnippet that the MCP clickstack_event_patterns tool returns for each pattern. They live in the MCP server today. This adds them to @hyperdx/common-utils/dist/drain so the app can build the same filter. The app mines patterns with Python Drain3 and all ASCII punctuation as extra delimiters, so the options describe the miner: extraDelimiters (one character each; a longer one throws, since a character class can't split on it the way Drain does) and whitespace: 'python' for Drain3's str.split(), which also splits on \x1C-\x1F and NEL but keeps a BOM inside a token. When every ASCII punctuation character is a delimiter, the class uses RE2's [:punct:]; ASCII_PUNCTUATION is exported for the app. parsePatternPredicate reads a predicate back into its value expression and template, and only accepts predicates these functions built. Without options the output is byte-identical to the MCP copy, which the next PR in the stack deletes. Part of FR-385. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
331bc7c to
0f71583
Compare
Adds the Event Patterns drill-down predicate that the MCP server builds for
clickstack_event_patternsto@hyperdx/common-utils/dist/drain, so the app can build the same filter. Part of FR-385: filter for or exclude a pattern in one click from Event Patterns.Stack
Layer 1 of 4 in a GitHub stack (stack #3682). GitHub shows each layer's base as the branch below it, but the stack's base is
main: merging any PR lands it and every PR below it onmain, one squashed commit each. Each PR's diff is only its own layer, and CI runs as if each targetsmain:Summary
The app mines patterns in the browser with Drain3 and all ASCII punctuation as extra delimiters. MCP mines with the TypeScript port and splits on whitespace only, so its regex does not fit browser templates: a browser template looks like
GET api v1 users <*> <*> <*>for events likeGET /api/v1/users/123 200 15ms.templateToRegexandbuildPatternPredicatetake an optionalextraDelimiters. Each delimiter joins the separator class, since Drain turns each one into a space before splitting. A delimiter longer than one character throws, because Drain splits on it as a whole string and a character class can't.whitespace: 'python'matches templates from Python Drain3, whosestr.split()also splits on\x1cto\x1fand NEL but not on BOM; JS\sis the other way round.[:punct:]instead of repeating 32 characters between every token.ASCII_PUNCTUATIONis exported so the app builds its delimiter list from it.parsePatternPredicatereads a predicate back into its value expression and template. It only accepts what these functions built, so a hand-writtenmatch()is not taken for a pattern. The app uses it to show pattern filters as pills (feat(app): filter for or exclude a pattern from Event Patterns #3665).MAX_PREDICATE_TEMPLATE_LENGTHcomes along unchanged.Without
extraDelimitersthe output is byte-identical to the MCP copy, which #3663 deletes.#3551 also edits
templateToRegexin the MCP copy (date masks). Whichever of the two lands second carries that change intopackages/common-utils/src/drain/pattern-predicate.ts.Test plan
pattern-predicate.test.tspins the MCP output format, checks punctuation delimiters (including],\,^and-, which are special inside a character class), Python whitespace, multi-character delimiters and the length limit, checks that each message mined with the same delimiters matches its own cluster's template and no other, and round-trips predicates throughparsePatternPredicatewith and without options (and rejects them under the other options)yarn ci:lintandyarn ci:unitinpackages/common-utils(47 suites, 3,158 tests)yarn dupes(no new duplicate code)