Skip to content

feat: ad hoc filter finishing touches - #3658

Merged
knudtty merged 6 commits into
mainfrom
aaron/adhoc-follow-up
Nov 17, 2026
Merged

knudtty merged 6 commits into
mainfrom
aaron/adhoc-follow-up

Conversation

@knudtty

@knudtty knudtty commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Finishing touches for ad hoc dashboard filters, which are still behind NEXT_PUBLIC_ENABLE_ADHOC_FILTERS.

  • The name is an identifier, not a variable. You can't reference an ad hoc filter as $name, but its name still has to be unique because it keys the filter's conditions in the filters= URL param. The edit form now says so. The filter-bar tooltip and the filters list no longer present it as a variable. getDashboardVariableFilters skips ad hoc filters, so checkAlerts, MCP validation and the alert chart stop declaring them. isVariableEnabled: true stays, because name-uniqueness validation reads it.
  • Broadcast is always on. With no variable, broadcasting is the only way the conditions apply. The schema now requires isBroadcastEnabled: z.literal(true), the same way STATIC_LIST and PROMETHEUS_LABEL use z.literal(false). The edit form drops the broadcast checkbox and always shows "Applies to sources".
  • Overlap warning. The edit form shows a warning when another ad hoc filter of the same source type already applies to any of the same sources. A click-to-filter on such a tile updates every one of those filters, which is confusing. It is a warning only; saving still works.
  • Collapsed chips. The filter input shows the first 3 conditions plus a "+N more" toggle that expands in place. Series with many labels, which is common in PromQL, made the input grow very tall.
  • IN operator. Each condition still holds one key and one value. When rendered, IN conditions sharing a key are OR'd: SQL renders toString(key) IN ('a', 'b') and PromQL renders key=~"a|b", with regex characters escaped. It is one stored value for both source types; the PromQL condition editor labels it =|, as Grafana does. A condition the filter's source type can't express (LIKE on PromQL) is skipped and its chip is marked invalid.

Compatibility: an ad hoc filter already stored with isBroadcastEnabled: false, or without the field, will no longer parse. The feature is behind a flag, so there is no migration.

Screenshots or video

I couldn't capture screenshots from my environment. @aaronknudtson, please add before/after shots of the ad hoc filter edit modal and of a filter input holding more than 3 conditions.

Before After

How to reproduce

Starting point: yarn dev. packages/app/.env.development sets NEXT_PUBLIC_ENABLE_ADHOC_FILTERS=true.

On main:

  1. Open a dashboard, click "Filters", then "Add filter", and choose the type "Ad hoc keys and values".
  2. The form has a "Broadcast filter conditions" checkbox, and the "Variable name" tooltip says it's how the variable is referenced.
  3. Save a filter over the Logs source, then add a second one over Logs. No warning appears.
  4. Add 6 conditions to one filter. The input wraps onto several rows.
  5. Open the operator select in the condition editor. There is no in option.

On this branch:

  1. In the same form there is no broadcast checkbox, and "Applies to sources" is always visible. Under "Variable name" it reads "Unique name for this filter. Ad hoc filters can't be referenced as variables in tile queries."
  2. Adding a second ad hoc filter over Logs shows "Another ad hoc filter (…) already applies to these sources."
  3. With 6 conditions, the input shows 3 chips and "+3 more". Clicking it shows every chip and "Show less".
  4. Add ServiceName in api and ServiceName in web. Tiles on Logs filter with toString(ServiceName) IN ('api', 'web'). On a Prometheus ad hoc filter, =| (stored as IN) conditions on the same label render as label=~"(a|b)".
  5. Hovering the filter label shows "Filters N sources", without "available as variable".

How to test on Vercel preview

N/A. The preview builds without NEXT_PUBLIC_ENABLE_ADHOC_FILTERS, so ad hoc filters aren't offered there.

Preview routes: N/A

Steps:

  1. N/A

References

  • Linear Issue: Closes HDX-5761

@knudtty
knudtty requested a review from a team as a code owner November 13, 2026 21:17
@changeset-bot

changeset-bot Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 0f16331

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Patch
@hyperdx/app Patch
@hyperdx/api Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Nov 17, 2026 5:04pm UTC
hyperdx-storybook Ready Ready Preview Nov 17, 2026 5:04pm UTC

Request Review

@github-actions github-actions Bot added the review/tier-3 Standard — full human review required label Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

  • Diff size: 357 production lines changed (Tier 2 max: < 250)
  • Cross-layer change: touches frontend (packages/app) + shared utils (packages/common-utils)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats
  • Production files changed: 9
  • Production lines changed: 357 (+ 347 in test files, excluded from tier calculation)
  • Branch: aaron/adhoc-follow-up
  • Author: knudtty

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 108 in total (1.18%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

2 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 2 minor

2 posted as inline comment(s) on the changed lines.


Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found. This flag-gated change (NEXT_PUBLIC_ENABLE_ADHOC_FILTERS) is well-tested and the core rendering logic is sound: SQL values route through escapeSqlString, PromQL values are regex-escaped then string-escaped (the double-escape is correct), and the collapsed-chip slice(0, …) keeps remove/replace indices aligned with the originals. The items below are recommended and discretionary.

🟡 P2 -- recommended

  • packages/common-utils/src/types.ts:2467 -- isBroadcastEnabled: z.literal(true) makes any previously-stored ad hoc filter with isBroadcastEnabled: false or the field absent fail to parse, and because DashboardSchema.filters is a z.array(DashboardFilterSchema) discriminated union, one bad ad hoc filter rejects the entire dashboard document on load, not just that filter.
    • Fix: Confirm no persisted ad hoc filters exist in any environment where the flag was enabled before merge, or parse filters individually so a single invalid filter degrades gracefully instead of failing the whole dashboard load.
🔵 P3 nitpicks (2)
  • packages/app/src/components/AdhocDashboardFilter/AdhocDashboardFilter.tsx:171 -- key={JSON.stringify(condition)} collides when two conditions share the same key, operator, and value (now more reachable since IN encourages multiple same-key rows), which can misattribute remove/replace during React reconciliation.
    • Fix: Incorporate the condition's index into the React key so duplicate conditions stay distinct.
  • packages/common-utils/src/types.ts:1351 -- A prior review thread raised consolidating IN and =| into one shared operator (as =~ is shared) and noted the nearby comment risks drifting; the diff keeps two separate stored operators, leaving that suggestion acknowledged but deferred.
    • Fix: Either file a follow-up to unify the operators or confirm the two-operator model is intentional and keep the explanatory comment in sync.

Reviewers (11): correctness, adversarial, security, testing, maintainability, project-standards, api-contract, kieran-typescript, previous-comments, agent-native, learnings-researcher.

Testing gaps: Consider an explicit test that loading a dashboard whose stored ad hoc filter lacks isBroadcastEnabled behaves as intended (whole-dashboard reject vs. graceful skip).

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 465 passed • 1 skipped • 1292s

Status Count
✅ Passed 465
❌ Failed 0
⚠️ Flaky 3
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

Comment thread packages/app/src/DashboardFilters.tsx Resolved
Comment thread packages/common-utils/src/filters.ts Resolved
Comment thread packages/common-utils/src/types.ts Resolved
Comment on lines 1326 to 1330
/**
* Operators an ad hoc filter condition can use, spelled as in PromQL so one set
* serves both source types. See `SQL_ADHOC_FILTER_OPERATORS` and
* `PROMQL_ADHOC_FILTER_OPERATORS` for which ones each source type supports.
*/

@pulpdrew pulpdrew Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment here is a little out of date now, or maybe =| and IN should be a single shared operator, like regex is?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was thinking about just making a single operator. I think that's a good idea

pulpdrew
pulpdrew previously approved these changes Nov 16, 2026

@pulpdrew pulpdrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, with a minor suggestion for consistency

- Add IN (SQL) and =| (PromQL) operators; conditions on the same key are OR'd
  into one IN (...) or =~"a|b" condition.
- Ad hoc filters always broadcast (isBroadcastEnabled: z.literal(true)), and
  their variable name is a unique identifier only, no longer declared as a
  $variable.
- Warn when another ad hoc filter already applies to the same sources.
- Collapse the filter input to three conditions behind a "+N more" toggle.
One predicate decides whether a filter's name is a referenceable variable,
used by getDashboardVariableFilters, getFilterEffect, and the filters list.
Updates the ad hoc filter e2e specs for the removed broadcast checkbox and
the list no longer showing an ad hoc filter's name as a variable.
@knudtty
knudtty force-pushed the aaron/adhoc-follow-up branch from 7070a05 to 0f16331 Compare November 17, 2026 17:01
@knudtty
knudtty merged commit f15b849 into main Nov 17, 2026
28 of 29 checks passed

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 0f163317 Deployed Nov 17, 2026 by vercel[bot]
Preview – hyperdx-storybook — 0f163317 Deployed Nov 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-3 Standard — full human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants