Repository navigation
feat: create SQL query dashboard filters and fill their dropdowns - #3678
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: cffcab2 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
cf6ccc5 to
d411792
Compare
Duplicate code analysis1 new clone introduced by this PR.
If one of these duplicates a helper that already exists, import it instead jscpd compared this branch against |
E2E Test Results✅ All tests passed • 476 passed • 1 skipped • 1028s
Tests ran across 4 shards in parallel. |
🟡 Tier 3 — StandardIntroduces new logic, modifies core functionality, or touches areas with non-trivial risk. Why this tier:
Review process: Full human review — logic, architecture, edge cases. Stats
|
PR Review
2 finding(s): 🔴 0 P0/P1 · 🟡 2 P2 · 🔵 0 P3 2 posted as inline comment(s) on the changed lines. Priority comes from a separate grading pass over the finished review. P3 is folded here instead of posted inline. No finding blocks merge automatically; a maintainer decides. |
Deep Review✅ No critical issues found. No P0/P1 ship-blockers. The feature reuses the established filter-value hook pattern cleanly, prior review feedback is verified as addressed, variable interpolation reuses the existing 🟡 P2 — recommended
🔵 P3 nitpicks (5)
Reviewers (13): correctness, testing, maintainability, project-standards, agent-native, learnings-researcher, kieran-typescript, julik-frontend-races, performance, reliability, security, adversarial, previous-comments. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
|
Addressed the findings outside the changed lines in e9c4dca:
The jscpd clone with |
e9c4dca to
c5edc1a
Compare
| ], | ||
| // Keep the last values while a new range or selection loads, rather than | ||
| // blanking the dropdown. | ||
| placeholderData: keepPreviousData, |
There was a problem hiding this comment.
🔵 minor — keepPreviousData has no effect inside useQueries when the key changes, so the dropdown goes blank on every reload
In react-query v5, useQueries matches observers by query hash. When the expanded SQL or the time params change, the query gets a new observer that has no previous data, so keepPreviousData returns undefined and values drops to [] until the fetch finishes. This contradicts the comment on the line above it. Follow the sibling hooks (usePromqlLabelFilterValues.tsx:102, useQueriedDashboardFilterValues.tsx:373): build a key prefix of ['dashboard-filter-sql-query-values', call.connection], then in placeholderData return the most recently updated cached entry from queryClient.getQueriesData({ queryKey: prefix }). This filter's own variables can change the SQL, so the prefix probably also needs the filter id.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| // Only the params the SQL reads, so a query that ignores the time range | ||
| // keeps its cache key, and isn't re-run, when the range changes. | ||
| params: Object.fromEntries( | ||
| Object.entries(timeParams).filter(([name]) => sql.includes(`{${name}:`)), |
There was a problem hiding this comment.
🔵 minor — A raw {intervalSeconds:Int64} / {intervalMilliseconds:Int64} param passes validation but is never sent, so the query fails at runtime
expandSqlQueryFilterTemplate rejects the interval macros (line 888), but the params only include start/end. A template that writes {intervalSeconds:Int64} directly, as TIME_CHART_EXAMPLE_SQL in rawSqlParams.ts shows, passes validateSqlQueryFilterTemplate in the form, API and MCP. Every run then fails in ClickHouse with a missing-substitution error. Reject templates that reference {intervalSeconds: or {intervalMilliseconds: in expandSqlQueryFilterTemplate, using the same "have no time buckets" message, so saving fails up front.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
c5edc1a to
517403d
Compare
pulpdrew
left a comment
There was a problem hiding this comment.
Couple of thoughts, looks pretty good though and works well! One other thing that could be nice is a couple of E2E exercising this filter type
517403d to
01249ed
Compare
8e87995 to
0e60b43
Compare
pulpdrew
left a comment
There was a problem hiding this comment.
LGTM! Thanks for adding the new connection variable support!
Add a "SQL query values" type to the dashboard filter editor and populate the filter's dropdown from its query. - Render the filter's query in common-utils: expand time macros and other filters' variables, select the first column as a string, and cap the result at 10,000 rows - Run each SQL query filter's query on its connection and merge the values and errors into the dashboard's filter values - Keep the query's row order in the dropdown and show a caution while a referenced variable has no selection - Add the editor form: connection picker, SQL editor with macro and variable autocomplete, variable name, and an on-demand preview of the values over the dashboard's time range Refs HDX-5524
- Reject interval macros in SQL query filters, which have no time buckets - Validate the SQL on save against the dashboard's other variables - Build the editor's unsupported-macro list from the shared constants - Show the preview loader while a re-run fetches
0e60b43 to
cffcab2
Compare
| ], | ||
| // Keep the last values while a new range or selection loads, rather than | ||
| // blanking the dropdown. | ||
| placeholderData: keepPreviousData, |
There was a problem hiding this comment.
🟡 P2 — placeholderData: keepPreviousData does nothing inside useQueries, so the dropdown blanks on every range or selection change
In TanStack Query v5, useQueries creates a new QueryObserver whenever a query's key changes, so keepPreviousData never has previous data to keep. A filter whose SQL uses $__timeFilter, or references another filter's variable, drops to a loader on each change, contrary to the comment. Use the prefix-based placeholderData lookup (queryClient.getQueriesData({ queryKey: [SQL_QUERY_FILTER_VALUES_QUERY_KEY, call.connection] }), newest by dataUpdatedAt) that usePromqlLabelFilterValues.tsx and useQueriedDashboardFilterValues.tsx already use.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| previewed?.connection === resolvedConnection && | ||
| previewed.sqlTemplate === sqlTemplate | ||
| ) { | ||
| void queryClient.invalidateQueries({ |
There was a problem hiding this comment.
🟡 P2 — Preview re-run invalidates every SQL query filter on the connection, and the first preview can return cached dashboard results
invalidateQueries({ queryKey: [KEY, resolvedConnection] }) is a prefix match, so each repeat click also refetches every SQL query filter the dashboard behind the modal shows on that connection. When editing an existing filter, the first click shows results the dashboard cached in the last 5 minutes (staleTime) instead of running the query. Re-run only the preview's own query on every click: return refetch from the hook, or invalidate the exact key including the SQL and params. The test only checks that invalidateQueries was called with a mocked key, so also assert that the query runs again.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Summary
Adds the "SQL query values" dashboard filter type to the UI: users can create the filter, and its dropdown lists the first column of a ClickHouse query. This is the last PR in the stack; earlier PRs added the schema (#3672) and API/MCP support (#3674).
renderSqlQueryFilterValuesQueryin common-utils expands time macros and other filters' variables, wraps the SQL asSELECT toString(tupleElement(tuple(*), 1)) AS value FROM (...) LIMIT 10000, and strips a trailing;. Time params are sent only when the SQL uses them.useSqlQueryFilterValuesruns each filter's query on its connection withmax_execution_time: 30. Nulls, empty strings and duplicates are dropped, and the query's order is kept, so the dropdown doesn't sort these values.useDashboardFilterValuesmerges these values and errors with the other filter types. A query error shows as the dropdown's caution, as do variables the query references that have no selection yet.$__sourceTableand$__filtersare left out, since the filter has no source), and the variable name. The filter is variable-only, so there are no broadcast or variable toggles.Screenshots or video
Untitled.mov
How to reproduce
On
main:On this branch:
Service, pick the demo connection, and enter:svc, click Save filter, and close the modal. The Service dropdown lists the same services in the query's order.Severitywith optionsINFOandERRORand variablesev. Change the SQL to addAND $__filter(SeverityText, $sev). Picking a severity narrows the Service dropdown.SELECT nope FROM default.otel_logsand preview. The preview shows the ClickHouse error, and after saving, the dropdown shows it as a caution.References