Repository navigation
feat: add data source and connection variable types to dashboards - #3651
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 825d764 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Duplicate code analysis3 new clones introduced by this PR.
If one of these duplicates a helper that already exists, import it instead jscpd compared this branch against |
E2E Test Results✅ All tests passed • 472 passed • 1 skipped • 996s
Tests ran across 4 shards in parallel. |
c74c1aa to
088aa3a
Compare
🔴 Tier 4 — CriticalTouches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI. Why this tier:
Additional context: touches background tasks or the delivery pipeline lightly (11 lines, under the 30-line bar for Tier 4) Review process: Deep review from a domain expert. Synchronous walkthrough may be required. Stats
|
PR Review
5 finding(s): 🔴 0 critical · 🟠 1 major · 🔵 4 minor 4 posted as inline comment(s) on the changed lines. 1 listed below. Findings outside the changed lines1 minor
Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
Deep Review✅ No critical issues found. This is a well-constructed, defensively-written, and unusually well-tested PR. Reference tokens are matched against a strict anchored pattern ( 🟡 P2 — recommended
🔵 P3 nitpicks (7)
Reviewers (11): correctness, adversarial, api-contract, kieran-typescript, reliability, julik-frontend-races, security, testing, maintainability, project-standards, previous-comments. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
0485234 to
95c555e
Compare
…accept variables with some usable kinds
95c555e to
d638aa1
Compare
…e the filter can't use
|
On the P0 - this is intentional, external API / MCP support is in PR 2 in the stack |
…and ignore unusable selections in filter forms
…d data source variables
| ]), | ||
| ); | ||
| const connectionVariableNames = new Set(getConnectionVariableNames(filters)); | ||
| const existingKeys = new Set( |
There was a problem hiding this comment.
Should a PATCH block deleting a variable that tiles still use?
At the moment a PATCH skips every reference the stored dashboard already had. So you can delete a data source variable while tiles still point at it, and those tiles break.
Could it skip only the references that were already broken before the update? Edits to unrelated filters would still save, but deleting a variable that's in use would return a 400. We should probably return a validation message claiming that it's still in use too.
# Conflicts: # packages/app/src/components/SourceSelect.tsx
| const [id] = selectionByFilterId.get(filterId)?.included ?? []; | ||
| return id == null ? undefined : String(id); | ||
| }; | ||
| const connectionValues = new Map( |
There was a problem hiding this comment.
🔵 minor — A stale connection variable selection is still used, while a stale source variable selection is dropped
Source variable selections are dropped when the variable no longer offers them, but connection selections are not checked. If the URL or saved value holds a deleted or unknown connection ID, ConnectionDashboardFilter shows an empty picker with the required caution. Meanwhile referenceVariableValues still resolves $conn to that ID, unsatisfiedRequiredFilters doesn't block, and raw SQL tiles query a connection that doesn't exist. Pass the team's connections in as well, and call selectionByFilterId.delete for connection variables whose ID isn't among them, the same way the source loop does.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| { | ||
| key: 'source', | ||
| target: 'source', | ||
| acceptedKinds: isPromql ? [SourceKind.Promql] : NON_PROMQL_SOURCE_KINDS, |
There was a problem hiding this comment.
🔵 minor — Server-side reference validation accepts every non-PromQL kind for builder tiles, regardless of display type
getConfigReferenceFields gives every non-PromQL tile NON_PROMQL_SOURCE_KINDS. So the API and provisioning accept a search or event_patterns tile on a metric-only variable, or a heatmap on a log-only variable. Those tiles can never run: at runtime getSavedConfigAllowedSourceKinds (guards.ts) rejects every selection, and the editor picker already uses the precise kinds. Use getSavedConfigAllowedSourceKinds(config) for the tile source field so the API matches the runtime and the UI.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| const keys = getFilterReferenceFields(filter).map(({ key }) => key); | ||
| return { | ||
| filter: resolved, | ||
| unresolvedVariables: getReferencedVariables(filter, keys).filter( |
There was a problem hiding this comment.
🔵 minor — An unselected variable in only appliesToSourceIds stops the filter's own dropdown from loading
unresolvedVariables counts references in every field, appliesToSourceIds included. Then DashboardFilters drops any filter listed in unresolvedReferenceVariablesByFilterId from queryableFilters. A QUERY_EXPRESSION filter with a fixed source and appliesToSourceIds: ['$logs'] therefore loads no values and shows as pending until $logs is picked, even though it has a source to query. Only count references in the fields the values query reads (source/sources/connection) as unresolved for queryableFilters.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| if (!savedFilterValues?.length) return []; | ||
| const declaredNames = new Set( | ||
| getDashboardVariableDeclarations(filters).map( | ||
| const declaredNames = new Set([ |
There was a problem hiding this comment.
🔵 minor — Reference variable names are collected by hand instead of with the existing helper
pairWithVariableNames(filters.filter(f => isReferenceVariableFilterType(f.type))) does the same thing as getReferenceVariableFilters (common-utils/src/referenceVariables.ts), which already accepts external filters. The sibling variables.ts wraps that helper as getReferenceVariableNames too. Use getReferenceVariableFilters(filters).map(({ name }) => name) here, or share the variables.ts helper.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| }; | ||
|
|
||
| /** The fields of a chart config that can hold a reference. */ | ||
| export function getConfigReferenceFields( |
There was a problem hiding this comment.
🟠 major — Save-time check accepts source variables a tile's chart type can never use
getConfigReferenceFields gives every non-PromQL tile NON_PROMQL_SOURCE_KINDS, so validateDashboardVariableReferences accepts e.g. a Search/Event patterns tile on a [metric]-only variable, or a heatmap on kinds getHeatmapSourceKinds excludes. Every selection then shows the 'can't be used with this chart type' placeholder in DBDashboardPage (and MCP getResolvedSourceError rejects it). The editor picker already refuses these, because useWithSourceVariableItems filters by getAllowedSourceKinds. Fix: have getConfigReferenceFields take its tile acceptedKinds from getSavedConfigAllowedSourceKinds(config) (guards.ts), which already exists in common-utils.
Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond
| @@ -101,7 +107,10 @@ async function validateFilterConnections( | |||
| filters | |||
| .filter(isSqlQueryFilter) | |||
There was a problem hiding this comment.
🔵 minor — Internal API never checks a DATA_SOURCE filter's connection ID belongs to the team
validateFilterConnections only collects isSqlQueryFilter connections, so the internal POST/PATCH stores any string as a DATA_SOURCE connection, including an unknown ID or another team's ID. The external API rejects these through getMissingConnections ('connectionId' in filter). Fix: also collect non-reference connection values from isDataSourceFilter filters, skipping unchanged ones the same way.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| .filter(name => !byName.has(name)); | ||
| .filter(name => !byName.has(name) && !referenceVariableNames.has(name)); | ||
| if (unknownVariableValues.length > 0) { | ||
| const available = |
There was a problem hiding this comment.
🔵 minor — MCP unknown-variable error leaves out the data source and connection variables it accepts
available is built from variables, which comes from getDashboardVariableDeclarations and now excludes reference variables. If an agent mistypes $logs, the error lists only the SQL variables, so it never learns the valid source or connection variable names. Separately, the single-select check skips reference variables, so passing several values for one is accepted silently unless the queried tile reads it. Fix: include referenceVariableNames in the listed names, and apply the values.length > 1 check to reference variables too.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| details: ( | ||
| <Stack gap={0}> | ||
| {referenceIssues.map(issue => ( | ||
| <Text key={issue} c="red"> |
There was a problem hiding this comment.
🔵 minor — New import error list uses raw c="red" instead of the semantic Text variant
The repository conventions ('Semantic component variants') say to use <Text variant="danger"> for status/validation text rather than raw palette colors like c="red". Fix: render each reference issue with <Text variant="danger">.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Summary
Adds data source and data connection type variables.
Source and connection variables allow for dashboards variable inputs that define a source or connection queried by a tile or other filters. Selected values are stored in source or connection id slots in the saved chart config, in the
$varNameformat, then converted to source or connection IDs at runtime when there is a selection.Variables are substituted prior to rendering the dashboard, at the top level, so that components and configs down the stack can just use source and connection IDs like they have been.
Direct
$sourcereferences are not yet allowed in SQL tiles - the source is selected at the chart level and $__sourceTable can be used to reference it. Connection variables are mostly only useful for (a) raw sql tiles and (b) source variables that can be scoped to a single connection, dynamically.How to test on Vercel preview
The E2E feature can be tested on PR 5: #3656
References