Skip to content

feat(app): add a duration heatmap view to trace search - #3184

Open
alex-fedotyev wants to merge 15 commits into
alex/red-metrics-trace-search-v2from
alex/trace-search-duration-heatmap
Open

alex-fedotyev wants to merge 15 commits into
alex/red-metrics-trace-search-v2from
alex/trace-search-duration-heatmap

Conversation

@alex-fedotyev

@alex-fedotyev alex-fedotyev commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Stack

Layer 2 of 2 in a GitHub stack (stack #3688). GitHub shows this PR's base as alex/red-metrics-trace-search-v2, the branch below it, but the stack's base is main: merging this PR lands it and #3183 on main, one squashed commit each. Its diff is only the heatmap layer, and CI runs as if it targets main:

  1. feat(app): show RED metrics on the trace search results view #3183 RED metrics on the trace search results view
  2. feat(app): add a duration heatmap view to trace search #3184 Duration heatmap view for trace search (this PR)

Summary

Adds a RED/Heatmap switch to the trace search results view. Heatmap replaces the three RED tiles with the duration distribution over time, as the same bare heatmap tile the dashboard renders (DBHeatmapChart with a toHeatmapQuery query), without the significant-fields comparison panel.

  • The choice lives in the URL as traceChartMode, like the other view toggles on this page, so a reload, a shared link, or Back/Forward reopens the same view. Picking a different source in the dropdown resets it to RED, so returning to a trace source after viewing another one doesn't silently reopen in Heatmap and switch live tail off again. The reset waits until the URL points at the new source, like the page's filter and group-by reconciles, so it lands on the new history entry, and it clears the param rather than writing red into every URL. It lives in useTraceChartModeReset.
  • The heatmap plots getDurationMsExpression (the source's duration scaled to ms by its durationPrecision) with the existing HEATMAP_DURATION_NUMBER_FORMAT, the same pairing the delta-view heatmap uses. Value, count and scale are fixed rather than read from that view's settings drawer, so this stays a plain duration heatmap independent of the delta view's query state.
  • DBHeatmapChart is used as is. The heatmap needs no query namespace of its own: live tail is off whenever it shows, so no tick can stack its queries.
  • Live tail stays off while the heatmap is the visible chart, the way delta and pattern modes keep it off: the TimePicker's Live option and the Resume Live Tail button are hidden, and a guard turns live tail back off if anything else turns it on, so a "Live tail ranges" pick lands on a fixed range. A tick moves the absolute time range, and the heatmap keys its uPlot instance on the chart config, so each refresh would otherwise tear the chart down and rebuild it with a recomputed y-axis. The guard lives in useHeatmapLiveTailGuard.
  • The switch only appears where the RED tiles render: a trace source with a duration column, a non-compact viewport, and no custom group-by (those fall back to the histogram). The chart branch, the switch and the live-tail pin read one derived source, so they can't disagree about which chart is showing, and a stale ?traceChartMode=heatmap can't disable live tail on a search that shows the histogram.
  • While the heatmap shows, the group-by picker is hidden: the heatmap doesn't group, and a custom group-by would swap it for the histogram. The heatmap also ignores the results table's sort, so sorting a column doesn't rebuild it.

Screenshots

Light theme:

heatmap light

Dark theme:

heatmap dark

Test plan

  • useHeatmapLiveTailGuard tests cover the guard: off when the heatmap appears, off again when resumed under it, untouched on RED.
  • useTraceChartModeReset tests cover the source-change reset: it waits for the URL to reach the picked source, fires once per pick, ignores Back/Forward, drops an undone pick, and follows the latest pick.
  • TraceRedMetricsChart render tests: heatmap mode mounts the heatmap, not the tiles, with the duration-in-ms value, count(), a log scale and the duration number format; RED mode mounts the three tiles, and only Duration pins its colors.
  • tsc --noEmit, eslint with no new warnings, knip, and the traceRedMetrics, DBTimeChart and DBSearchPage suites green against current main.
  • Verified on this PR's Vercel preview against the demo traces dataset, in light and dark: Heatmap renders the duration distribution and turns live tail off, with Resume Live Tail and the TimePicker's Live option hidden; a "Live tail ranges" pick under it lands on a fixed range that holds still across refresh intervals; switching back to RED offers Resume Live Tail again; the group-by picker hides under the heatmap and returns on RED; sorting a results column under the heatmap keeps the same chart instead of rebuilding it; the mode survives a reload; picking a Logs source hides the switch and clears traceChartMode, coming back to traces resets to RED, and Back from Logs returns to the heatmap (as do Forward and Back after it); a custom group-by and a compact (390px) viewport fall back to the histogram with the switch hidden. The heatmap shows Loading while its queries are in flight and "Not enough data points" for a WHERE that matches nothing, and a failed search query hides the chart area and shows the query error, with the Live option, Resume Live Tail and the group-by picker available again.

Notes

@vercel

vercel Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Nov 18, 2026 5:58am UTC
hyperdx-storybook Ready Ready Preview Nov 18, 2026 5:58am UTC

Request Review

@changeset-bot

changeset-bot Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 7e7a232

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/app Minor
@hyperdx/api Minor
@hyperdx/otel-collector Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@greptile-apps

greptile-apps Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness or repository-rule violations identified.

Summary

Adds a URL-backed RED/Heatmap switch to trace search and renders duration distributions using the shared dashboard heatmap implementation.

  • Resets the selected chart mode when the active source changes while preserving shared heatmap links on initial load.
  • Pauses live tail while the heatmap is visible and namespaces both heatmap queries with the search chart prefix.
  • Centralizes duration-heatmap formatting and separates span-status error predicates from log-severity handling.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Trace search results] --> B{traceChartMode}
    B -->|RED| C[Throughput, errors, duration]
    B -->|Heatmap| D[Duration heatmap]
    D --> E[Min/max query]
    E --> F[Bucket query]
    C --> G[Shared query-key prefix]
    D --> G
    G --> H[Search fetching gate]
    H --> I[Live-tail coordination]
Loading

Reviews (4) · Last reviewed commit: "fix(app): pin live tail off while the tr..."

Comment thread packages/app/src/DBSearchPage.tsx Outdated
// (undefined -> id) doesn't count as a change and a shared link that pins
// ?traceChartMode=heatmap survives.
const prevSourceIdRef = useRef<string | undefined>(undefined);
useEffect(() => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Reset effect discards the user's Heatmap choice even when the new source can render it

The effect fires on any searchedSource?.id change, so switching from trace source A to trace source B — both with a durationExpression, so both render the toggle via traceRedMetricsSource — silently flips the view back to RED. The sibling view toggle analysisMode (:1067) is deliberately not reset on source change even though delta needs a trace source too; that branch just renders nothing (:2637). Drop the effect (both the switch and the chart are already hidden when traceRedMetricsSource == null, so a lingering ?traceChartMode=heatmap is inert), or gate the reset on the new source not supporting the heatmap.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping a reset, but since 364e04c and 8c2175f it only runs for a source the user picks, not on every source id change. A lingering ?traceChartMode=heatmap isn't inert: coming back to a trace source would reopen the heatmap and switch live tail off again. Back/Forward keeps whatever mode the URL restores.

Comment thread packages/app/src/DBSearchPage.tsx Resolved
@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 P0/P1 in code this PR adds or changes: fix before asking for review.
  • 🟡 P2 in code this PR changes: your call. Fix if small, otherwise reply.
  • 🔵 P3 is collapsed at the bottom: nobody needs to act on it before merge.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    priority. Reply with one sentence; if it is P0, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

4 finding(s): 🔴 0 P0/P1 · 🟡 1 P2 · 🔵 3 P3

No inline comments. 2 P3 folded at the bottom. 2 unchanged from an earlier push (already inline above).

🔵 2 P3 — no action needed before merge
  • 🔵 P3 packages/app/src/DBSearchPage.tsx:2165 — traceHeatmapChartConfig duplicates the existing filtersChartConfig memo → filtersChartConfig (DBSearchPage.tsx:2255) already spreads chartConfig with orderBy: undefined, dateRange: searchedTimeRange and with: aliasWith, on the same dependencies. When chartConfig is set, the two produce the same config. Reuse filtersChartConfig for the heatmap, or factor out one shared memo, instead of keeping a second copy.
  • 🔵 P3 packages/app/src/DBSearchPage.tsx:2994 — The page wiring for the switch has no tests → The hook and component tests never render DBSearchPage. Nothing covers the onChange mapping that writes null for RED (the fix in 6cc9168), or isHeatmapChartVisible hiding the Live option, Resume Live Tail and the group-by picker. A regression there (for example, writing traceChartMode=red, or hiding Live when the histogram shows) would still pass. Extract the onChange mapping into a small tested helper, or add a DBSearchPage test that toggles the control and checks the URL state.

Priority comes from a separate grading pass over the finished review. P3 is folded here instead of posted inline. No finding blocks merge automatically; a maintainer decides.

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found.

This is a small, heavily-iterated frontend change (~228 production lines) that adds a URL-backed RED/Heatmap toggle to the trace search results view. The two extracted hooks are correct and unit-tested, the live-tail guard effect converges without a render loop, the mode-reset arm/disarm logic is sound, and the chart branch, switch, and live-tail pin all read one derived traceChartsSource so they cannot disagree. The large majority of the 50 prior review comments are genuinely resolved in the current diff.

🟡 P2 -- recommended

  • packages/app/src/DBSearchPage.tsx:2152 -- The page-level integration wiring for this feature is untested: the derived traceChartsSource / isHeatmapChartVisible conditions, the SegmentedControl onChange param-clearing, and the conditional hiding of the group-by picker, the TimePicker Live option, and Resume Live Tail have no unit coverage, while only the extracted hooks and TraceRedMetricsChart render branch are tested.
    • Fix: Add a DBSearchPage render test (or land the tracked e2e for this view) asserting the switch appears only under the duration/non-compact/no-group-by conditions and that selecting Heatmap hides those controls and pins live tail off.
🔵 P3 nitpicks (2)
  • packages/app/src/components/Search/TraceRedMetricsChart.tsx:110 -- getDurationMsExpression(source) and the heatmapQuery useMemo are computed on every render even in RED mode, where the heatmap is never rendered.
    • Fix: Optionally gate the heatmap-query construction behind mode === 'heatmap' to avoid building a query the RED branch discards.
  • packages/app/src/components/Search/useTraceChartModeReset.ts:30 -- arm(watchedSource ?? null) stores null when a picked source resolves to a nullish id, and the reset effect's pendingSourceIdRef.current != null guard then silently no-ops, so an armed reset would be dropped in that edge.
    • Fix: If a nullish source id is reachable on a real pick, track an explicit "armed" boolean separately from the pending id; otherwise document that arm is only called with a concrete id.

Reviewers (10): correctness, adversarial, frontend-races, kieran-typescript, testing, maintainability, project-standards, previous-comments, agent-native, learnings-researcher.

Testing gaps:

  • No unit coverage for the DBSearchPage wiring (derived visibility conditions, switch param-clearing, control hiding); author defers end-to-end coverage to issue E2E coverage for the trace search RED metrics view #3176.
  • The mode-reset hook is unit-tested in isolation, but its arm/disarm call sites inside the source-change effect in DBSearchPage.tsx are only verified on the preview, not in an automated test.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

Comment thread packages/app/src/DBSearchPage.tsx Resolved
Comment thread packages/app/src/DBSearchPage.tsx Resolved
Comment thread packages/app/src/DBSearchPage.tsx Resolved
Comment thread packages/app/src/DBSearchPage.tsx Resolved
Comment thread packages/app/src/DBSearchPage.tsx Outdated
// can't disable live tail for a search that shows the histogram.
const isHeatmapChartVisible =
analysisMode === 'results' &&
traceRedMetricsSource != null &&

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Switching back to RED leaves live tail off

The pin writes isLive=false into the URL, and nothing restores it. A user who looks at the heatmap for a moment and switches back to RED is left on a frozen time range with live tail off. Pausing live updates through useLiveUpdate's pause while the heatmap is visible, instead of overwriting isLive, would bring live tail back when the user switches to RED.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intended: delta and pattern modes turn live tail off the same way, and pausing instead would leave the TimePicker and the results count claiming live tail over a frozen range. Back on RED, Resume Live Tail brings it back in one click.

Comment thread packages/app/src/DBSearchPage.tsx Resolved
Comment thread packages/app/src/DBSearchPage.tsx Resolved
kodiakhq Bot pushed a commit that referenced this pull request Sep 25, 2026
…3221)

## Summary

Fixes the heatmap part of #3205: during a dashboard refresh, heatmap tiles now keep the current heatmap on screen and pulse, instead of replacing it with "Loading...".

- Both heatmap queries (the min/max bounds and the buckets) keep their previous result as placeholder data, the same way the time chart and the number, bar and pie tiles do.
- The bucket query waits for the refreshed bounds before it runs, so a refresh doesn't also query the new range bucketed with the previous range's min/max.
- The heatmap, and the "Not enough data points" message, pulse while either query is still showing previous data. `Heatmap` takes an optional `className` so the pulse goes on its existing container rather than a new wrapper, which would affect its size measurement.
- New tests in `DBHeatmapChart.render.test.tsx`: both queries keep their previous data, the tile pulses during a refresh and stops once fresh data loads, and the bucket query waits for fresh bounds. Three of the four fail without the change.

While checking the rest of the app for the same pattern, I found two more places: the Kubernetes dashboard's Pods, Nodes and Namespaces tables still switch to skeleton rows on every refresh, and the Services dashboard tiles (`DBListBarChart`, `DBHistogramChart`, the slowest events tile) keep their data but don't pulse. Happy to take those as follow-ups if you'd like.

### How to test on Vercel preview

**Preview routes:** /dashboards

**Steps:**

1. Create a dashboard with a Heatmap tile on the demo traces source (for example on `Duration`) and a Line time chart.
2. Choose a large time range so each query takes a moment.
3. Click "Refresh dashboard".
4. Verify the heatmap stays on screen and pulses while its queries run, like the line chart, and stops pulsing once the new heatmap appears.

### References

- Linear Issue: N/A
- Related PRs: #3211, #3212. #3184 also edits the `queryKey` of these two heatmap queries, so whichever lands second needs a small rebase.
- Closes part of #3205

Checks run locally: `jest` for the heatmap suites and `ChartPreviewPanel` (50 passed), `eslint` on changed files (0 errors, no new warnings), `prettier --check`, and `tsc --noEmit` for `packages/app`.
@alex-fedotyev
alex-fedotyev requested a review from a team as a code owner November 13, 2026 23:40
@alex-fedotyev
alex-fedotyev force-pushed the alex/trace-search-duration-heatmap branch from 46b3d9e to 5809452 Compare November 17, 2026 02:00
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 476 passed • 1 skipped • 1160s

Status Count
✅ Passed 476
❌ Failed 0
⚠️ Flaky 1
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

@github-actions github-actions Bot added review/tier-2 Low risk — AI review + quick human skim and removed review/tier-3 Standard — full human review required labels Nov 18, 2026
alex-fedotyev and others added 15 commits November 17, 2026 21:54
Adds a RED/Heatmap switch to the trace results chart area. Heatmap renders
the duration distribution over time as the same tile the dashboard does. The
mode is URL state, so a reload or a shared link reopens the same view, and it
resets to RED when the source changes.

The heatmap's bounds and bucket queries take the search's RED query-key
prefix (DBHeatmapChart gains the queryKeyPrefix prop DBTimeChart already
has; keys are unchanged for callers that don't pass one), so live tail waits
on them instead of stacking a fresh pair each tick. Live tail is also pinned
off while the heatmap is the visible chart: a tick moves the absolute time
range, and the heatmap keys its uPlot instance on the chart config, so each
refresh would otherwise tear the chart down and rebuild it.

The switch, the chart branch and the live-tail pin share one derived source,
so the heatmap renders (and pins live tail) only where the RED tiles would: a
trace source with a duration column, a non-compact viewport, and no custom
group-by.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The live-tail pin only ran when the heatmap first appeared, so Resume
Live Tail (or the TimePicker's Live option) turned live tail back on
while the heatmap was still showing, and every tick rebuilt the chart.

- Hide the TimePicker's Live option and the Resume Live Tail button while
  the heatmap is the visible chart, the way delta and pattern modes hide
  them, and make the guard check isLive so any other path that turns live
  tail on (a relative range pick) is switched off again.
- Move the source-change reset and the live-tail guard into
  useTraceChartModeGuards, with tests for first load, a real source
  switch, and resuming live tail under the heatmap.
- Drop the @source tag on the heatmap query in favor of a plain note,
  and document that queryKeyPrefix only applies to distribution heatmaps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nuqs 1.x writes an explicitly set default into the URL, so resetting the
trace chart mode with 'red' added ?traceChartMode=red on every source
switch, log sources included. Clear the param instead; the mode still
falls back to RED.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mode union lived in TraceRedMetricsChart and a hand-written copy in
the URL parser, so a mode added to the type alone would compile but parse
back to RED. Export TRACE_CHART_MODES, derive TraceChartMode from it, and
use it in the parser and the view switch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prefix let live tail wait on the heatmap queries, but live tail is
always off while the heatmap is the visible chart, so it never changed
anything. It did give these queries different keys from the identical
delta-view heatmap. Without it, DBHeatmapChart and useHeatmapData go
back to their current shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reset ran in an effect on the searched source id, so it also fired
when Back or Forward restored a URL with traceChartMode=heatmap, and
rewrote that history entry back to RED. Clear the mode in the page's
existing user-source-change branch instead, which catch-up (history
navigation, a source name resolving) already skips. The hook keeps only
the live-tail guard, renamed useHeatmapLiveTailGuard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clearing the mode from the dropdown branch ran before the debounced
search submit pushed the new source, so it rewrote the previous history
entry, and Back still landed on RED. Mark the reset as pending there, like
the filter and group-by reconciles, and clear the mode once the URL points
at the new source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The heatmap config carried the results table's orderBy. The heatmap's
own queries don't use it, but the chart keys its uPlot instance on the
config, so sorting a column tore the heatmap down and rebuilt it. Drop the
sort from the base config, as the histogram config does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The heatmap doesn't group, but the picker stayed in the stats row, and
picking a custom group-by swapped the heatmap and the RED/Heatmap switch
for the histogram with no hint why. Hide the picker while the heatmap is
the visible chart; it comes back on the RED view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Picking a source arms a reset that fires once the URL points at it. If
the user re-selected the searched source before that search ran, the
reset stayed armed for the abandoned source, so a later navigation to it
with traceChartMode=heatmap cleared the mode it meant to restore. Clear
it whenever the form catches up to the searched source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
isHeatmapChartVisible ignored the chart branch's other conditions, so
while the chart config was resolving or a query error hid the chart
area, it still kept live tail off and hid the Live option, Resume Live
Tail and the group-by picker. Derive it from the same conditions the
chart branch uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reset that clears traceChartMode after a source pick lived inline in
DBSearchPage as a ref and an effect, with no tests, and it needed
several follow-up fixes. Move it into useTraceChartModeReset, which the
page arms on a user pick and disarms when the pick is undone, and test
the cases those fixes were about: it fires only once the searched source
reaches the pick, once per pick, never for Back/Forward, not after an
undone pick, and for the latest pick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Condense the comments this PR adds to the reasons a reader needs, and
restore one comment it had reworded. No behavior change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The source-change reset already clears the param because nuqs 1.x
writes an explicitly set default into the URL, but choosing RED in the
switch still wrote traceChartMode=red into the URL and shared links.
Clear it there too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Render the component in each mode with the chart components stubbed and
the real query builder kept: heatmap mode mounts the heatmap, not the
tiles, with the duration-in-ms value, count(), a log scale and the
duration number format; RED mode mounts the three tiles, and only
Duration opts into its pinned colors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 7e7a232d Deployed Nov 18, 2026 by vercel[bot]
Preview – hyperdx-storybook — 7e7a232d Deployed Nov 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-2 Low risk — AI review + quick human skim

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants