Skip to content

Timestamp-only SELECT could open the wrong event #3518

Closed

Description

@libbkmz

What happened?

HyperDX can display the wrong request when SELECT contains only timestamp .

Steps to reproduce

Only necessary fields for the bug are here:

CREATE TABLE default.hyperdx_row_repro
(
    timestamp DateTime('UTC'),
    event JSON(max_dynamic_paths=0, `@request_id` Nullable(String))
)
ENGINE = MergeTree
PARTITION BY toDate(timestamp)
ORDER BY timestamp;

INSERT INTO default.hyperdx_row_repro VALUES
('2026-11-02 12:53:33', '{"@request_id":"request-A"}'),
('2026-11-02 12:53:33', '{"@request_id":"request-B"}');

Reproduces on play.hyperdx.io? didn't try

  1. Create a HyperDX log source for this table.
  2. Set Timestamp Column to  timestamp.
  3. Set the absolute time range to  2026-11-02 12:53:00–12:54:00 UTC.
  4. Apply this SQL filter:
event.`@request_id` = 'request-B'
  1. Set SELECT to  timestamp .
  2. Open the row details.
  3. Check the returned  @request_id. If there are too little rows it could get request-B But it's not because of the filter, it's just because random sometimes could return what you asked. In case of 2 rows it's 50/50.

Actual: the production test returned another request from the same second. The detail query omitted the request filter:

SELECT * FROM default.hyperdx_row_repro
WHERE timestamp = parseDateTime64BestEffort('2026-11-02T12:53:33Z', 9)
  AND toDate(timestamp) = parseDateTime64BestEffort('2026-11-02', 9)
LIMIT 1;

This query can return either request in the fixture. The captured production response contained the wrong request ID.

Expected: display the selected event. If the query cannot identify one event, display an error.

How are you running HyperDX?

Clickhouse Cloud

Where does it show up?

Debug info

frontend: 2.41.0-sha38acb5a
backend: 2.41.0-sha38acb5a
mode: cloud
user: 6aad226315bbfa844bbf9fe3
team: 6ae883cb114b02c912d4515b
features: alertDetails, aiAssistant
url: /team
screen: 1800x1169 (viewport 1800x1066, dpr 2)
platform: MacIntel
language: en-US
ua: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:153.0) Gecko/20100101 Firefox/153.0
session: 85d51be394f63b7343630a8509512844

Logs

Activity

  1. added
    bugSomething isn't working
    Needs FeedbackUnder discussion; not ready for a PR yet
    on Nov 2, 2026
  2. pulpdrew commented on Nov 2, 2026

    @pulpdrew
    Contributor

    Thanks for reporting @libbkmz. This is a known issue. The recommended fix for ensuring that the side panel opens the correct clicked row is to apply the following settings to your source table: enable_block_number_column = 1 and enable_block_offset_column = 1.

    This ensures that each row has a unique ID that can be looked up by the side panel query, see: #2071

  3. libbkmz commented on Nov 2, 2026

    @libbkmz
    Author

    Are there any ways how to point the HyperDX to a custom column that uniquely identifies row? Like in that example we have event.`@request_id` field which could serve that purpose.

    I guess enable_block_number_column and enable_block_offset_column are table engine level settings and would require ALTER call to enable, which I currently can't afford on that heavy table I use, are there any other options?

  4. pulpdrew commented on Nov 2, 2026

    @pulpdrew
    Contributor

    Are there any ways how to point the HyperDX to a custom column that uniquely identifies row? Like in that example we have @request_id field which could serve that purpose.

    Another option would be to add additional distinguishing columns to your select expression

    I guess enable_block_number_column and enable_block_offset_column are table engine level settings and would require ALTER call to enable, which I currently can't afford on that heavy table I use, are there any other options?

    ALTER TABLE ... MODIFY SETTING on a MergeTree table just updates the table’s stored settings. It doesn’t rewrite or mutate any existing parts, and it doesn’t schedule merges or a mutation. It’s not like MATERIALIZE COLUMN or ADD COLUMN plus MATERIALIZE.

    The change applies as part writing happens:

    • New inserts create parts that include the persisted _block_number column.
    • Background merges write the setting’s column into the merged output part. Merges that were going to happen anyway start producing parts that have it.
    • Existing parts are untouched and stay as they are until they get merged. A table will therefore be a mix of parts with and without the physical column for a while.
  5. libbkmz commented on Nov 2, 2026

    @libbkmz
    Author

    Another option would be to add additional distinguishing columns to your select expression

    I tried, but it breaks other parts of the HyperDX #3519

  6. pulpdrew commented on Nov 18, 2026

    @pulpdrew
    Contributor

    #3519 has been fixed in the latest release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs FeedbackUnder discussion; not ready for a PR yetbugSomething isn't workingexternal

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions