schema: unify pg_stat_ch events_raw with prod Arrow path, move into Goose layout

[JoshDreamland]

Summary

Pre-GA unification of pg_stat_ch's ClickHouse schema. The docker quickstart schema (docker/init/00-schema.sql) and the production Arrow receiver schema (datagres_otel.query_logs_arrow in clickgres-platform) had drifted apart on both column naming and column types. This PR makes pg_stat_ch the source of truth for the unified shape and moves the canonical schema into a Goose migrations layout under schema/migrations/ matching the runner clickgres-platform already uses (pressly/goose v3, DialectClickHouse, embed.FS).

Two commits, structured so git rename detection can follow the file's evolution cleanly:

Commit 1 — 9c39ecd: in-place unification + tests pass

In-place edits to docker/init/00-schema.sql so the docker quickstart schema matches what prod actually writes to, plus the CH-native exporter and TAP tests updated to use the new column names.

Column renames (prod-side wins — closer to OTel semantic conventions, minimizes downstream churn):

• ts_start → ts
• db → db_name
• username → db_user
• cmd_type → db_operation
• query → query_text

Type fix: err_sqlstate FixedString(5) → LowCardinality(String). FixedString doesn't round-trip through Arrow IPC cleanly, and ~270 SQLSTATE codes are dictionary-friendly. CH-native exporter switches from MetricFixedString(5, …) to TagString(…) (clickhouse-cpp ColumnString → CH LowCardinality(String) is fine on the wire).

Envelope columns added with DEFAULT '' so the CH-native exporter (which doesn't yet emit these) still inserts successfully: instance_ubid, server_ubid, server_role, region, cell, service_version, host_id, pod_name.

Engine/partitioning aligned with prod:

• ORDER BY ts → ORDER BY (instance_ubid, ts) (tenant locality)
• TTL toDate(ts) + INTERVAL 180 DAY
• SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1

All four MVs (events_recent_1h, query_stats_5m, db_app_user_1m, errors_recent) updated to reference the new column names and to include instance_ubid in their ORDER BY / GROUP BY / SELECT projections.

parent_query_id is intentionally not included here — it belongs to PR #95 (parent-query-id-surgical) and will land as its own follow-up migration in schema/migrations/ after this PR.

Commit 2 — 299115b: rename + goose annotations

git mv docker/init/00-schema.sql schema/migrations/20260519000001_create_initial_schema.sql (96% similarity per git's rename detection), with:

• Header banner rewritten from "CANONICAL SCHEMA REFERENCE / single source of truth" framing to "initial migration" framing.
• -- +goose Up / -- +goose Down section markers added.
• Each CREATE wrapped in -- +goose StatementBegin / StatementEnd.
• Pre-CREATE DROP TABLE IF EXISTS X idioms removed — those existed for docker init idempotency on restart; goose tracks state via goose_db_version. Drops live exclusively in the -- +goose Down section in reverse dependency order.

Also adds schema/migrations/00000000000001_bootstrap.sql, a no-op SELECT 1 migration required by goose to seed its version table (copied verbatim from clickgres-platform's bootstrap).

What's been validated locally

• docker/init/00-schema.sql (commit 1) applies cleanly on clickhouse/clickhouse-server:26.1 (the version pinned in docker/docker-compose.test.yml); all 51 columns and 4 MVs land with the expected types.
• Envelope columns' DEFAULT '' lets INSERTs that omit them succeed.
• schema/migrations/ (commit 2) round-trips clean via goose v3.27.1 up and goose reset against CH 26.1.
• C++ source edits compile-clean syntactically (no diagnostics from the column-name changes); the local build needs a Linux env (presets are all Linux-targeted), so CI's the real check.

Out of scope (follow-on PRs)

• Wire docker/init/ and docker-compose.test.yml to run goose-up at container start. Currently docker/init/00-schema.sql is gone, so the docker quickstart and the test compose need a small shim to apply migrations from schema/migrations/ (clickhouse-server's docker entrypoint can't parse -- +goose Up/Down directly).
• Coordinated cutover from clickgres-platform's query_logs_arrow to the unified events_raw, including historical backfill via INSERT INTO events_raw SELECT … FROM query_logs_arrow with explicit casts for the renamed/retyped columns.
• After PR feat: emit parent_query_id to link nested SPI queries #95 merges, add a follow-on migration schema/migrations/<ts>_add_parent_query_id.sql for the column it introduces.

Test plan

• TAP suite passes on PG 16/17/18 × amd64/arm64 (CI)
• PGXN / code style / Clang checks (CI)
• Spot-check the rename diff on commit 2 — git should report it as a 96% similarity rename

🤖 Generated with Claude Code

[serprex]

does this need corresponding change in clickgres-platform? eg for query_id being Int64 vs String

can clickgres-platform submodule pg_stat_ch for schema?

description talks of cutover, but seems that's not too necessary since CH exporter not being used in prod

can we add test that does both CH & arrow? potentially one-after-the-other. to test schema compatibility