docs(devlog): plan default-on (opt-out) telemetry and the opencodex.me usage dashboard

devlog/_plan/261107_telemetry_dashboard/000_plan.md

@@ -0,0 +1,105 @@
+# Default-on (opt-out) telemetry and the opencodex.me usage dashboard
+
+## Objective and scope
+
+Add anonymous, default-on (opt-out) product telemetry to the opencodex proxy and a public, project-wide
+analytics page at `https://opencodex.me/dashboard`. The proxy reports install/version/platform
+pings, active time, request and token totals, and error counts; a collector stores daily
+aggregates; the dashboard renders installs, active time, usage, errors and downloads.
+
+Delivery is a manual chain of stacked pull requests, one per work phase below, each based on the
+previous layer and the bottom on `dev`. No deploy of the collector, no DNS change, no merge and no
+release are part of this unit; those are maintainer actions after review.
+
+The visual and structural reference is [junhoyeo/tokscale](https://github.com/junhoyeo/tokscale)
+(MIT): its contribution heatmap, KPI header, client/model breakdown tables, and the
+validated aggregate submit API. Patterns are adapted, not vendored; credit lives in the dashboard
+footer and in the PR descriptions.
+
+## Owner decisions (2026-11-07)
+
+| Id | Decision |
+| --- | --- |
+| hosting | Collector is a Cloudflare Worker with D1 at `telemetry.opencodex.me`. The repository ships its code, schema and tests; deployment is a maintainer step. |
+| consent | Superseded by default-on. |
+| update-popup | An install that updates and has no recorded decision sees the same consent popup once after the update. |
+| default-on | Telemetry defaults to **on** (opt-out). The GUI first-run/post-update popup (checkbox pre-checked) and the first interactive `ocx start` (Y/n, default yes) let the user choose. `ocx init` and `ocx service install` ask the same question. An agent-driven start prints a notice and never answers for the user or records the notice. No payload is sent until a person has seen the disclosure. Advanced settings and `ocx telemetry disable` turn it off later. |
+| scope | The dashboard is public and project-wide (aggregates only). No per-user profiles or leaderboards. |
+
+## Privacy contract (applies to every layer)
+
+- Effective state: env opt-out (`OCX_TELEMETRY=0` or `DO_NOT_TRACK=1`) → off; decision `declined` → off; `accepted` → on; undecided → on only after `noticeShownAt` is recorded, which only a person can cause (GUI popup seen, interactive TTY prompt answered). An agent-driven start prints a notice but records nothing, so an agent can never switch sending on.
+- Identity is a random install id (UUID v4) generated on first send, never derived from
+  hardware, accounts, hostnames or paths. Disabling telemetry deletes it; re-enabling mints a new one.
+- Payloads carry only closed-vocabulary values and counters: version, OS, arch, install channel,
+  provider *kind* (closed vendor roster), coarse model family, closed
+  error codes, and integer counts. Never request or response bodies, prompts, model output, API
+  keys, OAuth tokens, account ids or e-mails, custom base URLs, hostnames, file paths, or free-form
+  error text.
+- The exact JSON the next send would contain is visible before consent ("What is sent") and via
+  the management API and `ocx telemetry status`.
+- The collector stores no IP address and no raw install id: reports carry a per-day install hash,
+  rate limiting uses a salted hourly IP hash pruned after two hours, and only the first report per
+  install and day counts.
+- No backfill: only UTC days strictly after the day of the disclosure or decision are sent, days
+  covered by an `OCX_TELEMETRY=0` / `DO_NOT_TRACK=1` opt-out are never sent (060 D2), and at most one
+  day is sent per scheduler tick.
+- The published privacy statement (README and its seven translations, the docs-site code-signing
+  page in English and Turkish, `SPONSORS.md`) describes default-on telemetry and every way to turn it
+  off. It changes in layer 2, the first layer that can send, never later (060 D12).
+- The collector URL is a constant; it is not configurable from `config.json`.
+- Telemetry code stays off the core request path: `src/router.ts`,
+  `src/server/lifecycle.ts` and `src/server/responses/core.ts` must not import `src/telemetry/`.
+  Aggregates are computed from the existing usage log on a timer, never inline in a request.
+
+## Work phases (stack order)
+
+| Layer | Doc | Thesis |
+| --- | --- | --- |
+| 1 | this unit | Docs-only roadmap. |
+| 2 | [010_client_core.md](010_client_core.md) | Config field, consent state, install id, aggregate builder, payload v1, sender, management API, `ocx telemetry`, `structure/telemetry.md`. |
+| 2b | [060_review_round.md](060_review_round.md) D21 | Windows installer page (disclosure, policy link, opt-out checkbox) and the desktop app carrying an installer opt-out into the config. |
+| 3 | [020_gui_consent.md](020_gui_consent.md) | First-run card, post-update popup, Advanced toggle, preview dialog, all locales. |
+| 4 | [030_collector.md](030_collector.md) | Cloudflare Worker + D1: ingest validation, limits, daily rollups, public stats API. |
+| 5 | [040_docs_privacy.md](040_docs_privacy.md) | Telemetry guide, sidebar, configuration reference. |
+| 6 | [050_dashboard.md](050_dashboard.md) | Astro page `/dashboard`: heatmap, KPIs, breakdowns, errors, downloads. |
+
+## Verification floor per layer
+
+`bun run typecheck`; focused tests for the layer plus `bun run test:changed`;
+`bun run structure:check`, `bun run privacy:scan`, file-size ratchet and test-layout guards for
+any layer adding files; `bun run lint:gui` and `bun run build:gui` for layer 3; docs-site build
+for layers 5 and 6; `bun run skill:surface:check` when a CLI capability is added. Each layer gets an
+independent review before its PR is opened.
+
+
+## Audit disposition (round 1)
+
+Independent audit (argon, read-only): FAIL with 9 blockers and 6 majors; all accepted and folded.
+Agent-driven starts no longer record the notice; the notice and decision routes need a human session;
+`ocx telemetry enable` and `ocx config set telemetry` refuse agent-driven use; no persisted endpoint;
+no pre-consent backfill and one day per tick; an import-free shared contract module with closed rosters
+that the collector imports; strict version regex; `OcxTelemetryConfig` in `src/types/telemetry.ts`
+and no edits to `src/cli/index.ts` or `scripts/test-layout/layout.json` (ratchet headroom 0–4 lines);
+`telemetry` added to the skill-surface roots; session-only route exemptions; the docs guide now lands
+before the dashboard; collector typecheck test; a tick-driven scheduler that re-reads config and stops in
+`stopProcessLoops`, with `OCX_TELEMETRY=0` in the test preload; explicit install-channel mapping; GUI
+dialog sequencing via `onSettled`, notice only on dismiss, `readOnly`; unified `env` and kebab-case
+causes; disjoint token buckets; `TelemetryStatsV1` with `dailyByProvider` and a 365-day heatmap; dedup
+on `(day, install_day_hash)`; prompts in `ocx init` and `ocx service install`; sidebar, configuration
+reference and both header menus. Goalplan note: work phase wp4 delivers 040 (docs) and wp5 delivers 050
+(dashboard).
+
+
+## Audit disposition (round 2)
+
+Same reviewer: PASS, 0 blockers; all 9 blockers and 6 majors verified resolved. Two minors folded: the
+collector `tsconfig.json` overrides `rootDir`/`declaration` (030), and the new capability objects in
+`src/cli/capabilities-observe-system.ts` stay plain literals because `tests/helpers/cli-capability-data.ts`
+rejects `/` tokens and host identifiers there (010).
+
+## PR review round 1
+
+Reviewer feedback on the open stack is recorded, with owner decisions D1–D20, in
+[060_review_round.md](060_review_round.md). Those decisions supersede conflicting text in 000–050, including
+the public privacy statement change that must land with the first layer able to send.

devlog/_plan/261107_telemetry_dashboard/010_client_core.md

@@ -0,0 +1,179 @@
+# 010 — Client telemetry core
+
+Stack layer 2. Everything the proxy needs to decide, build, preview and send a daily report, the
+management API and CLI that later layers use, the shared wire contract, and the structure doc that
+claims the new `src/` area. Anchors were read on `c61e957900`.
+
+## Shared wire contract (`src/telemetry/contract.ts`)
+
+An **import-free** module (same rule as `src/usage/telemetry-contract.ts`) holding the closed rosters
+and the payload validator: `TELEMETRY_SCHEMA_VERSION = 1`, `PROVIDER_KINDS`, `MODEL_FAMILIES`,
+`INSTALL_CHANNELS`, `RELEASE_CHANNELS`, `OS_VALUES`, `ARCH_VALUES`, the outcome classes and the failure
+causes (re-declared and pinned equal to `REQUEST_OUTCOME_CLASSES` / `REQUEST_FAILURE_CAUSES` by a test),
+and `validateTelemetryPayloadV1(input, now)`. The collector (030) imports this file directly, so the
+client cannot emit a key or value the collector rejects and the two cannot drift.
+
+- `PROVIDER_KINDS` (vendor-level, closed): openai, chatgpt, anthropic, claude-subscription, google,
+  vertex, xai, deepseek, moonshot, qwen, zai, minimax, mistral, openrouter, groq, cerebras, together,
+  fireworks, cursor, kiro, copilot, devin, ollama, lmstudio, local-other, other. The client maps a
+  provider's registry preset to a kind; custom/unknown providers become `other`. User-chosen provider
+  keys, labels and base URLs are never read into the payload.
+- `MODEL_FAMILIES` (closed prefix buckets): gpt-6, gpt-5, gpt-4, o-series, codex, claude-opus,
+  claude-sonnet, claude-haiku, gemini-3, gemini-2, grok, deepseek, qwen, kimi, glm, minimax, mistral,
+  llama, other.
+- `version` must match `^\d+\.\d+\.\d+(-preview\.\d{8}(\.\d+)?)?$` (no build metadata).
+
+## State model
+
+Type `OcxTelemetryConfig` lives in a new `src/types/telemetry.ts`; `src/types/config.ts` (1996 lines,
+ratchet threshold 2000) gains exactly one import line and one field line, or an existing block moves
+out first if the count says otherwise. Schema in `src/config/schema/config-schema.ts` as
+`telemetryConfigSchema.optional().catch(undefined)`, strict live-write validation in
+`src/config/diagnostics.ts`, load-degrade warning in `src/config/load-degrade.ts`, baseline adoption
+`adoptPersistedTelemetry` in `src/config/live-reconcile.ts`.
+
+```ts
+type OcxTelemetryConfig = {
+  decision?: "accepted" | "declined";   // absent = undecided
+  decidedAt?: string;                   // ISO
+  noticeShownAt?: string;               // a human saw the disclosure (GUI or interactive TTY)
+};
+```
+
+There is **no persisted endpoint**: the collector URL is the constant
+`https://telemetry.opencodex.me/v1/events` in `sender.ts` with a test-only injection seam.
+
+`resolveTelemetryStatus(config, env)` (`src/telemetry/consent.ts`):
+
+| Condition (first match) | `effective` | `reason` |
+| --- | --- | --- |
+| `OCX_TELEMETRY` ∈ {0,false,off,no} | false | `env-ocx-telemetry` |
+| `DO_NOT_TRACK` ∈ {1,true,yes} | false | `env-do-not-track` |
+| `decision === "declined"` | false | `declined` |
+| `decision === "accepted"` | true | `accepted` |
+| undecided and `noticeShownAt` set | true | `default-on` |
+| undecided, no notice | false | `awaiting-notice` |
+
+## Who may change it (agent boundary)
+
+Telemetry is on by default, so the only thing that can switch sending on for an undecided install is
+a human seeing the disclosure. Following `AGENTS_INSTALL.md` and `src/cli/star-prompt.ts`:
+
+- An agent-driven start (`isAgentDriven()`, non-TTY, or `OCX_SERVICE=1`) prints a one-line notice at
+  most once per version (`.telemetry-deferred` marker in the config dir) and writes **neither**
+  `noticeShownAt` nor `decision`. The install stays `awaiting-notice` until a human sees the GUI popup
+  or an interactive prompt.
+- `PUT /api/telemetry` and `POST /api/telemetry/notice` both require `hasHumanDashboardSession(ctx)`.
+- `ocx telemetry enable` refuses with `agent_consent_required` (non-zero exit) when `isAgentDriven()`;
+  `ocx telemetry disable` is always allowed (opting out is never a consent risk).
+- `src/cli/config-command.ts` `set`/`unset` refuse the `telemetry` key, and `import` drops a
+  `telemetry` block when agent-driven, pointing to `ocx telemetry`.
+- `AGENTS_INSTALL.md` gains a short "Telemetry" entry in its user-consent list.
+
+## Identity
+
+`src/telemetry/install-id.ts` keeps `<OPENCODEX_HOME>/telemetry-id.json` (`0o600`, exclusive no-follow
+create, pattern of `src/quota-pool/identity.ts`), created on the first send and deleted when the
+decision becomes `declined`. The raw id never leaves the machine; each report carries
+`installDayHash = sha256(installId + ":" + day)` and `newInstall: true` only on the very first report
+of a freshly minted id (the collector counts new installs per day, version and channel from it).
+
+## Aggregation (`src/telemetry/aggregate.ts`)
+
+Scans `usage.jsonl` with `scanUsageLedgerCooperatively` (`src/usage/ledger-scanner.ts`) for one
+completed UTC day:
+
+- `activeMinutes`: distinct `floor(timestamp / 60000)` values (≤ 1440).
+- `requests`: counts per outcome class via `classifyRequestOutcome` (`src/usage/request-outcome.ts`).
+- `models`: per (`providerKind`, `modelFamily`) requests and **disjoint** token buckets —
+  `inputTokens` = uncached input (`normalizeCostTokens`, `src/usage/cost.ts`), `cacheReadTokens`,
+  `cacheWriteTokens`, `outputTokens` = output minus reasoning, `reasoningTokens`. The five sum to the
+  display total (`usageDisplayTotalTokens`); a test pins that identity.
+- `errors`: counts keyed by failure cause (kebab-case, e.g. `rate-limit`).
+
+## Payload v1
+
+```json
+{ "schemaVersion": 1, "batchId": "<uuid>", "day": "2026-11-06", "installDayHash": "<64 hex>",
+  "newInstall": false,
+  "env": { "version": "2.40.1", "releaseChannel": "latest", "os": "darwin", "arch": "arm64",
+           "installChannel": "npm-global" },
+  "activeMinutes": 312,
+  "requests": { "completed": 840, "failed": 12, "incomplete": 3, "aborted": 5 },
+  "models": [ { "providerKind": "anthropic", "modelFamily": "claude-opus", "requests": 400,
+                "inputTokens": 0, "cacheReadTokens": 0, "cacheWriteTokens": 0,
+                "outputTokens": 0, "reasoningTokens": 0 } ],
+  "errors": { "rate-limit": 7 } }
+```
+
+`installChannel` mapping in `src/telemetry/payload.ts`: `OCX_DESKTOP_SUPERVISED === "1"` → desktop;
+container markers (`/.dockerenv`, `/run/.containerenv`) → docker; `detectInstallOwnership().installer`
+npm → npm-global, pnpm → pnpm-global, bun → bun-global, mise → mise, standalone → standalone,
+source → source; otherwise unknown. Every field is named explicitly in the builder and the result is
+run through `validateTelemetryPayloadV1` before sending or previewing.
+
+## Scheduling and sending
+
+- `src/server/background-lifecycle.ts` `startProcessLoops` starts a dormant `.unref()` scheduler via
+  `void import("../telemetry/scheduler")…` unless an env opt-out is set; `stopProcessLoops` stops it
+  (handle kept in background-lifecycle), and `registerOptionalShutdownHook("telemetry", stop)` covers
+  drain. `src/server/index.ts` and `src/cli/index.ts` (1999 lines) are not touched.
+- Each tick (startup jitter, then every 6 h) re-reads config passively from disk and re-evaluates
+  `resolveTelemetryStatus`, so CLI and GUI changes take effect without a restart. The management
+  routes also call `adoptPersistedTelemetry` and nudge the scheduler.
+- It sends only completed days on or after the UTC day of `decidedAt ?? noticeShownAt` (no pre-consent
+  backfill), at most **one day per tick**, oldest first, at most the last 7 days; 10 s
+  `signalWithTimeout`, no in-tick retry. Sent days live in `<OPENCODEX_HOME>/telemetry-state.json`.
+- `tests/preload.ts` sets `OCX_TELEMETRY=0` so no test process ever sends.
+
+## Management API
+
+Thin `src/server/management/telemetry-routes.ts` (seen by `management-route-registry.test.ts`)
+delegating to `src/telemetry/routes.ts`, lazily namespaced from `src/server/management-api.ts` and
+declared in `route-registry.ts`:
+
+- `GET /api/telemetry` → effective/reason/decision/env override, install channel, last sent day,
+  `preview` (the exact payload for the latest completed day; hash shown as a placeholder when no id
+  exists).
+- `PUT /api/telemetry` `{ decision }` — human session only; `exempt: { reason: "session-only" }`.
+- `POST /api/telemetry/notice` — human session only, idempotent; `exempt: { reason: "session-only" }`.
+
+## CLI
+
+`src/cli/telemetry.ts`: `ocx telemetry status [--json]` (route `GET /api/telemetry`, falls back to
+local status when the proxy is down), `enable` and `disable` (local config mutation, `routes: []`,
+`mutates: true`). Capability objects stay plain string/array/object literals (the lexical guard in `tests/helpers/cli-capability-data.ts` rejects `/` tokens and host identifiers). Registered in `capabilities-observe-system.ts`, `registry.ts`, `dispatch.ts`,
+`help.ts`; `telemetry` is added to the `observe-system` roots in
+`scripts/generate-ocx-skill-surface.ts` and `tests/ci-workflows/skill-ocx-generated.test.ts`, then
+`bun run skill:surface`.
+
+Interactive prompt `Share anonymous usage statistics? [Y/n]` (default yes; Enter = yes; records
+`decision` and `noticeShownAt`) shown once when undecided, from: `maybeShowStarPrompt()`'s module on
+interactive `ocx start` (no new line in `src/cli/index.ts`), `ocx service install` next to its star
+prompt (`src/service/cli.ts`), and the end of `ocx init` (`src/cli/init.ts`).
+
+## Ownership
+
+`structure/telemetry.md` registered in `structure/manifest.json` claiming `src/telemetry/`,
+`src/types/telemetry.ts`, `src/server/management/telemetry-routes.ts` and `src/cli/telemetry.ts`,
+naming only paths that exist in this layer; `bun run structure:index`. The four 600-line structure
+docs are not extended.
+
+## Verification
+
+- `tests/usage/usage-telemetry-client.test.ts`: status matrix, env precedence, id file mode and
+  deletion, aggregate bucketing and the disjoint-token identity on a fixture ledger, payload validator
+  rejects extra keys/unknown roster values/semver metadata, roster parity with the usage contract, no
+  pre-consent days, one day per tick, sender timeout, nothing sent when not effective.
+- `tests/usage/usage-telemetry-core-boundary.test.ts`: `router.ts`, `lifecycle.ts`,
+  `responses/core.ts`, `management-api.ts` have no load-time path to `src/telemetry/`.
+- `tests/server/server-telemetry-routes.test.ts`: GET preview; PUT and notice refused with admin token
+  and remembered-admin; accepted with a human session; notice idempotent.
+- `tests/cli/cli-telemetry.test.ts`: status/enable/disable, enable refused when agent-driven, config
+  set refusal, agent-driven start writes no notice, prompt default yes.
+- New tests registered in a domain shard (`layout-usage.json` etc.; `layout.json` is at 1999 lines)
+  and `tests/fixtures/test-layout-expected.json`.
+- `bun run typecheck`, focused files, `bun run test:changed`, `bun run structure:check`,
+  `bun run skill:surface:check`, `bun run privacy:scan`, file-size and layout guards,
+  `tests/cli/cli-capabilities.test.ts`, `tests/server/management-route-registry.test.ts`.
+