From b9ca17478ce3032bdbcfa7387b0c365d63b9e2cd Mon Sep 17 00:00:00 2001 From: JUN Date: Sat, 7 Nov 2026 22:52:29 +0900 Subject: [PATCH 1/4] docs(devlog): plan opt-in telemetry and the opencodex.me usage dashboard Roadmap unit for a six-layer stack: client telemetry core, GUI consent popup, Cloudflare Worker collector, telemetry guide, and the public /dashboard page. Docs only. --- .../261107_telemetry_dashboard/000_plan.md | 93 +++++++++ .../010_client_core.md | 179 ++++++++++++++++++ .../020_gui_consent.md | 55 ++++++ .../030_collector.md | 80 ++++++++ .../040_docs_privacy.md | 31 +++ .../050_dashboard.md | 43 +++++ 6 files changed, 481 insertions(+) create mode 100644 devlog/_plan/261107_telemetry_dashboard/000_plan.md create mode 100644 devlog/_plan/261107_telemetry_dashboard/010_client_core.md create mode 100644 devlog/_plan/261107_telemetry_dashboard/020_gui_consent.md create mode 100644 devlog/_plan/261107_telemetry_dashboard/030_collector.md create mode 100644 devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md create mode 100644 devlog/_plan/261107_telemetry_dashboard/050_dashboard.md diff --git a/devlog/_plan/261107_telemetry_dashboard/000_plan.md b/devlog/_plan/261107_telemetry_dashboard/000_plan.md new file mode 100644 index 00000000000..e694f22d51a --- /dev/null +++ b/devlog/_plan/261107_telemetry_dashboard/000_plan.md @@ -0,0 +1,93 @@ +# Opt-in telemetry and the opencodex.me usage dashboard + +## Objective and scope + +Add anonymous, opt-in product telemetry to the opencodex proxy and a public, project-wide +analytics page at `https://opencodex.me/dashboard`. The proxy reports install/version/platform +pings, active time, request and token totals, and error counts; a collector stores daily +aggregates; the dashboard renders installs, active time, usage, errors and downloads. + +Delivery is a manual chain of stacked pull requests, one per work phase below, each based on the +previous layer and the bottom on `dev`. No deploy of the collector, no DNS change, no merge and no +release are part of this unit; those are maintainer actions after review. + +The visual and structural reference is [junhoyeo/tokscale](https://github.com/junhoyeo/tokscale) +(MIT): its contribution heatmap, KPI header, client/model breakdown tables, and the +validated aggregate submit API. Patterns are adapted, not vendored; credit lives in the dashboard +footer and in the PR descriptions. + +## Owner decisions (2026-11-07) + +| Id | Decision | +| --- | --- | +| hosting | Collector is a Cloudflare Worker with D1 at `telemetry.opencodex.me`. The repository ships its code, schema and tests; deployment is a maintainer step. | +| consent | Superseded by default-on. | +| update-popup | An install that updates and has no recorded decision sees the same consent popup once after the update. | +| default-on | Telemetry defaults to **on** (opt-out). The GUI first-run/post-update popup (checkbox pre-checked) and the first interactive `ocx start` (Y/n, default yes) let the user choose. `ocx init` and `ocx service install` ask the same question. An agent-driven start prints a notice and never answers for the user or records the notice. No payload is sent until a person has seen the disclosure. Advanced settings and `ocx telemetry disable` turn it off later. | +| scope | The dashboard is public and project-wide (aggregates only). No per-user profiles or leaderboards. | + +## Privacy contract (applies to every layer) + +- Effective state: env opt-out (`OCX_TELEMETRY=0` or `DO_NOT_TRACK=1`) → off; decision `declined` → off; `accepted` → on; undecided → on only after `noticeShownAt` is recorded, which only a person can cause (GUI popup seen, interactive TTY prompt answered). An agent-driven start prints a notice but records nothing, so an agent can never switch sending on. +- Identity is a random install id (UUID v4) generated on first send, never derived from + hardware, accounts, hostnames or paths. Disabling telemetry deletes it; re-enabling mints a new one. +- Payloads carry only closed-vocabulary values and counters: version, OS, arch, install channel, + provider *kind* (closed vendor roster), coarse model family, closed + error codes, and integer counts. Never request or response bodies, prompts, model output, API + keys, OAuth tokens, account ids or e-mails, custom base URLs, hostnames, file paths, or free-form + error text. +- The exact JSON the next send would contain is visible before consent ("What is sent") and via + the management API and `ocx telemetry status`. +- The collector stores no IP address and no raw install id: reports carry a per-day install hash, + rate limiting uses a salted hourly IP hash pruned after two hours, and only the first report per + install and day counts. +- No backfill: days before the disclosure are never sent; at most one day is sent per scheduler tick. +- The collector URL is a constant; it is not configurable from `config.json`. +- Telemetry code stays off the core request path: `src/router.ts`, + `src/server/lifecycle.ts` and `src/server/responses/core.ts` must not import `src/telemetry/`. + Aggregates are computed from the existing usage log on a timer, never inline in a request. + +## Work phases (stack order) + +| Layer | Doc | Thesis | +| --- | --- | --- | +| 1 | this unit | Docs-only roadmap. | +| 2 | [010_client_core.md](010_client_core.md) | Config field, consent state, install id, aggregate builder, payload v1, sender, management API, `ocx telemetry`, `structure/telemetry.md`. | +| 3 | [020_gui_consent.md](020_gui_consent.md) | First-run card, post-update popup, Advanced toggle, preview dialog, all locales. | +| 4 | [030_collector.md](030_collector.md) | Cloudflare Worker + D1: ingest validation, limits, daily rollups, public stats API. | +| 5 | [040_docs_privacy.md](040_docs_privacy.md) | Telemetry guide, sidebar, configuration reference. | +| 6 | [050_dashboard.md](050_dashboard.md) | Astro page `/dashboard`: heatmap, KPIs, breakdowns, errors, downloads. | + +## Verification floor per layer + +`bun run typecheck`; focused tests for the layer plus `bun run test:changed`; +`bun run structure:check`, `bun run privacy:scan`, file-size ratchet and test-layout guards for +any layer adding files; `bun run lint:gui` and `bun run build:gui` for layer 3; docs-site build +for layers 5 and 6; `bun run skill:surface:check` when a CLI capability is added. Each layer gets an +independent review before its PR is opened. + + +## Audit disposition (round 1) + +Independent audit (argon, read-only): FAIL with 9 blockers and 6 majors; all accepted and folded. +Agent-driven starts no longer record the notice; the notice and decision routes need a human session; +`ocx telemetry enable` and `ocx config set telemetry` refuse agent-driven use; no persisted endpoint; +no pre-consent backfill and one day per tick; an import-free shared contract module with closed rosters +that the collector imports; strict version regex; `OcxTelemetryConfig` in `src/types/telemetry.ts` +and no edits to `src/cli/index.ts` or `scripts/test-layout/layout.json` (ratchet headroom 0–4 lines); +`telemetry` added to the skill-surface roots; session-only route exemptions; the docs guide now lands +before the dashboard; collector typecheck test; a tick-driven scheduler that re-reads config and stops in +`stopProcessLoops`, with `OCX_TELEMETRY=0` in the test preload; explicit install-channel mapping; GUI +dialog sequencing via `onSettled`, notice only on dismiss, `readOnly`; unified `env` and kebab-case +causes; disjoint token buckets; `TelemetryStatsV1` with `dailyByProvider` and a 365-day heatmap; dedup +on `(day, install_day_hash)`; prompts in `ocx init` and `ocx service install`; sidebar, configuration +reference and both header menus. Goalplan note: work phase wp4 delivers 040 (docs) and wp5 delivers 050 +(dashboard). + + +## Audit disposition (round 2) + +Same reviewer: PASS, 0 blockers; all 9 blockers and 6 majors verified resolved. Two minors folded: the +collector `tsconfig.json` overrides `rootDir`/`declaration` (030), and the new capability objects in +`src/cli/capabilities-observe-system.ts` stay plain literals because `tests/helpers/cli-capability-data.ts` +rejects `/` tokens and host identifiers there (010). diff --git a/devlog/_plan/261107_telemetry_dashboard/010_client_core.md b/devlog/_plan/261107_telemetry_dashboard/010_client_core.md new file mode 100644 index 00000000000..62a3cb13885 --- /dev/null +++ b/devlog/_plan/261107_telemetry_dashboard/010_client_core.md @@ -0,0 +1,179 @@ +# 010 — Client telemetry core + +Stack layer 2. Everything the proxy needs to decide, build, preview and send a daily report, the +management API and CLI that later layers use, the shared wire contract, and the structure doc that +claims the new `src/` area. Anchors were read on `c61e957900`. + +## Shared wire contract (`src/telemetry/contract.ts`) + +An **import-free** module (same rule as `src/usage/telemetry-contract.ts`) holding the closed rosters +and the payload validator: `TELEMETRY_SCHEMA_VERSION = 1`, `PROVIDER_KINDS`, `MODEL_FAMILIES`, +`INSTALL_CHANNELS`, `RELEASE_CHANNELS`, `OS_VALUES`, `ARCH_VALUES`, the outcome classes and the failure +causes (re-declared and pinned equal to `REQUEST_OUTCOME_CLASSES` / `REQUEST_FAILURE_CAUSES` by a test), +and `validateTelemetryPayloadV1(input, now)`. The collector (030) imports this file directly, so the +client cannot emit a key or value the collector rejects and the two cannot drift. + +- `PROVIDER_KINDS` (vendor-level, closed): openai, chatgpt, anthropic, claude-subscription, google, + vertex, xai, deepseek, moonshot, qwen, zai, minimax, mistral, openrouter, groq, cerebras, together, + fireworks, cursor, kiro, copilot, devin, ollama, lmstudio, local-other, other. The client maps a + provider's registry preset to a kind; custom/unknown providers become `other`. User-chosen provider + keys, labels and base URLs are never read into the payload. +- `MODEL_FAMILIES` (closed prefix buckets): gpt-6, gpt-5, gpt-4, o-series, codex, claude-opus, + claude-sonnet, claude-haiku, gemini-3, gemini-2, grok, deepseek, qwen, kimi, glm, minimax, mistral, + llama, other. +- `version` must match `^\d+\.\d+\.\d+(-preview\.\d+)?$` (no build metadata). + +## State model + +Type `OcxTelemetryConfig` lives in a new `src/types/telemetry.ts`; `src/types/config.ts` (1996 lines, +ratchet threshold 2000) gains exactly one import line and one field line, or an existing block moves +out first if the count says otherwise. Schema in `src/config/schema/config-schema.ts` as +`telemetryConfigSchema.optional().catch(undefined)`, strict live-write validation in +`src/config/diagnostics.ts`, load-degrade warning in `src/config/load-degrade.ts`, baseline adoption +`adoptPersistedTelemetry` in `src/config/live-reconcile.ts`. + +```ts +type OcxTelemetryConfig = { + decision?: "accepted" | "declined"; // absent = undecided + decidedAt?: string; // ISO + noticeShownAt?: string; // a human saw the disclosure (GUI or interactive TTY) +}; +``` + +There is **no persisted endpoint**: the collector URL is the constant +`https://telemetry.opencodex.me/v1/events` in `sender.ts` with a test-only injection seam. + +`resolveTelemetryStatus(config, env)` (`src/telemetry/consent.ts`): + +| Condition (first match) | `effective` | `reason` | +| --- | --- | --- | +| `OCX_TELEMETRY` ∈ {0,false,off,no} | false | `env-ocx-telemetry` | +| `DO_NOT_TRACK` ∈ {1,true,yes} | false | `env-do-not-track` | +| `decision === "declined"` | false | `declined` | +| `decision === "accepted"` | true | `accepted` | +| undecided and `noticeShownAt` set | true | `default-on` | +| undecided, no notice | false | `awaiting-notice` | + +## Who may change it (agent boundary) + +Telemetry is on by default, so the only thing that can switch sending on for an undecided install is +a human seeing the disclosure. Following `AGENTS_INSTALL.md` and `src/cli/star-prompt.ts`: + +- An agent-driven start (`isAgentDriven()`, non-TTY, or `OCX_SERVICE=1`) prints a one-line notice at + most once per version (`.telemetry-deferred` marker in the config dir) and writes **neither** + `noticeShownAt` nor `decision`. The install stays `awaiting-notice` until a human sees the GUI popup + or an interactive prompt. +- `PUT /api/telemetry` and `POST /api/telemetry/notice` both require `hasHumanDashboardSession(ctx)`. +- `ocx telemetry enable` refuses with `agent_consent_required` (non-zero exit) when `isAgentDriven()`; + `ocx telemetry disable` is always allowed (opting out is never a consent risk). +- `src/cli/config-command.ts` `set`/`unset` refuse the `telemetry` key, and `import` drops a + `telemetry` block when agent-driven, pointing to `ocx telemetry`. +- `AGENTS_INSTALL.md` gains a short "Telemetry" entry in its user-consent list. + +## Identity + +`src/telemetry/install-id.ts` keeps `/telemetry-id.json` (`0o600`, exclusive no-follow +create, pattern of `src/quota-pool/identity.ts`), created on the first send and deleted when the +decision becomes `declined`. The raw id never leaves the machine; each report carries +`installDayHash = sha256(installId + ":" + day)` and `newInstall: true` only on the very first report +of a freshly minted id (the collector counts new installs per day, version and channel from it). + +## Aggregation (`src/telemetry/aggregate.ts`) + +Scans `usage.jsonl` with `scanUsageLedgerCooperatively` (`src/usage/ledger-scanner.ts`) for one +completed UTC day: + +- `activeMinutes`: distinct `floor(timestamp / 60000)` values (≤ 1440). +- `requests`: counts per outcome class via `classifyRequestOutcome` (`src/usage/request-outcome.ts`). +- `models`: per (`providerKind`, `modelFamily`) requests and **disjoint** token buckets — + `inputTokens` = uncached input (`normalizeCostTokens`, `src/usage/cost.ts`), `cacheReadTokens`, + `cacheWriteTokens`, `outputTokens` = output minus reasoning, `reasoningTokens`. The five sum to the + display total (`usageDisplayTotalTokens`); a test pins that identity. +- `errors`: counts keyed by failure cause (kebab-case, e.g. `rate-limit`). + +## Payload v1 + +```json +{ "schemaVersion": 1, "batchId": "", "day": "2026-11-06", "installDayHash": "<64 hex>", + "newInstall": false, + "env": { "version": "2.40.1", "releaseChannel": "latest", "os": "darwin", "arch": "arm64", + "installChannel": "npm-global" }, + "activeMinutes": 312, + "requests": { "completed": 840, "failed": 12, "incomplete": 3, "aborted": 5 }, + "models": [ { "providerKind": "anthropic", "modelFamily": "claude-opus", "requests": 400, + "inputTokens": 0, "cacheReadTokens": 0, "cacheWriteTokens": 0, + "outputTokens": 0, "reasoningTokens": 0 } ], + "errors": { "rate-limit": 7 } } +``` + +`installChannel` mapping in `src/telemetry/payload.ts`: `OCX_DESKTOP_SUPERVISED === "1"` → desktop; +container markers (`/.dockerenv`, `/run/.containerenv`) → docker; `detectInstallOwnership().installer` +npm → npm-global, pnpm → pnpm-global, bun → bun-global, mise → mise, standalone → standalone, +source → source; otherwise unknown. Every field is named explicitly in the builder and the result is +run through `validateTelemetryPayloadV1` before sending or previewing. + +## Scheduling and sending + +- `src/server/background-lifecycle.ts` `startProcessLoops` starts a dormant `.unref()` scheduler via + `void import("../telemetry/scheduler")…` unless an env opt-out is set; `stopProcessLoops` stops it + (handle kept in background-lifecycle), and `registerOptionalShutdownHook("telemetry", stop)` covers + drain. `src/server/index.ts` and `src/cli/index.ts` (1999 lines) are not touched. +- Each tick (startup jitter, then every 6 h) re-reads config passively from disk and re-evaluates + `resolveTelemetryStatus`, so CLI and GUI changes take effect without a restart. The management + routes also call `adoptPersistedTelemetry` and nudge the scheduler. +- It sends only completed days on or after the UTC day of `decidedAt ?? noticeShownAt` (no pre-consent + backfill), at most **one day per tick**, oldest first, at most the last 7 days; 10 s + `signalWithTimeout`, no in-tick retry. Sent days live in `/telemetry-state.json`. +- `tests/preload.ts` sets `OCX_TELEMETRY=0` so no test process ever sends. + +## Management API + +Thin `src/server/management/telemetry-routes.ts` (seen by `management-route-registry.test.ts`) +delegating to `src/telemetry/routes.ts`, lazily namespaced from `src/server/management-api.ts` and +declared in `route-registry.ts`: + +- `GET /api/telemetry` → effective/reason/decision/env override, install channel, last sent day, + `preview` (the exact payload for the latest completed day; hash shown as a placeholder when no id + exists). +- `PUT /api/telemetry` `{ decision }` — human session only; `exempt: { reason: "session-only" }`. +- `POST /api/telemetry/notice` — human session only, idempotent; `exempt: { reason: "session-only" }`. + +## CLI + +`src/cli/telemetry.ts`: `ocx telemetry status [--json]` (route `GET /api/telemetry`, falls back to +local status when the proxy is down), `enable` and `disable` (local config mutation, `routes: []`, +`mutates: true`). Capability objects stay plain string/array/object literals (the lexical guard in `tests/helpers/cli-capability-data.ts` rejects `/` tokens and host identifiers). Registered in `capabilities-observe-system.ts`, `registry.ts`, `dispatch.ts`, +`help.ts`; `telemetry` is added to the `observe-system` roots in +`scripts/generate-ocx-skill-surface.ts` and `tests/ci-workflows/skill-ocx-generated.test.ts`, then +`bun run skill:surface`. + +Interactive prompt `Share anonymous usage statistics? [Y/n]` (default yes; Enter = yes; records +`decision` and `noticeShownAt`) shown once when undecided, from: `maybeShowStarPrompt()`'s module on +interactive `ocx start` (no new line in `src/cli/index.ts`), `ocx service install` next to its star +prompt (`src/service/cli.ts`), and the end of `ocx init` (`src/cli/init.ts`). + +## Ownership + +`structure/telemetry.md` registered in `structure/manifest.json` claiming `src/telemetry/`, +`src/types/telemetry.ts`, `src/server/management/telemetry-routes.ts` and `src/cli/telemetry.ts`, +naming only paths that exist in this layer; `bun run structure:index`. The four 600-line structure +docs are not extended. + +## Verification + +- `tests/usage/usage-telemetry-client.test.ts`: status matrix, env precedence, id file mode and + deletion, aggregate bucketing and the disjoint-token identity on a fixture ledger, payload validator + rejects extra keys/unknown roster values/semver metadata, roster parity with the usage contract, no + pre-consent days, one day per tick, sender timeout, nothing sent when not effective. +- `tests/usage/usage-telemetry-core-boundary.test.ts`: `router.ts`, `lifecycle.ts`, + `responses/core.ts`, `management-api.ts` have no load-time path to `src/telemetry/`. +- `tests/server/server-telemetry-routes.test.ts`: GET preview; PUT and notice refused with admin token + and remembered-admin; accepted with a human session; notice idempotent. +- `tests/cli/cli-telemetry.test.ts`: status/enable/disable, enable refused when agent-driven, config + set refusal, agent-driven start writes no notice, prompt default yes. +- New tests registered in a domain shard (`layout-usage.json` etc.; `layout.json` is at 1999 lines) + and `tests/fixtures/test-layout-expected.json`. +- `bun run typecheck`, focused files, `bun run test:changed`, `bun run structure:check`, + `bun run skill:surface:check`, `bun run privacy:scan`, file-size and layout guards, + `tests/cli/cli-capabilities.test.ts`, `tests/server/management-route-registry.test.ts`. + diff --git a/devlog/_plan/261107_telemetry_dashboard/020_gui_consent.md b/devlog/_plan/261107_telemetry_dashboard/020_gui_consent.md new file mode 100644 index 00000000000..96a0a3bc3b7 --- /dev/null +++ b/devlog/_plan/261107_telemetry_dashboard/020_gui_consent.md @@ -0,0 +1,55 @@ +# 020 — GUI consent popup, post-update popup and Advanced toggle + +Stack layer 3. Depends on 010 (`GET /api/telemetry`, `PUT /api/telemetry`, `POST /api/telemetry/notice`). + +## Behaviour + +- `TelemetryConsentCard` is a native `` mounted beside + `` in `gui/src/App.tsx`. It is enabled only when + `targetsSettled && !targets.connected && sharedSessionReady && starOnboardingSettled`. + `DesktopStarOnboarding` gains an `onSettled` callback (fired when it closes or decides it will not + show), so the two dialogs never open together. +- It opens when the status reports an undecided install with no env override and the proxy says + `reason === "awaiting-notice"` or `"default-on"` without a decision. That covers first run and + installs that update into this version (they have no decision yet): the post-update popup. +- The checkbox is **pre-checked**. Continue sends `PUT /api/telemetry { decision: checked ? "accepted" : + "declined" }` (this also records the notice server-side). Escape/backdrop without a decision sends + `POST /api/telemetry/notice`: the user saw the disclosure and the default (on) applies. Opening the + dialog alone records nothing, so an unread or automated page load never enables sending. +- The card lists what is sent and what is never sent and has a "What is sent" disclosure with the + exact preview JSON from the status endpoint. +- `TelemetrySettingsPanel` renders in Proxy details (`SettingsSection id="proxy-telemetry"` in + `dashboard-overview-panels.tsx`, id appended to `PROXY_SECTION_IDS`) with `Switch` from `ui.tsx`, + `readOnly` honoured for Remote Link children, and the preview dialog. With `OCX_TELEMETRY=0` or + `DO_NOT_TRACK=1` the switch is disabled and a `Notice tone="warn"` names the variable. + `AdvancedHub.tsx` gets `telemetry`/`privacy` search words on the proxy entry. +- The panel and card fetch on their own with `createBoundedFetch(15_000)`, outside + `useDashboardData` (dashboard-contracts poll-wave rule). + +## Files + +| File | Change | +| --- | --- | +| `gui/src/telemetry-api.ts` | DTO types and fetch helpers (non-component, oxlint `only-export-components`). | +| `gui/src/components/telemetry-consent-card.tsx` | First-run / post-update dialog. | +| `gui/src/components/TelemetrySettingsPanel.tsx` | Toggle row and preview dialog. | +| `gui/src/components/desktop-star-onboarding.tsx` | `onSettled` prop. | +| `gui/src/App.tsx` | Mount the card, track `starOnboardingSettled`. | +| `gui/src/pages/dashboard-section-ids.ts`, `dashboard-overview-panels.tsx`, `AdvancedHub.tsx` | Section, panel, search words. | +| `gui/src/i18n/v2-pages/proxy-copy.ts` | All `proxyV2.telemetry*` rows in the 11-column table (en, ko, de, fr, zh, zh-TW, ru, ja, tr, vi, pt). | + +No CSS is added to `gui/src/styles.css` (ratchet cap); reuse `.modal-overlay`/`.modal-card` and v2 +utilities. + +## Verification + +- `gui/tests/telemetry-consent-card.test.tsx`: waits for star onboarding; shows only when undecided + and no env override; checkbox pre-checked; nothing recorded on open; Continue sends accepted / + declined; Escape sends notice only; hidden once decided. +- `gui/tests/telemetry-settings-panel.test.tsx`: switch mirrors effective state, toggles via PUT, + disabled with env notice and in read-only mode, preview dialog shows the JSON. +- `gui/tests/desktop-star-onboarding.test.tsx` extended for `onSettled`. +- `cd gui && bun test tests && bun run lint && bun run lint:i18n && bun run build`; root + `bun run typecheck`. +- Screenshot of the card and panel for the PR description (uploaded via `pr-assets`, never committed). + diff --git a/devlog/_plan/261107_telemetry_dashboard/030_collector.md b/devlog/_plan/261107_telemetry_dashboard/030_collector.md new file mode 100644 index 00000000000..a8a02208b02 --- /dev/null +++ b/devlog/_plan/261107_telemetry_dashboard/030_collector.md @@ -0,0 +1,80 @@ +# 030 — Telemetry collector (Cloudflare Worker + D1) + +Stack layer 4. Imports the wire contract `src/telemetry/contract.ts` from 010 directly. + +## Layout (dependency-free) + +| File | Role | +| --- | --- | +| `services/telemetry-collector/wrangler.toml` | Worker name, route `telemetry.opencodex.me/*`, D1 binding `DB` with placeholder id `00000000-0000-0000-0000-000000000000`, cron trigger for pruning. Real ids are set by a maintainer at deploy time. | +| `services/telemetry-collector/schema.sql` | D1 tables (below). | +| `services/telemetry-collector/tsconfig.json` | Extends root settings with `rootDir: "../.."`, `noEmit: true`, `declaration: false` (as in `scripts/test-layout/tsconfig.verify.json`) so importing `../../src/telemetry/contract.ts` does not hit TS6059. | +| `services/telemetry-collector/src/types.ts` | Minimal local `D1Database`/`D1PreparedStatement`/`ExecutionContext` interfaces, `CollectorEnv`, `TelemetryStatsV1`. | +| `services/telemetry-collector/src/ingest.ts` | `POST /v1/events`. | +| `services/telemetry-collector/src/stats.ts` | `GET /v1/stats?range=7d|30d|90d|365d`. | +| `services/telemetry-collector/src/prune.ts` | Scheduled pruning. | +| `services/telemetry-collector/src/index.ts` | `fetch`/`scheduled` handlers, CORS, 404/405. | +| `services/telemetry-collector/README.md` | Deploy: `bunx wrangler d1 create`, `d1 execute --file schema.sql`, secrets (`RATE_LIMIT_SALT`), `wrangler deploy`. | + +No root `package.json`/`bun.lock` change. The root typecheck only covers `src/`; this layer adds a +test that typechecks the collector project with the repository's own TypeScript toolchain (command +chosen and proven in B, since `node_modules/.bin/tsc` is absent and `bun check` is the root checker). + +## Storage + +- `daily_installs(day, install_day_hash, version, release_channel, os, arch, install_channel, + new_install, active_minutes, PRIMARY KEY(day, install_day_hash))` — pruned after 400 days. +- `daily_totals(day, reports, new_installs, active_minutes, completed, failed, incomplete, aborted, + input, cache_read, cache_write, output, reasoning)`. +- `daily_models(day, provider_kind, model_family, requests, input, cache_read, cache_write, output, + reasoning)`. +- `daily_errors(day, cause, count)`. +- `rate_limits(bucket_hour, ip_hash, hits)` — pruned after 2 h. + +**Dedup:** ingest first `INSERT OR IGNORE` into `daily_installs`; only when that inserted a new +`(day, install_day_hash)` row are the rollups incremented (one D1 `batch`). A re-sent day — new +`batchId` or not — changes nothing, so lost client state cannot inflate totals. Raw install ids and IPs +are never stored; `ip_hash = sha256(RATE_LIMIT_SALT + ip + hour)`. + +## Validation and limits + +`validateTelemetryPayloadV1` from the contract (closed keys at every level, closed rosters for +provider kind, model family, channels, os, arch, failure causes; calendar-checked `day` in +[today − 7, today + 1]; strict version regex; bounded integers; ≤ 64 model rows). Body ≤ 16 KiB +(413), JSON only (400), 30 reports/hour per IP hash (429 with `Retry-After`). + +## Stats API (`TelemetryStatsV1`, consumed by 050) + +```ts +type TelemetryStatsV1 = { + schemaVersion: 1; generatedAt: string; range: "7d" | "30d" | "90d" | "365d"; + totals: { activeInstalls: number; newInstalls: number; activeMinutes: number; requests: number; + tokens: { input: number; cacheRead: number; cacheWrite: number; output: number; reasoning: number } }; + daily: Array<{ day: string; activeInstalls: number; newInstalls: number; activeMinutes: number; + requests: number; failed: number; tokens: number }>; + dailyByProvider: Array<{ day: string; providerKind: string; requests: number; tokens: number }>; + heatmap: Array<{ day: string; activeInstalls: number }>; // always the last 365 days + providers: Array<{ providerKind: string; requests: number; tokens: number }>; + models: Array<{ modelFamily: string; requests: number; tokens: number }>; + versions: Array<{ version: string; installs: number; newInstalls: number }>; + platforms: Array<{ os: string; arch: string; installs: number }>; + channels: Array<{ installChannel: string; installs: number; newInstalls: number }>; + outcomes: { completed: number; failed: number; incomplete: number; aborted: number }; + errors: Array<{ cause: string; count: number }>; +}; +``` + +Served with `Cache-Control: public, max-age=300` and CORS for `https://opencodex.me` and +`http://localhost:4321`. + +## Verification + +- `tests/usage/usage-telemetry-collector-worker.test.ts`: `bun:sqlite` in-memory D1 shim loading + `schema.sql` via `repoPath`; ingest accepts a payload built by the 010 builder, rejects unknown keys + and roster values, dedups re-sent days, rate-limits with 429, sets CORS, returns every stats field + for each range with a full 365-day heatmap, never stores a raw id or IP; pruning. +- `tests/usage/usage-telemetry-collector-typecheck.test.ts`: the collector project typechecks. +- Registered in `scripts/test-layout/layout-usage.json` and `tests/fixtures/test-layout-expected.json`; + add `ci:full` if the import graph misses them. +- `bun run typecheck`, `bun run privacy:scan` (placeholder ids only), file-size and layout guards. + diff --git a/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md b/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md new file mode 100644 index 00000000000..b50b0589b7e --- /dev/null +++ b/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md @@ -0,0 +1,31 @@ +# 040 — Telemetry guide and configuration reference + +Stack layer 5. Lands before the dashboard (050) because the dashboard links to this guide and the +docs build rejects links to missing pages. + +## Public docs + +- `docs-site/src/content/docs/guides/telemetry.md` (English source): what is collected, what is never + collected, default-on with "nothing is sent until a person has seen the notice", agent-driven starts + never enable it, how to turn it off (GUI popup, Proxy details toggle, `ocx telemetry disable`, + `OCX_TELEMETRY=0`, `DO_NOT_TRACK=1`), how to see the exact payload (`ocx telemetry status --json`, + "What is sent"), storage and retention (daily aggregates, install hash rotates daily, no IPs), and + the collector source location. No link to `/dashboard` yet (050 adds it). +- `docs-site/astro.config.mjs`: add `guides/telemetry` to the Guides sidebar. +- `docs-site/src/content/docs/reference/configuration/server.md`: `telemetry` config block and the + two environment variables. +- `guides/web-dashboard.md`: one line pointing to the Proxy details telemetry toggle. +- Translated locales follow the docs-site locale rule (translated or absent with English fallback); + nothing may contradict the English page. + +## Maintainer docs + +`structure/telemetry.md` (from 010) gains the collector location (`services/telemetry-collector/`, +now present) and the guide path. + +## Verification + +docs-site build (`cd docs-site && bun install --frozen-lockfile && bun run build`), +`tests/ci-workflows/docs-link-targets.test.ts` and the other `docs-*` claim guards, +`bun run structure:check`, `bun run privacy:scan`. + diff --git a/devlog/_plan/261107_telemetry_dashboard/050_dashboard.md b/devlog/_plan/261107_telemetry_dashboard/050_dashboard.md new file mode 100644 index 00000000000..41609671e36 --- /dev/null +++ b/devlog/_plan/261107_telemetry_dashboard/050_dashboard.md @@ -0,0 +1,43 @@ +# 050 — opencodex.me/dashboard + +Stack layer 6. Reads `GET https://telemetry.opencodex.me/v1/stats` (`TelemetryStatsV1`, 030) from the browser and links the telemetry guide from 040. + +## Page + +`docs-site/src/content/docs/dashboard.mdx` with `template: splash`, mounting +``. A content page (not `src/pages/`) gets Starlight's locale fallback routes, +so the header's language links (`/ko/dashboard/` ...) resolve and `internal-links.mjs` and +`tests/ci-workflows/docs-link-targets.test.ts` pass unchanged. + +## Sections (adapted from tokscale, MIT, credited in the footer) + +1. Header: title, range pills (7d / 30d / 90d / 365d), last-updated time, privacy callout linking the + telemetry guide and the opt-out variables. +2. KPI strip (tokscale `ProfileOverview`): active installs, requests, tokens, active hours — + `tabular-nums`, compact K/M/B with exact values in `title`. +3. Token mix bar (tokscale `TokenBreakdown`): input / cache read / cache write / output / reasoning — disjoint buckets from the payload contract, so the segments sum to the total. +4. Activity: a 52-week heatmap of daily active installs from the always-365-day `heatmap` series (tokscale `ProfileContributionGraph`, 5 grades) + and an ECharts stacked daily chart from `dailyByProvider` (tokens or requests by provider kind). ECharts is already a + docs-site dependency (`FrontierBoards.astro`) and re-themes on `data-theme` changes. +5. Provider-kind and model-family share tables (tokscale `ProfileModels` colour dots). +6. Platform, version and install-channel distribution, new installs per version/channel; reliability (outcome ratio and top error codes). +7. Downloads: npm weekly/total downloads (`api.npmjs.org/downloads`) and GitHub release asset download + counts (`api.github.com/repos///releases`), fetched client-side with graceful fallback. + +Loading, empty ("collector not deployed yet") and error states are explicit; no layout jumps. + +## Files + +`docs-site/src/components/TelemetryDashboard.astro` (shell + client script), +`docs-site/src/components/telemetry-dashboard/*.ts` (pure formatting / series helpers testable from +`tests/`), `docs-site/src/data/telemetry-dashboard-i18n.ts` (copy for the 8 site locales), and the localized `/dashboard/` link in both the desktop nav and the splash +mobile menu of `docs-site/src/components/Header.astro`, plus a link from `guides/telemetry.md`. + +## Verification + +- `tests/ci-workflows/docs-telemetry-dashboard.test.ts`: the route exists, helpers format and bucket a + fixture stats response correctly, empty/error states render, no hard-coded secrets. +- `cd docs-site && bun install --frozen-lockfile && bun run build`. +- Browser check of the built page against a fixture stats JSON (screenshot for the PR). + + From 3e4597d1ba3c85d4049d6c01e3b3c15036d72b8f Mon Sep 17 00:00:00 2001 From: JUN Date: Sun, 8 Nov 2026 02:58:44 +0900 Subject: [PATCH 2/4] docs(devlog): record PR review round 1 decisions for the telemetry stack --- .../261107_telemetry_dashboard/000_plan.md | 10 ++- .../010_client_core.md | 2 +- .../060_review_round.md | 76 +++++++++++++++++++ 3 files changed, 85 insertions(+), 3 deletions(-) create mode 100644 devlog/_plan/261107_telemetry_dashboard/060_review_round.md diff --git a/devlog/_plan/261107_telemetry_dashboard/000_plan.md b/devlog/_plan/261107_telemetry_dashboard/000_plan.md index e694f22d51a..317491c152e 100644 --- a/devlog/_plan/261107_telemetry_dashboard/000_plan.md +++ b/devlog/_plan/261107_telemetry_dashboard/000_plan.md @@ -1,8 +1,8 @@ -# Opt-in telemetry and the opencodex.me usage dashboard +# Default-on (opt-out) telemetry and the opencodex.me usage dashboard ## Objective and scope -Add anonymous, opt-in product telemetry to the opencodex proxy and a public, project-wide +Add anonymous, default-on (opt-out) product telemetry to the opencodex proxy and a public, project-wide analytics page at `https://opencodex.me/dashboard`. The proxy reports install/version/platform pings, active time, request and token totals, and error counts; a collector stores daily aggregates; the dashboard renders installs, active time, usage, errors and downloads. @@ -91,3 +91,9 @@ Same reviewer: PASS, 0 blockers; all 9 blockers and 6 majors verified resolved. collector `tsconfig.json` overrides `rootDir`/`declaration` (030), and the new capability objects in `src/cli/capabilities-observe-system.ts` stay plain literals because `tests/helpers/cli-capability-data.ts` rejects `/` tokens and host identifiers there (010). + +## PR review round 1 + +Reviewer feedback on the open stack is recorded, with owner decisions D1–D20, in +[060_review_round.md](060_review_round.md). Those decisions supersede conflicting text in 000–050, including +the public privacy statement change that must land with the first layer able to send. diff --git a/devlog/_plan/261107_telemetry_dashboard/010_client_core.md b/devlog/_plan/261107_telemetry_dashboard/010_client_core.md index 62a3cb13885..d6d39999357 100644 --- a/devlog/_plan/261107_telemetry_dashboard/010_client_core.md +++ b/devlog/_plan/261107_telemetry_dashboard/010_client_core.md @@ -21,7 +21,7 @@ client cannot emit a key or value the collector rejects and the two cannot drift - `MODEL_FAMILIES` (closed prefix buckets): gpt-6, gpt-5, gpt-4, o-series, codex, claude-opus, claude-sonnet, claude-haiku, gemini-3, gemini-2, grok, deepseek, qwen, kimi, glm, minimax, mistral, llama, other. -- `version` must match `^\d+\.\d+\.\d+(-preview\.\d+)?$` (no build metadata). +- `version` must match `^\d+\.\d+\.\d+(-preview\.\d{8}(\.\d+)?)?$` (no build metadata). ## State model diff --git a/devlog/_plan/261107_telemetry_dashboard/060_review_round.md b/devlog/_plan/261107_telemetry_dashboard/060_review_round.md new file mode 100644 index 00000000000..eceb29c3fc1 --- /dev/null +++ b/devlog/_plan/261107_telemetry_dashboard/060_review_round.md @@ -0,0 +1,76 @@ +# 060 — PR review round 1 + +Review feedback on the stack (#8293, #8300, #8303, #8304, #8305, #8308) from Ingwannu, CodeRabbit and the +Codex reviewer. These decisions supersede any conflicting text in 000–050. Each lands on the layer that +owns the code, and each layer's PR description records it. + +## Why the privacy statement changes (blocking) + +`README.md` (and its seven translations under `readme/`) and `docs-site/src/content/docs/reference/code-signing.md` +carry the SignPath Foundation sentence "This program will not transfer any information to other networked +systems unless specifically requested…". With default-on telemetry that sentence would be false. SignPath's +conditions (https://signpath.org/terms.html) allow software that sends data to systems the user did not +specify only if the behaviour is described in a privacy policy, shown during installation, and can be +disabled at installation. The replacement statement (D12) says exactly that, lands on layer 2 (the first +layer that can send), and layer 5 links it to the telemetry guide. The interactive install-time prompt and +the first-run dashboard popup are the "shown during installation, can be disabled" points. + +## Decisions + +D1 Consent-day cutoff: only UTC days strictly AFTER the UTC day of (decidedAt ?? noticeShownAt) are ever sent. +D2 Env opt-out: whenever OCX_TELEMETRY or DO_NOT_TRACK opt-out is observed (proxy startup in background-lifecycle even though the scheduler is not started, and on every tick), persist ineligibleThroughDay = today's UTC day in telemetry-state.json (atomic no-follow write) and delete the install id; only days strictly after ineligibleThroughDay are sent. Tests: consented user runs with OCX_TELEMETRY=0 for days, removes it, those days are never sent and a fresh id is minted. +D3 Version regex everywhere (contract): ^\d+\.\d+\.\d+(-preview\.\d{8}(\.\d+)?)?$ ; test vectors 2.40.1, 2.6.15-preview.20260701, 2.6.15-preview.20260701.1, 2.7.9-preview.20260712.2; reject build metadata. +D4 ocx telemetry enable: requires an interactive human terminal (isatty stdin AND stdout, not OCX_SERVICE, not isAgentDriven); prints the disclosure block, then asks a confirm (default yes) and records accepted; otherwise exits non-zero with agent_consent_required / tty_required pointing to the dashboard. disable always allowed. +D5 Disclosure block printed before every CLI Y/n prompt (ocx start/init/service install and enable): what is sent, what is never sent, that aggregates are public at opencodex.me/dashboard, how to see the exact payload (ocx telemetry status --json), how to turn it off (ocx telemetry disable, OCX_TELEMETRY=0, DO_NOT_TRACK=1, Proxy details toggle). A CLI test pins the text. +D6 ocx config import: for EVERY caller, preserve the persisted telemetry block unchanged (ignore imported telemetry). Tests: undecided and declined installs. +D7 Re-read persisted consent and env immediately before transmission; if not effective, abort without sending. Ticks carry an AbortSignal cancelled by stop and by consent changes (nudge). +D8 Aggregation accumulates counters inside onEntry for the target day only; never retains ledger rows. +D9 CLI enable/disable: if mutatePersistedConfig returns unavailable (or throws), exit non-zero with an actionable error, print no success line, and do not delete the id unless the decline persisted. +D10 GET /api/telemetry preview uses the same day the scheduler would send next (oldest eligible unsent day). If none is eligible, preview the latest completed day and return previewEligible:false (field added); never imply it will be sent. +D11 Client honors HTTP 429 Retry-After (and 503): persist retryAfterUntil in telemetry-state.json, skip ticks until then; spreads sends with jitter. +D12 Privacy statement (README.md, all 7 readme/README.*.md, docs-site/src/content/docs/reference/code-signing.md): replace the SignPath "will not transfer any information..." sentence with an accurate statement: opencodex sends one anonymous daily usage summary (version, platform, counts; never prompts, outputs, keys, accounts, hosts or paths) to telemetry.opencodex.me only after the person installing or operating it has seen the notice (dashboard popup or interactive CLI prompt) and has not declined; it can be turned off at install time in that prompt and anytime with ocx telemetry disable, OCX_TELEMETRY=0, DO_NOT_TRACK=1 or Proxy details; project-wide aggregates are public; otherwise data goes only where the user configures. Keep the README Privacy line in English in every translation (as today) unless readme/i18n-manifest.json rules require otherwise. code-signing.md: also fix the "carry none of your ... usage data" claim. +D13 GUI popup dismissal (Escape/backdrop/close) records NOTHING; only Continue (PUT /api/telemetry) records decision and notice; the popup returns on the next app start while undecided. Remove POST /api/telemetry/notice entirely (server route, registry entry, capability exemptions, GUI call, tests, docs). +D14 Stats semantics: activeInstalls in totals and installs in breakdowns are INSTALL-DAYS (sum of daily reporting installs); add totals.avgDailyActiveInstalls (install-days / days in range, rounded to 1 decimal). Dashboard shows "Avg daily active installs" as the KPI with install-days in the tooltip. +D15 Small-cohort protection (collector): add an installs column (count of contributing reports) to daily_models and daily_errors (and dailyByProvider derivation); in every breakdown (providers, models, versions, platforms, channels, errors, dailyByProvider per day) fold rows with fewer than 10 install-days in the selected range into "other"; drop "other" if it is itself below 10; totals/daily/heatmap unchanged. Document the residual subtraction risk. +D16 Rate limit: 120 reports/hour per salted IP hash (shared IP / CGNAT capacity); 429 with Retry-After. +D17 Dashboard labels all figures "self-reported by opencodex installs"; docs note reports are unauthenticated and can be forged within rate limits. +D18 Collector-first rollout: any roster/contract change must be deployed to the collector before a client release emits it (README of collector + structure doc). +D19 Guide discloses the salted hourly IP hash for rate limiting (pruned after 2 hours). +D20 Dashboard: GitHub release download totals follow pagination (per_page=100, follow Link rel=next up to 10 pages, mark "partial" if capped); applying an empty snapshot clears the heatmap. + + +## Finding → decision map + +| Finding | Source | Decision | Layer | +| --- | --- | --- | --- | +| Published privacy statement becomes false | Ingwannu (blocking) | D12 | 2, 5 | +| Env opt-out days are backfilled | Ingwannu (blocking) | D2 | 2 | +| Consent day usage before the notice | Ingwannu | D1 | 2 | +| Preview tags rejected by the version regex | Ingwannu | D3 | 2 (contract) | +| activeInstalls over a range is a sum | Ingwannu | D14 | 4, 6 | +| CLI prompt is not a disclosure | Ingwannu | D5 | 2 | +| Opt-in wording | CodeRabbit | title/objective now say default-on (opt-out) | 1 | +| Roster changes break older collectors | CodeRabbit | D18 | 4, 5 | +| `ocx telemetry enable` without a TTY | CodeRabbit | D4 | 2 | +| Imported consent can backdate sending | CodeRabbit | D6 | 2 | +| Identity kept under an env opt-out | CodeRabbit | D2 | 2 | +| Preview day differs from the next send | CodeRabbit | D10 | 2, 3 | +| Automated dismissal records the notice | CodeRabbit | D13 | 2, 3 | +| Atomic dedup and rollup | CodeRabbit | already in #8304 (single conditional batch) | 4 | +| Forged reports | CodeRabbit | D17 (labelled self-reported; rate limit) | 4, 5, 6 | +| More than 64 model rows | CodeRabbit | already folded into `other`/`other` in #8300 | 2 | +| Shared-IP capacity and Retry-After | CodeRabbit | D11, D16 | 2, 4 | +| Small-cohort exposure | CodeRabbit | D15 | 4, 6 | +| IP-hash disclosure | CodeRabbit | D19 | 5 | +| Consent re-check before sending | Codex P1 | D7 | 2 | +| Aggregation keeps the whole ledger | Codex P2 | D8 | 2 | +| CLI reports unpersisted changes | Codex P2 | D9 | 2 | +| Public docs for the controls | Codex P1 (#8300, #8303) | already in #8305 | 5 | +| Release pagination, heatmap reset | Codex P2 (#8308) | D20 | 6 | + +## Residual risks + +- Reports are unauthenticated. Rate limits and the self-reported label bound, but do not prevent, forged + figures. +- Folding small rows into `other` limits single-install exposure; a suppressed value can still be + approximated by subtracting published rows from published totals when only one row is folded. From b6c9314fa227e4774e7502ef725a0c8eae0c04c3 Mon Sep 17 00:00:00 2001 From: JUN Date: Sun, 8 Nov 2026 18:27:50 +0900 Subject: [PATCH 3/4] docs(devlog): plan the env opt-out marker, privacy statement files, and the open SignPath condition - 000: the no-backfill rule covers env opt-out days and the disclosure day; the privacy statement changes in layer 2. - 040: lists every file that carries the published privacy statement (README, seven translations, both code-signing pages, SPONSORS.md) and says they land in layer 2. - 060: D2 adds the envOptOutActiveSince marker for a proxy that runs opted out for days; records SignPath's install-time condition verbatim as an open owner decision instead of assuming first-run disclosure satisfies it. --- .../261107_telemetry_dashboard/000_plan.md | 7 +++++- .../040_docs_privacy.md | 18 +++++++++++++++ .../060_review_round.md | 23 +++++++++++++++---- 3 files changed, 43 insertions(+), 5 deletions(-) diff --git a/devlog/_plan/261107_telemetry_dashboard/000_plan.md b/devlog/_plan/261107_telemetry_dashboard/000_plan.md index 317491c152e..63da28bb8c8 100644 --- a/devlog/_plan/261107_telemetry_dashboard/000_plan.md +++ b/devlog/_plan/261107_telemetry_dashboard/000_plan.md @@ -41,7 +41,12 @@ footer and in the PR descriptions. - The collector stores no IP address and no raw install id: reports carry a per-day install hash, rate limiting uses a salted hourly IP hash pruned after two hours, and only the first report per install and day counts. -- No backfill: days before the disclosure are never sent; at most one day is sent per scheduler tick. +- No backfill: only UTC days strictly after the day of the disclosure or decision are sent, days + covered by an `OCX_TELEMETRY=0` / `DO_NOT_TRACK=1` opt-out are never sent (060 D2), and at most one + day is sent per scheduler tick. +- The published privacy statement (README and its seven translations, the docs-site code-signing + page in English and Turkish, `SPONSORS.md`) describes default-on telemetry and every way to turn it + off. It changes in layer 2, the first layer that can send, never later (060 D12). - The collector URL is a constant; it is not configurable from `config.json`. - Telemetry code stays off the core request path: `src/router.ts`, `src/server/lifecycle.ts` and `src/server/responses/core.ts` must not import `src/telemetry/`. diff --git a/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md b/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md index b50b0589b7e..8f399f4ff06 100644 --- a/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md +++ b/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md @@ -3,6 +3,24 @@ Stack layer 5. Lands before the dashboard (050) because the dashboard links to this guide and the docs build rejects links to missing pages. +## Privacy statement (lands in layer 2, not here) + +The project's published privacy statement must stop saying that nothing is sent before the first +layer that can send merges, so these files change in layer 2 (060 D12) and layer 5 only links them to +the guide: + +- `README.md` (Code signing policy → Privacy) and the same line in `readme/README.fr.md`, + `README.ja.md`, `README.ko.md`, `README.ru.md`, `README.tr.md`, `README.zh-CN.md` and + `README.zh-TW.md`, with `readme/i18n-manifest.json` refreshed. +- `docs-site/src/content/docs/reference/code-signing.md` and its Turkish copy + `docs-site/src/content/docs/tr/reference/code-signing.md` (the Privacy section, including the + "carry none of your … usage data" sentence). +- `SPONSORS.md` ("opencodex does not collect any" telemetry). + +The statement replaces SignPath's "will not transfer any information …" sentence, which SignPath +allows only as an alternative to linking a privacy policy. Whether first-run disclosure satisfies +SignPath's install-time conditions is an open owner decision (060, "SignPath conditions"). + ## Public docs - `docs-site/src/content/docs/guides/telemetry.md` (English source): what is collected, what is never diff --git a/devlog/_plan/261107_telemetry_dashboard/060_review_round.md b/devlog/_plan/261107_telemetry_dashboard/060_review_round.md index eceb29c3fc1..b85b03b89a9 100644 --- a/devlog/_plan/261107_telemetry_dashboard/060_review_round.md +++ b/devlog/_plan/261107_telemetry_dashboard/060_review_round.md @@ -12,13 +12,28 @@ systems unless specifically requested…". With default-on telemetry that senten conditions (https://signpath.org/terms.html) allow software that sends data to systems the user did not specify only if the behaviour is described in a privacy policy, shown during installation, and can be disabled at installation. The replacement statement (D12) says exactly that, lands on layer 2 (the first -layer that can send), and layer 5 links it to the telemetry guide. The interactive install-time prompt and -the first-run dashboard popup are the "shown during installation, can be disabled" points. +layer that can send), and layer 5 links it to the telemetry guide. + +## SignPath conditions (open owner decision) + +SignPath Foundation's conditions (https://signpath.org/terms, read 2026-11-08), "Respect user privacy and +security": "Software that collects user data and transfers it to systems not specified by the user must +a) describe this behavior in a privacy policy, b) display this policy during installation and c) include +installation options to disable these functions." Its website condition allows either a privacy-policy link +or the "will not transfer" sentence. + +(a) is met by the D12 statement. (b) and (c) are not clearly met. SignPath signs the standalone `ocx.exe`, +which has no installer, and the MSI, whose installer shows no policy and has no disable option. The CLI +prompt and the dashboard popup appear on first run, after installation. The release workflow still uses the +`test-signing` policy, so no production-signed build exists yet. Round 1 treated first-run disclosure as +"during installation"; that reading is not confirmed. The owner decides whether to add an MSI dialog +(policy and a disable checkbox) before switching to `release-signing`, to ask SignPath first, or to +accept first-run disclosure. No layer that can send merges before that decision. ## Decisions D1 Consent-day cutoff: only UTC days strictly AFTER the UTC day of (decidedAt ?? noticeShownAt) are ever sent. -D2 Env opt-out: whenever OCX_TELEMETRY or DO_NOT_TRACK opt-out is observed (proxy startup in background-lifecycle even though the scheduler is not started, and on every tick), persist ineligibleThroughDay = today's UTC day in telemetry-state.json (atomic no-follow write) and delete the install id; only days strictly after ineligibleThroughDay are sent. Tests: consented user runs with OCX_TELEMETRY=0 for days, removes it, those days are never sent and a fresh id is minted. +D2 Env opt-out: whenever OCX_TELEMETRY or DO_NOT_TRACK opt-out is observed (proxy startup in background-lifecycle even though the scheduler is not started, and on every tick), persist ineligibleThroughDay = today's UTC day and an envOptOutActiveSince marker in telemetry-state.json (atomic no-follow write) and delete the install id; only days strictly after ineligibleThroughDay are sent. A proxy started under the opt-out reads the env once and runs no ticks, so the marker covers the whole time it runs: while it is set nothing is eligible, and the period closes (ineligible through the closing day, marker cleared) on that proxy's clean stop or on the first tick without the opt-out. Tests: consented user runs with OCX_TELEMETRY=0 for days, removes it, those days are never sent and a fresh id is minted; a proxy that runs opted out from Monday to Thursday, ended by a crash and by a clean stop, never lets those days be sent (failing-first in layer 2). D3 Version regex everywhere (contract): ^\d+\.\d+\.\d+(-preview\.\d{8}(\.\d+)?)?$ ; test vectors 2.40.1, 2.6.15-preview.20260701, 2.6.15-preview.20260701.1, 2.7.9-preview.20260712.2; reject build metadata. D4 ocx telemetry enable: requires an interactive human terminal (isatty stdin AND stdout, not OCX_SERVICE, not isAgentDriven); prints the disclosure block, then asks a confirm (default yes) and records accepted; otherwise exits non-zero with agent_consent_required / tty_required pointing to the dashboard. disable always allowed. D5 Disclosure block printed before every CLI Y/n prompt (ocx start/init/service install and enable): what is sent, what is never sent, that aggregates are public at opencodex.me/dashboard, how to see the exact payload (ocx telemetry status --json), how to turn it off (ocx telemetry disable, OCX_TELEMETRY=0, DO_NOT_TRACK=1, Proxy details toggle). A CLI test pins the text. @@ -28,7 +43,7 @@ D8 Aggregation accumulates counters inside onEntry for the target day only; neve D9 CLI enable/disable: if mutatePersistedConfig returns unavailable (or throws), exit non-zero with an actionable error, print no success line, and do not delete the id unless the decline persisted. D10 GET /api/telemetry preview uses the same day the scheduler would send next (oldest eligible unsent day). If none is eligible, preview the latest completed day and return previewEligible:false (field added); never imply it will be sent. D11 Client honors HTTP 429 Retry-After (and 503): persist retryAfterUntil in telemetry-state.json, skip ticks until then; spreads sends with jitter. -D12 Privacy statement (README.md, all 7 readme/README.*.md, docs-site/src/content/docs/reference/code-signing.md): replace the SignPath "will not transfer any information..." sentence with an accurate statement: opencodex sends one anonymous daily usage summary (version, platform, counts; never prompts, outputs, keys, accounts, hosts or paths) to telemetry.opencodex.me only after the person installing or operating it has seen the notice (dashboard popup or interactive CLI prompt) and has not declined; it can be turned off at install time in that prompt and anytime with ocx telemetry disable, OCX_TELEMETRY=0, DO_NOT_TRACK=1 or Proxy details; project-wide aggregates are public; otherwise data goes only where the user configures. Keep the README Privacy line in English in every translation (as today) unless readme/i18n-manifest.json rules require otherwise. code-signing.md: also fix the "carry none of your ... usage data" claim. +D12 Privacy statement (README.md, all 7 readme/README.*.md, docs-site/src/content/docs/reference/code-signing.md and its tr copy, SPONSORS.md): replace the SignPath "will not transfer any information..." sentence with an accurate statement: opencodex sends one anonymous daily usage summary (version, platform, counts; never prompts, outputs, keys, accounts, hosts or paths) to telemetry.opencodex.me only after the person installing or operating it has seen the notice (dashboard popup or interactive CLI prompt) and has not declined; it can be turned off at install time in that prompt and anytime with ocx telemetry disable, OCX_TELEMETRY=0, DO_NOT_TRACK=1 or Proxy details; project-wide aggregates are public; otherwise data goes only where the user configures. Keep the README Privacy line in English in every translation (as today) unless readme/i18n-manifest.json rules require otherwise. code-signing.md: also fix the "carry none of your ... usage data" claim. D13 GUI popup dismissal (Escape/backdrop/close) records NOTHING; only Continue (PUT /api/telemetry) records decision and notice; the popup returns on the next app start while undecided. Remove POST /api/telemetry/notice entirely (server route, registry entry, capability exemptions, GUI call, tests, docs). D14 Stats semantics: activeInstalls in totals and installs in breakdowns are INSTALL-DAYS (sum of daily reporting installs); add totals.avgDailyActiveInstalls (install-days / days in range, rounded to 1 decimal). Dashboard shows "Avg daily active installs" as the KPI with install-days in the tooltip. D15 Small-cohort protection (collector): add an installs column (count of contributing reports) to daily_models and daily_errors (and dailyByProvider derivation); in every breakdown (providers, models, versions, platforms, channels, errors, dailyByProvider per day) fold rows with fewer than 10 install-days in the selected range into "other"; drop "other" if it is itself below 10; totals/daily/heatmap unchanged. Document the residual subtraction risk. From 0e3c03a041ae77eb6131e836df46952f9bdf8835 Mon Sep 17 00:00:00 2001 From: JUN Date: Sun, 8 Nov 2026 19:04:37 +0900 Subject: [PATCH 4/4] docs(devlog): record the owner's SignPath decision (D21) and the Windows installer layer --- .../261107_telemetry_dashboard/000_plan.md | 1 + .../040_docs_privacy.md | 5 +++-- .../060_review_round.md | 22 +++++++++++++++---- 3 files changed, 22 insertions(+), 6 deletions(-) diff --git a/devlog/_plan/261107_telemetry_dashboard/000_plan.md b/devlog/_plan/261107_telemetry_dashboard/000_plan.md index 63da28bb8c8..36750627e48 100644 --- a/devlog/_plan/261107_telemetry_dashboard/000_plan.md +++ b/devlog/_plan/261107_telemetry_dashboard/000_plan.md @@ -58,6 +58,7 @@ footer and in the PR descriptions. | --- | --- | --- | | 1 | this unit | Docs-only roadmap. | | 2 | [010_client_core.md](010_client_core.md) | Config field, consent state, install id, aggregate builder, payload v1, sender, management API, `ocx telemetry`, `structure/telemetry.md`. | +| 2b | [060_review_round.md](060_review_round.md) D21 | Windows installer page (disclosure, policy link, opt-out checkbox) and the desktop app carrying an installer opt-out into the config. | | 3 | [020_gui_consent.md](020_gui_consent.md) | First-run card, post-update popup, Advanced toggle, preview dialog, all locales. | | 4 | [030_collector.md](030_collector.md) | Cloudflare Worker + D1: ingest validation, limits, daily rollups, public stats API. | | 5 | [040_docs_privacy.md](040_docs_privacy.md) | Telemetry guide, sidebar, configuration reference. | diff --git a/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md b/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md index 8f399f4ff06..7d944d29330 100644 --- a/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md +++ b/devlog/_plan/261107_telemetry_dashboard/040_docs_privacy.md @@ -18,8 +18,9 @@ the guide: - `SPONSORS.md` ("opencodex does not collect any" telemetry). The statement replaces SignPath's "will not transfer any information …" sentence, which SignPath -allows only as an alternative to linking a privacy policy. Whether first-run disclosure satisfies -SignPath's install-time conditions is an open owner decision (060, "SignPath conditions"). +allows only as an alternative to linking a privacy policy. How the stack meets SignPath's +install-time conditions is recorded in 060 ("SignPath conditions", D21); layer 2b adds the Windows +installer option to these files. ## Public docs diff --git a/devlog/_plan/261107_telemetry_dashboard/060_review_round.md b/devlog/_plan/261107_telemetry_dashboard/060_review_round.md index b85b03b89a9..1567b0c898d 100644 --- a/devlog/_plan/261107_telemetry_dashboard/060_review_round.md +++ b/devlog/_plan/261107_telemetry_dashboard/060_review_round.md @@ -14,7 +14,7 @@ specify only if the behaviour is described in a privacy policy, shown during ins disabled at installation. The replacement statement (D12) says exactly that, lands on layer 2 (the first layer that can send), and layer 5 links it to the telemetry guide. -## SignPath conditions (open owner decision) +## SignPath conditions SignPath Foundation's conditions (https://signpath.org/terms, read 2026-11-08), "Respect user privacy and security": "Software that collects user data and transfers it to systems not specified by the user must @@ -26,9 +26,23 @@ or the "will not transfer" sentence. which has no installer, and the MSI, whose installer shows no policy and has no disable option. The CLI prompt and the dashboard popup appear on first run, after installation. The release workflow still uses the `test-signing` policy, so no production-signed build exists yet. Round 1 treated first-run disclosure as -"during installation"; that reading is not confirmed. The owner decides whether to add an MSI dialog -(policy and a disable checkbox) before switching to `release-signing`, to ask SignPath first, or to -accept first-run disclosure. No layer that can send merges before that decision. +"during installation"; that reading is not confirmed. + +Owner decision (2026-11-08), D21: telemetry stays default-on and the stack satisfies all three: + +- (a) The D12 privacy statement (README and translations, code-signing page in English and Turkish, + `SPONSORS.md`) and the layer 5 guide describe exactly what is sent and how to opt out. +- (b, c) MSI: a new layer between 2 and 3 adds an installer page generated from + `src/telemetry/disclosure.ts`, the same text the CLI prompt prints. It links the policy and has a + "Send anonymous usage statistics" checkbox, checked by default. Unchecked, the choice is recorded in + HKCU and the desktop app persists `telemetry.decision = "declined"` into the user's config before it + resolves, attaches to or starts any proxy. Tests pin the dialog text source and the persisted opt-out. +- (b, c) Standalone `ocx.exe` and npm installs have no installer: nothing is sent before + `noticeShownAt`, and the notice (the CLI prompt or the GUI popup) shows the policy summary and offers + disabling at that moment. A test proves zero sends before `noticeShownAt`. + +Layers that can send (2 and above) merge only after the installer layer and the zero-send test are in +the stack and have passed their reviews and CI. ## Decisions