Repository navigation
Description
Story
As a person investigating an expensive implementation, I want to connect an XMD prompt to its internal model calls and context-gathering activity, so I can tell whether tokens went to useful work, retries, compaction, subagents or repeated investigation.
For example, a planner prompt consumes much more than comparable prompts. Its diagnostic view shows the available model requests and tool activity, including repeated reads of the same returned content, while identifying work the agent does not expose.
Current gap
An outer-prompt total cannot explain the coding agent's internal execution. Claude and Codex provide native telemetry and session records, but their version-dependent formats and identifiers are not yet joined to XMD workflow attribution.
Required outcome
- Add explicitly supported native-data ingestion for the bundled Claude and Codex versions used by XMD. Select the smallest useful source for each; OpenTelemetry events and existing transcript/rollout files are candidates, not a requirement to build every parser.
- Join internal events to XMD prompts and sessions using available stable identities. Mark inferred or ambiguous joins and report unmatched records.
- Make available model-request usage, latency, retries, compaction, subagent activity and tool-result measurements queryable, with an explicit coverage matrix.
- Reconcile overlapping ACPX, native telemetry and transcript measurements. Multiple observations of one request do not add multiple copies of its spend.
- Deduplicate using source record/request identity and verified counter semantics. Equal token counts and nearby timestamps are insufficient.
- Where exposed, record a retrieval's requested file/range/symbol selection and the identity/size of the returned content. Different selections from one unchanged file are not automatically duplicate reads. Missing content identity remains unknown.
- Distinguish measured token usage from tool-output size estimates and context occupancy.
- Keep subscription-window measurements at their actual account/window scope unless the source proves finer attribution.
- Collection is optional and preserves the existing process/session topology. Raw API-body capture is a separate, explicit diagnostic choice and is not required for ordinary accounting.
Acceptance and evidence
- Supply versioned, documented ingestion fixtures and a bounded live verification for each supported agent source. Record the bundled versions and the fields that remain unavailable.
- A prompt with multiple model/tool steps produces an inspectable association to those steps. Include subagent or compaction coverage where supported and explicitly report any omission.
- Feeding overlapping ACPX/native/transcript data leaves canonical consumption unchanged. Re-importing a source is idempotent, while two distinct requests with identical counts both remain counted.
- Concurrent lanes and sessions sharing a runtime keep their associations. Demonstrate an ambiguous/missing-identity case that remains unresolved instead of being silently assigned.
- Reading two different ranges of a file is distinguished from reading identical returned content again; both are diagnostic observations rather than automatic declarations of wasted work.
- A representative expensive run exposes at least one evidence-backed investigation path, such as retries, history growth or repeated retrieval, with coverage limitations visible.
Design and relationships
Depends on #1159 and #1160. This adds diagnostic depth; basic cost-per-feature reporting need not wait for it.
- Claude Code monitoring
- Codex observability configuration
- Inspect live XMD execution through the Effection inspector #894 owns the Effection operation-tree inspector. Reuse useful correlation without duplicating that product.
Settle source selection and exporter lifecycle against the actual bundled agents before implementation. An OpenTelemetry Collector may be used for a supported signal, but a hosted observability platform, provider proxy, new agent framework or compulsory full prompt-body collection is outside this story.
Disposition — 2026-11-08
Closed as not planned at the user's direction. The current cost-tracking goal is per-message/turn token counts, completed entry/round summaries, and a running Sidekick total, tracked in #1201. This broader analytics story is not required for that delivery and is removed from the queue, with no automatic follow-on implementation. Closing this issue does not claim its requirements were implemented. The merged usage-capture foundation in #1166 remains in place.
Activity