Repository navigation
feat(traces): agent trace producers in Phoenix — Claude Code, dsh, LiteLLM, OpenClaw - #16
Merged
Merged
Conversation
…teLLM, OpenClaw Phoenix-only mapping processors in traces/phoenix (Tempo/ClickHouse unchanged): - transform/phoenix_claude: Claude Code 2.1.x beta spans (claude_code.interaction/llm_request/ tool/hook) -> OpenInference kind, input/output, llm.model_name, token counts incl. cache. - transform/phoenix_openclaw: agent id + channel from the system prompt's Runtime line; drop the duplicate openclaw.content.* copy (~33% smaller model-call spans in Phoenix). - transform/phoenix_litellm: arize_phoenix callback spans -> one LLM span per call, key alias as user.id, cached tokens; drop the proxy span's duplicate content and token counts. Tempo span-metrics gain llm.model_name + gen_ai.request.model dimensions for the new "Agent Traces" dashboard (uid agent-traces). smoke.sh fails when Claude Code llm_request spans stop mapping to LLM (beta span names can change on upgrade). README documents producers, projects, nesting and kill switches. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AnNpvt9FGCYPErJvHzyji5
Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AnNpvt9FGCYPErJvHzyji5
…+ LiteLLM) Owner privacy decision: Phoenix is LAN/no-auth. Spans of OpenClaw agent mira, and LiteLLM calls whose system message carries 'Runtime: agent=mira', keep timing/tokens/model but lose messages (tag privacy.content_dropped=mira). The raw_gen_ai_request reply copy is dropped for every call so a filtered request cannot leak through the sibling span. Tempo is unchanged. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AnNpvt9FGCYPErJvHzyji5
…ingress) otlp/twin: HTTP on container port 4320, NOT published on the host (only cloudflared on `obs` will reach it), guarded by bearertokenauth/twin (token file rendered from .env TWIN_OTEL_BEARER_TOKEN into secrets/twin_otel_token, mounted read-only). filter/twin drops any span whose service.name is not mairp-digital-twin; resource/twin tags ingress=cloudflare-tunnel. traces/twin -> Tempo + ClickHouse, traces/twin-phoenix -> Phoenix (project digital-twin via the Worker's own openinference.project.name; the Phoenix transforms are == nil guarded). Verified: no/wrong bearer 401, right bearer 200, foreign service dropped, host :4320 closed. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AnNpvt9FGCYPErJvHzyji5
Dashboard uid digital-twin-traces: requests/hour by outcome, p50/p95 total vs Compass latency, guard blocks by type (turnstile, rate limit, daily cap, output guard, injection hint), top countries, last 50 question->answer turns and Compass calls with tokens (Tempo tables). Tempo span-metrics gain twin.outcome, twin.guard.output.verdict, twin.guard.injection_scan.matched and cf.country (<= 800 req/day, low cardinality). README: digital twin row in the producer table. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AnNpvt9FGCYPErJvHzyji5
Remotely-managed Cloudflare Tunnel (published application otel-twin -> http://otel-collector:4320, the bearer-gated otlp/twin receiver), protected at the edge by a Cloudflare Access Service Auth policy. No host port; token in gitignored secrets/cloudflared_tunnel.env (TUNNEL_TOKEN=...). Verified: no Access headers -> 403, Access + wrong bearer -> 401, both -> 200 and in Tempo + Phoenix digital-twin; a foreign service.name is dropped. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AnNpvt9FGCYPErJvHzyji5
…ithout host secrets - dashboards: no LAN IP (Phoenix = host port 3006), twin service matched as *-digital-twin - collector filter/twin: IsMatch ^[a-z0-9]+-digital-twin$ (same effect for the Worker) - README: ~/ paths, no hostnames - cloudflared env_file is optional (required: false), so CI's clean checkout validates Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AnNpvt9FGCYPErJvHzyji5
Contributor
Merge Queue Status
This pull request spent 47 seconds in the queue, including 27 seconds running CI. Required conditions to merge
|
9 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phoenix-only mapping for the agent trace producers added on 2026-09-28 (plan:
/root/phoenix/prompts/agent-otel-traces-claude-dsh-openclaw.md). Tempo and ClickHouse get spans unchanged.transform/phoenix_claude: Claude Code 2.1.x beta spans → OpenInference (AGENT/LLM/TOOL, input/output, model, tokens incl. cache).transform/phoenix_openclaw: agent id + channel from the system prompt; drops the duplicateopenclaw.content.*copy (~33% smaller in Phoenix).transform/phoenix_litellm: arize_phoenix callback → one LLM span per call, key alias →user.id, cached tokens; drops the proxy span's duplicate content/tokens.llm.model_name+gen_ai.request.modeldimensions; new dashboardagent-traces.tests/smoke.sh: fails when Claude Codellm_requestspans stop mapping to LLM.Already live on the host (collector + Tempo restarted from this branch).
tests/validate.shpasses.https://claude.ai/code/session_01AnNpvt9FGCYPErJvHzyji5