Skip to content

feat(mlflow): one-switch tracing for Claude and Codex#36

Open
mpkrass7 wants to merge 1 commit into
mainfrom
feat/mlflow-tracing-codex-gemini
Open

feat(mlflow): one-switch tracing for Claude and Codex#36
mpkrass7 wants to merge 1 commit into
mainfrom
feat/mlflow-tracing-codex-gemini

Conversation

@mpkrass7
Copy link
Copy Markdown
Collaborator

@mpkrass7 mpkrass7 commented May 11, 2026

What does this PR do?

Addresses #35

Adds MLFLOW_TRACING_ENABLED: a single env var that turns on MLflow tracing for all three CLIs into the same /Users/{owner}/{app} experiment. It's in the .yaml file now

  • setup_mlflow.py: gates Claude's Stop hook on the new var (default off)
  • setup_codex.py: installs @mlflow/codex, wires the notify hook, writes mlflow-tracing.json + MLflow env vars when enabled
  • utils.py: resolve_mlflow_experiment_id() helper (codex needs the experiment ID, not name)
  • README + app.yaml.template: document the new switch

Co-authored-by: Isaac

Testing

Please test your changes on dogfood before merging.

  • Deployed and tested on dogfood
  • Added a screenshot to this PR

@mpkrass7 mpkrass7 changed the title feat(mlflow): one-switch tracing for Claude, Codex, and Gemini feat(mlflow): one-switch tracing for Claude and Codex May 11, 2026
@mpkrass7
Copy link
Copy Markdown
Collaborator Author

mpkrass7 commented May 11, 2026

Gemini cli looks broken for auto tracing FYI. Feel free to verify for yourselves by following the directions here and watching your logs not update. I flagged in the mlflow channel

https://mlflow.org/docs/latest/genai/tracing/integrations/listing/gemini_cli/#configuration-examples

Comment thread setup_codex.py Outdated
@mpkrass7
Copy link
Copy Markdown
Collaborator Author

image

Adds MLFLOW_TRACING_ENABLED — a single env var that turns on MLflow
tracing for all three CLIs into the same /Users/{owner}/{app} experiment.

- setup_mlflow.py: gates Claude's Stop hook on the new var (default off);
  trims docstring and drops redundant tracing_value
- setup_codex.py: installs @mlflow/codex, wires the notify hook, writes
  mlflow-tracing.json + MLflow env vars when enabled
- setup_gemini.py: appends Gemini's native OTLP env vars pointed at the
  Databricks OTLP collector (/api/2.0/otel) when enabled, then reverted
  because Databricks OTLP doesn't accept JSON
- utils.py: resolve_mlflow_experiment_id() helper using databricks-sdk
  (codex needs the experiment ID, not name)
- cli_auth.py: rotates the bearer token inside Gemini's OTEL headers so
  OTLP ingest survives PAT rotation
- README + app.yaml.template: document the new switch

Co-authored-by: Isaac
@mpkrass7 mpkrass7 force-pushed the feat/mlflow-tracing-codex-gemini branch from 6ac9655 to 3b8e627 Compare May 11, 2026 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant