Skip to content

Log AitoolsMcpEvent from aitools mcp and the serve proxy - #7554

Draft
simonfaltum wants to merge 1 commit into
mainfrom
aitools-mcp/10-telemetry
Draft

simonfaltum wants to merge 1 commit into
mainfrom
aitools-mcp/10-telemetry

Conversation

@simonfaltum

@simonfaltum simonfaltum commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

This pull request is part of the stack that adds the hidden databricks aitools mcp command group, on top of #7552. It adds typed AitoolsMcpEvent telemetry for the management commands and for serve's start and sign-in prompts. It depends on a telemetry schema change that lives outside this repository and is not merged yet, so this pull request stays a draft.

Changes

  • AitoolsMcpEvent, its server result and their enums in libs/telemetry/protos. The MCP error categories join AitoolsErrorCategory; their JSON strings are unchanged.
  • add logs ADD and SIGN_IN; login and logout log SIGN_IN and SIGN_OUT; remove logs REMOVE when it touched a server; serve logs PROXY_STARTED and SIGN_IN_PROMPTED. When add --login cannot build a server's sign-in URL, its SIGN_IN event is FAILED with sign-in state PENDING.
  • Each command uploads with the profile it bound, including one picked in the picker, and with credentials that never refresh a token. A run that sees a rejected credential still logs its event and uploads like any other.
  • remove uploads only when every server it removes is bound to one profile and config file, and that profile still matches them; otherwise it logs its event and uploads nothing. A server pinned to another config file uploads with that file, and a same-named profile in another file no longer stops the upload. An unpinned server uploads with the default config file, the one its binding is checked against.
  • serve binds the upload to its profile once startup succeeds and uploads PROXY_STARTED right then, while the token is known to work, so a killed or long-idle serve keeps it. SIGN_IN_PROMPTED uploads at exit, with the profile bound at start. At exit, serve checks the binding again, and skips the upload if the profile no longer names the host it bound.
  • libs/telemetry gets an upload slot (SetUploadConfig and UploadConfig) and SkipUpload, which no config overrides. Upload removes the events the service accepted, so a later upload doesn't resend them. install is unchanged.
  • cmd/root exports TelemetryExecutionContext, which its exit upload and serve's start upload share.
  • Events carry enums and counts only. A service outside system.ai is CUSTOM, and its name is not sent.
  • The upload's host metadata lookup logs failures at debug level, so a successful command prints no warning.
  • No changelog fragment: the command group is hidden, so nothing here is user-visible yet.

Why

The MCP commands and the proxy have no telemetry, so their use cannot be measured. Background auth resolution, as install uses, could send a refreshed token or another profile's credential, or upload under a profile the command did not bind.

Decisions to review

  • A run that saw a rejected credential uploads, with the bound profile's credential, which never refreshes. The upload presents the rejected token once more, to the bound host, and makes no token request. Cost: one more request with a token the service has already rejected, and the run's events are sent even though that token was refused.
  • A failed start uploads nothing. PROFILE_NOT_FOUND and PROFILE_HOST_CHANGED are never sent, since the token must not reach a host the record does not name, and AUTH_FAILED would repeat the rejected request. These failures are undercounted.
  • The upload reads the bound host's metadata, an unauthenticated request. remove may wait for the upload after printing its output.
  • serve waits for its PROXY_STARTED upload before it serves, bounded by the 3 s upload timeout. That upload's execution context has exit code 0 and the time to start, and shares the run's exec ID with the exit upload.
  • SIGN_IN_PROMPTED uploads at exit with credentials that never refresh, so a session whose access token has expired by then loses it. These events are undercounted for long idle sessions.
  • The enum names and JSON keys must match the telemetry schema change.

Tests

  • aitools_mcp_test.go pins the JSON keys. mcp_telemetry_test.go covers each service, outcome, profile count and rejected credentials. TestMcpRemoveUploadIgnoresEnvironmentToken checks that an upload never uses another workspace's token. TestSignInInstalledURLFailureLogsFailedSignIn covers the URL failure, TestMcpRemoveUploadUsesConfigFileOfServer and TestMcpRemoveUploadIgnoresSameNamedProfileInOtherFile the remove grouping, TestMcpServeUploadUsesProfileBoundAtStart the bound profile, TestMcpServeDropsUploadAfterRepoint the drop after a re-point, and TestMcpRemoveUploadSkipsServersOfSeveralProfiles a remove across two profiles.
  • TestMcpServeUploadsProxyStartedAtStart checks that PROXY_STARTED reaches the service while serve runs and isn't sent again at exit. TestUploadSendsEachAcceptedEventOnce, TestUploadKeepsEventsNotAccepted and TestUploadSkippedDespiteConfigUsed cover the logger.
  • TestLogMcpEventLogsRejectedCredential checks that an event naming a rejected credential is logged and uploads with the run. TestSetUploadConfigLogsNoWarningForFailedHostMetadata covers the metadata warning.
  • Acceptance acceptance/cmd/aitools/telemetry/mcp-add, mcp-remove, mcp-sign-in and mcp-serve print the events each command uploaded.
  • Not tested: the command-funnel events, which come with the funnel pull request, and a live workspace.

How to review

Refactors that keep behavior: debug_only.go moved to libs/log/handler/, and the enum mirrors in libs/telemetry/protos/. Read closely: cmd/aitools/mcp_telemetry.go for what each command sends, libs/telemetry/logger.go for the upload slot and its locking, cmd/aitools/mcp_serve.go for dropUploadIfRebound, and libs/telemetry/protos/aitools_mcp.go for the enums.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 482c31879 | Workflow run

Regression test report

Selected 4 changed tests (limit=20, 0 not selected)

4 tests: regression=4 unreleased=0 coverage=0 failing=0
test branch main (ae033ef) latest (v1.27.0)
TestAccept/cmd/aitools/telemetry/mcp-add ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/mcp-remove ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/mcp-serve ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/mcp-sign-in ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/mcp-add: branch ✅ | main (ae033ef) ❌ | latest (v1.27.0) ❌

main (ae033ef):

acceptance_test.go:1268: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46379" ("[DATABRICKS_HOST]")
    acceptance_test.go:1268: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46379" ("[DATABRICKS_HOST]")
    acceptance_test.go:1400: Diff:
        --- cmd/aitools/telemetry/mcp-add/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetrymcp-add2143794911/002/output.txt
        @@ -2,93 +2,11 @@
         === A new install for two agents: CHANGED per (service, agent); github needs sign-in, so its sign-in state is PENDING
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,codex --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CODEX",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GITHUB",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "PENDING"
        -    },
        -    {
        -      "service": "GITHUB",
        -      "agent": "CODEX",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "PENDING"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
         
         === The same servers again are re-asserted: UNCHANGED
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,codex --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CODEX",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GITHUB",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "PENDING"
        -    },
        -    {
        -      "service": "GITHUB",
        -      "agent": "CODEX",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "PENDING"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
         
         === An agent config that does not parse fails that agent only: FAILED with AGENT_CONFIG_INVALID
         
         >>> musterr [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code,codex --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CODEX",
        -      "outcome": "FAILED",
        -      "error_category": "AGENT_CONFIG_INVALID",
        -      "sign_in_state": "NOT_REQUIRED"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}

latest (v1.27.0):

acceptance_test.go:1268: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34857" ("[DATABRICKS_HOST]")
    acceptance_test.go:1268: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34857" ("[DATABRICKS_HOST]")
    acceptance_test.go:1400: Diff:
        --- cmd/aitools/telemetry/mcp-add/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetrymcp-add3915186160/002/output.txt
        @@ -2,93 +2,29 @@
         === A new install for two agents: CHANGED per (service, agent); github needs sign-in, so its sign-in state is PENDING
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,codex --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CODEX",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GITHUB",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "PENDING"
        -    },
        -    {
        -      "service": "GITHUB",
        -      "agent": "CODEX",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "PENDING"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        +Error: unknown flag: --names
         
        -=== The same servers again are re-asserted: UNCHANGED
        +Usage:
        +  databricks aitools [command]
         
        ->>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp,system.ai.github --agents claude-code,codex --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CODEX",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GITHUB",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "PENDING"
        -    },
        -    {
        -      "service": "GITHUB",
        -      "agent": "CODEX",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "PENDING"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        +Available Commands:
        +  auto-install Turn the automatic installation of Databricks AI tools on or off
        +  install      Install Databricks skills and plugins for coding agents
        +  list         List installed skills and plugins
        +  uninstall    Uninstall Databricks skills and plugins
        +  update       Update installed skills and plugins
        +  version      Show installed skills and plugins version
         
        -=== An agent config that does not parse fails that agent only: FAILED with AGENT_CONFIG_INVALID
        +Flags:
        +  -h, --help[=true|false]   help for aitools
         
        ->>> musterr [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code,codex --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    },
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CODEX",
        -      "outcome": "FAILED",
        -      "error_category": "AGENT_CONFIG_INVALID",
        -      "sign_in_state": "NOT_REQUIRED"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
        +
        +Exit code: 1
TestAccept/cmd/aitools/telemetry/mcp-remove: branch ✅ | main (ae033ef) ❌ | latest (v1.27.0) ❌

main (ae033ef):

acceptance_test.go:1268: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33855" ("[DATABRICKS_HOST]")
    acceptance_test.go:1268: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33855" ("[DATABRICKS_HOST]")
    acceptance_test.go:1400: Diff:
        --- cmd/aitools/telemetry/mcp-remove/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetrymcp-remove3184915101/002/output.txt
        @@ -2,94 +2,21 @@
         === Installed for two profiles: the bound profile count is 2 after the second add
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
         
         >>> [CLI] aitools mcp add --profile dogfood --names system.ai.genie_one_mcp --agents claude-code --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    }
        -  ],
        -  "bound_profile_count": 2
        -}
         
         === remove --profile dogfood: CHANGED for its agent entry, and one profile stays bound
         
         >>> [CLI] aitools mcp remove --profile dogfood --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "REMOVE",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
         
         === A Databricks entry changed outside this CLI is left in place: SKIPPED
         
         >>> [CLI] aitools mcp remove --profile logfood --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "REMOVE",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "SKIPPED"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
         
         === An entry already gone from the agent config is CHANGED, and the state record goes
         
         >>> [CLI] aitools mcp remove --names genie_one_mcp-logfood --profile logfood --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "REMOVE",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
         
         === A record with no agent left goes with a REMOVE of its own: CHANGED with no agent, and no profile bound
         
         >>> [CLI] aitools mcp remove --names dbsql-logfood --profile logfood --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "REMOVE",
        -  "servers": [
        -    {
        -      "service": "DBSQL",
        -      "outcome": "CHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}

latest (v1.27.0):

acceptance_test.go:1268: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46537" ("[DATABRICKS_HOST]")
    acceptance_test.go:1268: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46537" ("[DATABRICKS_HOST]")
    acceptance_test.go:1400: Diff:
        --- cmd/aitools/telemetry/mcp-remove/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetrymcp-remove1145144591/002/output.txt
        @@ -2,94 +2,29 @@
         === Installed for two profiles: the bound profile count is 2 after the second add
         
         >>> [CLI] aitools mcp add --profile logfood --names system.ai.genie_one_mcp --agents claude-code --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        +Error: unknown flag: --names
         
        ->>> [CLI] aitools mcp add --profile dogfood --names system.ai.genie_one_mcp --agents claude-code --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    }
        -  ],
        -  "bound_profile_count": 2
        -}
        +Usage:
        +  databricks aitools [command]
         
        -=== remove --profile dogfood: CHANGED for its agent entry, and one profile stays bound
        +Available Commands:
        +  auto-install Turn the automatic installation of Databricks AI tools on or off
        +  install      Install Databricks skills and plugins for coding agents
        +  list         List installed skills and plugins
        +  uninstall    Uninstall Databricks skills and plugins
        +  update       Update installed skills and plugins
        +  version      Show installed skills and plugins version
         
        ->>> [CLI] aitools mcp remove --profile dogfood --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "REMOVE",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        +Flags:
        +  -h, --help[=true|false]   help for aitools
         
        -=== A Databricks entry changed outside this CLI is left in place: SKIPPED
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
         
        ->>> [CLI] aitools mcp remove --profile logfood --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "REMOVE",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "SKIPPED"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        +Use "databricks aitools [command] --help" for more information about a command.
         
        -=== An entry already gone from the agent config is CHANGED, and the state record goes
         
        ->>> [CLI] aitools mcp remove --names genie_one_mcp-logfood --profile logfood --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "REMOVE",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        -
        -=== A record with no agent left goes with a REMOVE of its own: CHANGED with no agent, and no profile bound
        -
        ->>> [CLI] aitools mcp remove --names dbsql-logfood --profile logfood --output json
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "REMOVE",
        -  "servers": [
        -    {
        -      "service": "DBSQL",
        -      "outcome": "CHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        +Exit code: 1
TestAccept/cmd/aitools/telemetry/mcp-serve: branch ✅ | main (ae033ef) ❌ | latest (v1.27.0) ❌

main (ae033ef):

acceptance_test.go:1268: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33883" ("[DATABRICKS_HOST]")
    acceptance_test.go:1268: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33883" ("[DATABRICKS_HOST]")
    acceptance_test.go:1400: Diff:
        --- cmd/aitools/telemetry/mcp-serve/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetrymcp-serve2148358282/002/output.txt
        @@ -3,16 +3,6 @@
         
         >>> [CLI] aitools mcp serve --profile logfood --service system.ai.genie_one_mcp --agent claude-code
         No install record for system.ai.genie_one_mcp on profile "logfood"; using its current host [DATABRICKS_URL]
        -{
        -  "calling_agent": "CLAUDE_CODE",
        -  "operation": "PROXY_STARTED",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
         
         === A binding that fails logs no PROXY_STARTED, so no upload follows
         
        @@ -20,25 +10,3 @@
         Error: MCP server system.ai.genie_one_mcp needs Databricks profile "gone", which does not exist in [TEST_TMP_DIR]/.databrickscfg. Set it up with: databricks auth login --profile gone
         
         === authenticate opens the sign-in page: PROXY_STARTED, then SIGN_IN_PROMPTED with the sign-in state after the wait
        -{
        -  "calling_agent": "CLAUDE_CODE",
        -  "operation": "PROXY_STARTED",
        -  "servers": [
        -    {
        -      "service": "GITHUB"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        -{
        -  "calling_agent": "CLAUDE_CODE",
        -  "operation": "SIGN_IN_PROMPTED",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "error_category": "SIGN_IN_REQUIRED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}

latest (v1.27.0):

acceptance_test.go:1268: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42695" ("[DATABRICKS_HOST]")
    acceptance_test.go:1268: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42695" ("[DATABRICKS_HOST]")
    acceptance_test.go:1400: Diff:
        --- cmd/aitools/telemetry/mcp-serve/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetrymcp-serve1904868695/002/output.txt
        @@ -2,43 +2,29 @@
         === serve uploads PROXY_STARTED after the binding checks, while it serves
         
         >>> [CLI] aitools mcp serve --profile logfood --service system.ai.genie_one_mcp --agent claude-code
        -No install record for system.ai.genie_one_mcp on profile "logfood"; using its current host [DATABRICKS_URL]
        -{
        -  "calling_agent": "CLAUDE_CODE",
        -  "operation": "PROXY_STARTED",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        +Error: unknown flag: --service
         
        -=== A binding that fails logs no PROXY_STARTED, so no upload follows
        +Usage:
        +  databricks aitools [command]
         
        ->>> musterr [CLI] aitools mcp serve --profile gone --service system.ai.genie_one_mcp --agent codex
        -Error: MCP server system.ai.genie_one_mcp needs Databricks profile "gone", which does not exist in [TEST_TMP_DIR]/.databrickscfg. Set it up with: databricks auth login --profile gone
        +Available Commands:
        +  auto-install Turn the automatic installation of Databricks AI tools on or off
        +  install      Install Databricks skills and plugins for coding agents
        +  list         List installed skills and plugins
        +  uninstall    Uninstall Databricks skills and plugins
        +  update       Update installed skills and plugins
        +  version      Show installed skills and plugins version
         
        -=== authenticate opens the sign-in page: PROXY_STARTED, then SIGN_IN_PROMPTED with the sign-in state after the wait
        -{
        -  "calling_agent": "CLAUDE_CODE",
        -  "operation": "PROXY_STARTED",
        -  "servers": [
        -    {
        -      "service": "GITHUB"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        -{
        -  "calling_agent": "CLAUDE_CODE",
        -  "operation": "SIGN_IN_PROMPTED",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "error_category": "SIGN_IN_REQUIRED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        +Flags:
        +  -h, --help[=true|false]   help for aitools
        +
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
        +
        +Exit code: 1
TestAccept/cmd/aitools/telemetry/mcp-sign-in: branch ✅ | main (ae033ef) ❌ | latest (v1.27.0) ❌

main (ae033ef):

acceptance_test.go:1268: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33957" ("[DATABRICKS_HOST]")
    acceptance_test.go:1268: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33957" ("[DATABRICKS_HOST]")
    acceptance_test.go:1400: Diff:
        --- cmd/aitools/telemetry/mcp-sign-in/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetrymcp-sign-in3743646352/002/output.txt
        @@ -3,104 +3,21 @@
         
         >>> [CLI] aitools mcp login system.ai.genie_one_mcp --profile logfood
         system.ai.genie_one_mcp does not need sign-in.
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_IN",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
         
         === login completes once the credential is ACTIVE: CHANGED, sign-in state ACTIVE
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_IN",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
         
         === login while already signed in: UNCHANGED
         
         >>> [CLI] aitools mcp login system.ai.github --profile logfood
         Already signed in to system.ai.github on profile logfood.
         Public repositories only until a repository owner installs https://github.com/apps/databricks-github-connector (on AWS GovCloud, https://github.com/apps/databricks-gh-connector-gov-cloud).
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_IN",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
         
         === logout removes the sign-in: CHANGED; a second logout finds nothing: UNCHANGED
         
         >>> [CLI] aitools mcp logout system.ai.github --profile logfood
         Signed out of system.ai.github on profile logfood.
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_OUT",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "CHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
         
         >>> [CLI] aitools mcp logout system.ai.github --profile logfood
         Not signed in to system.ai.github on profile logfood.
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_OUT",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "UNCHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
         
         === add --login signs in during the same run: SIGN_IN first, then ADD with sign-in state ACTIVE
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_IN",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        
FAIL

latest (v1.27.0):

acceptance_test.go:1268: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37323" ("[DATABRICKS_HOST]")
    acceptance_test.go:1268: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37323" ("[DATABRICKS_HOST]")
    acceptance_test.go:1400: Diff:
        --- cmd/aitools/telemetry/mcp-sign-in/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetrymcp-sign-in612434248/002/output.txt
        @@ -2,105 +2,37 @@
         === login to a service that needs no sign-in: UNCHANGED, sign-in state NOT_REQUIRED
         
         >>> [CLI] aitools mcp login system.ai.genie_one_mcp --profile logfood
        -system.ai.genie_one_mcp does not need sign-in.
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_IN",
        -  "servers": [
        -    {
        -      "service": "GENIE_ONE",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "NOT_REQUIRED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
         
         === login completes once the credential is ACTIVE: CHANGED, sign-in state ACTIVE
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_IN",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        -=== login while already signed in: UNCHANGED
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
         
        ->>> [CLI] aitools mcp login system.ai.github --profile logfood
        -Already signed in to system.ai.github on profile logfood.
        -Public repositories only until a repository owner installs https://github.com/apps/databricks-github-connector (on AWS GovCloud, https://github.com/apps/databricks-gh-connector-gov-cloud).
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_IN",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "UNCHANGED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
         
        -=== logout removes the sign-in: CHANGED; a second logout finds nothing: UNCHANGED
        +Usage:
        +  databricks aitools [command]
         
        ->>> [CLI] aitools mcp logout system.ai.github --profile logfood
        -Signed out of system.ai.github on profile logfood.
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_OUT",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "CHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        +Available Commands:
        +  auto-install Turn the automatic installation of Databricks AI tools on or off
        +  install      Install Databricks skills and plugins for coding agents
        +  list         List installed skills and plugins
        +  uninstall    Uninstall Databricks skills and plugins
        +  update       Update installed skills and plugins
        +  version      Show installed skills and plugins version
         
        ->>> [CLI] aitools mcp logout system.ai.github --profile logfood
        -Not signed in to system.ai.github on profile logfood.
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_OUT",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "UNCHANGED"
        -    }
        -  ],
        -  "bound_profile_count": 0
        -}
        +Flags:
        +  -h, --help[=true|false]   help for aitools
         
        -=== add --login signs in during the same run: SIGN_IN first, then ADD with sign-in state ACTIVE
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "SIGN_IN",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        -{
        -  "calling_agent": "TYPE_UNSPECIFIED",
        -  "operation": "ADD",
        -  "servers": [
        -    {
        -      "service": "GITHUB",
        -      "agent": "CLAUDE_CODE",
        -      "outcome": "CHANGED",
        -      "sign_in_state": "ACTIVE"
        -    }
        -  ],
        -  "bound_profile_count": 1
        -}
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +retry: condition not met after 60 attempts
        +
        +Exit code: 1
        
    acceptance_test.go:1190: Test produced unexpected files:
        login.txt
FAIL

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 482c318

Run: 44580631420

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 272 23 11:19
💚​ aws-windows-latest-4core-16gb 1 274 21 10:35
💚​ azure linux-2core-8gb 1 271 23 15:13
💚​ azure-windows-latest-4core-16gb 1 273 21 12:19
💚​ gcp linux-2core-8gb 1 272 23 15:23
💚​ gcp-windows-latest-4core-16gb 1 274 21 8:43
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 23 slowest tests (at least 2 minutes):
duration env testname
4:32 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
4:20 azure-windows-latest-4core-16gb TestAccept
4:20 gcp-windows-latest-4core-16gb TestAccept
4:00 aws-windows-latest-4core-16gb TestAccept
3:41 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsDelete
3:24 azure linux-2core-8gb TestFilerReadWrite/workspace_files
3:22 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
3:06 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsDelete
3:01 azure-windows-latest-4core-16gb TestSyncIncrementalFileSync
3:01 aws-windows-latest-4core-16gb TestImportDir
3:00 gcp linux-2core-8gb TestFilerWorkspaceNotebook/rNb.r
2:57 gcp linux-2core-8gb TestSyncIncrementalFileSync
2:44 aws-windows-latest-4core-16gb TestFilerWorkspaceNotebook/pythonJupyterNb.ipynb
2:38 azure-windows-latest-4core-16gb TestExportDir
2:21 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:12 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:12 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsStat
2:10 aws-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files
2:10 gcp linux-2core-8gb TestFilerRecursiveDelete/workspace_files
2:02 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsRead
2:01 aws linux-2core-8gb TestImportDirDoesNotOverwrite
2:00 aws-windows-latest-4core-16gb TestExportDir
2:00 azure-windows-latest-4core-16gb TestImportDirWithOverwriteFlag

@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 2dd54b7 to a1fd867 Compare November 12, 2026 14:44
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 62533cc to a1d0d64 Compare November 12, 2026 14:47
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from a1fd867 to b90afff Compare November 12, 2026 17:15
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from a1d0d64 to 55e2a1d Compare November 12, 2026 17:16
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from b90afff to 0aa2ee4 Compare November 13, 2026 10:45
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 55e2a1d to a603d10 Compare November 13, 2026 11:06
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 0aa2ee4 to 2dbdfe3 Compare November 13, 2026 12:28
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from a603d10 to 1391f64 Compare November 13, 2026 12:28
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 1fff1da to e9bad5b Compare November 13, 2026 13:28
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 1391f64 to c3e93cc Compare November 13, 2026 14:27
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from b30caa0 to 43a64fd Compare November 13, 2026 15:06
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from f6918d2 to 48880f0 Compare November 13, 2026 15:12
@pietern pietern added the aitools AI tools (databricks aitools) related changes label Nov 16, 2026
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 43a64fd to b12f2bf Compare November 16, 2026 12:47
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch 2 times, most recently from 8fe9c49 to d705658 Compare November 16, 2026 13:19
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from b12f2bf to 1ee83b9 Compare November 16, 2026 13:19
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from d705658 to b3a08ad Compare November 16, 2026 13:32
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 1ee83b9 to 29624cd Compare November 16, 2026 13:32
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from b3a08ad to d206004 Compare November 16, 2026 14:49
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch 2 times, most recently from a7c5766 to 52a1885 Compare November 16, 2026 15:14
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from d206004 to 34ee3fb Compare November 16, 2026 15:14
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 52a1885 to e652cdf Compare November 16, 2026 15:53
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch 2 times, most recently from e1a2287 to a06230b Compare November 17, 2026 10:52
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch 2 times, most recently from 793107d to a3a8757 Compare November 17, 2026 11:21
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch 2 times, most recently from 8e7258c to 211c894 Compare November 17, 2026 12:02
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from a3a8757 to 27ecba1 Compare November 17, 2026 12:02
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 211c894 to c77fc98 Compare November 17, 2026 12:36
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 27ecba1 to 8047e0c Compare November 17, 2026 12:36
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from c77fc98 to e309481 Compare November 17, 2026 15:11
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 8047e0c to 828e6f7 Compare November 17, 2026 15:11
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from d4abf00 to b8bc227 Compare November 17, 2026 18:54
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch 2 times, most recently from a1bf5bf to f1ee623 Compare November 17, 2026 18:57
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch 2 times, most recently from 7bde116 to 5f95215 Compare November 17, 2026 19:58
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch 2 times, most recently from 201f9a7 to ffc760f Compare November 17, 2026 22:17
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 5f95215 to 9910702 Compare November 17, 2026 22:17

@anton-107 anton-107 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One blocking finding, inline on cmd/aitools/mcp_serve.go: serve's PROXY_STARTED is lost for any session that sits idle longer than the access token lasts.

I ran the unit tests for the changed packages (including -race on libs/telemetry, proxy and cmd/aitools) and the cmd/aitools/telemetry and cmd/aitools/mcp acceptance tests locally. All pass.

Non-blocking notes:

  • Two paths depend on there being no cmdctx.ConfigUsed. dropUploadIfRebound calls SetUploadConfig(ctx, nil), and a profile that fails to resolve in setUploadConfig never sets a config. Either way UploadConfig falls back to cmdctx.ConfigUsed. That's safe today only because no mcp command sets it. If the funnel PR wires withTelemetryAuth into these commands, "drop the upload" turns into "upload with ConfigUsed". An explicit no-upload state in the logger would keep the intent.
  • remove uploads nothing when the selected servers belong to more than one profile, for example remove --all with servers on two profiles (sharedProfile returns the zero key). "remove groups its servers by profile and config file" in the description reads as if it uploads per group. No test covers a remove that spans several profiles.
  • signInHooks.Result now names its ctx parameter but doesn't use it (libs/aitools/mcp/proxy/challenge.go:162).
  • KnownLeaves doesn't need //deadcode:allow: deadcode -test already counts callers in tests.

This review was written by Isaac.

Comment thread cmd/aitools/mcp_serve.go Resolved
@simonfaltum

Copy link
Copy Markdown
Member Author

The notes are in too:

  • 612835c: dropping the upload sets an explicit no-upload state, telemetry.SkipUpload, which wins over cmdctx.ConfigUsed. serve's re-point check and a profile that fails to resolve use it, and so do remove's two drop paths, which had the same fallback.
  • remove across several profiles: the behavior stays. The description now says it uploads only when all its servers share one profile and config file, and TestMcpRemoveUploadSkipsServersOfSeveralProfiles pins it (1d52cc5).
  • The context of signInHooks.Result is unnamed again (0a8eff6).
  • The //deadcode:allow on KnownLeaves is gone (c2b5373), and the deadcode check passes.

The start upload's execution context is built by root.TelemetryExecutionContext, which the exit upload now uses too.

@anton-107 anton-107 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 2 (9910702..a688fa4): the blocking finding is fixed. serve now uploads PROXY_STARTED right after its startup checks. That upload reads the token Bind has just refreshed into the cache, and Upload removes the events the service accepted, so the exit upload doesn't send them again.

I checked the shared changes for effects on other commands:

  • Every other command uploads once, so removing accepted events changes nothing for them. The one exception is the ssh agent shim's exec-failure path. That path no longer resends the success event the shim already uploaded, which is what its launched comment intended.
  • Partial accepts behave as before: the batch is retried whole and kept on failure. Only serve's exit upload could resend it, and at start the batch is just PROXY_STARTED.
  • SkipUpload is only called from aitools mcp paths.

Handshake delay: the start upload adds one POST before the proxy reads stdin, and at most 3 s if /telemetry-ext hangs. The agent's initialize waits in the pipe and isn't dropped, so this fits within agent startup timeouts.

Tests: the touched packages and TestAccept/cmd/aitools/telemetry pass.

@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from b32f61f to 9cb89d5 Compare November 19, 2026 11:42
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from a688fa4 to 4f9607b Compare November 19, 2026 11:42
@simonfaltum
simonfaltum force-pushed the aitools-mcp/09-pickers branch from 9cb89d5 to be36588 Compare November 19, 2026 17:06
Base automatically changed from aitools-mcp/09-pickers to main November 19, 2026 17:42
One commit for the restack onto main. The individual commits, up to a688fa4, are in the pull request's history.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum
simonfaltum force-pushed the aitools-mcp/10-telemetry branch from 4f9607b to 482c318 Compare November 19, 2026 17:44

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aitools AI tools (databricks aitools) related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants