Repository navigation
Add and remove agent MCP entries through agent CLIs, except Codex - #7595
Conversation
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/cmd/aitools/mcp/add-remove-agent-clis: branch ✅ | main (de394ed) ❌ | latest (v1.27.0) ❌main (de394ed): latest (v1.27.0): |
✅ Integration test reportCommit: 03d0647
Top 40 slowest tests (at least 2 minutes):
|
7122f6a to
93e536b
Compare
1abf7d0 to
6b41591
Compare
3ed1398 to
8cca61b
Compare
8cca61b to
50318b1
Compare
6b41591 to
48208cf
Compare
50318b1 to
f46d6a0
Compare
48208cf to
922dad8
Compare
f46d6a0 to
d049189
Compare
7e7f6c7 to
865e2f5
Compare
6955d59 to
84abb41
Compare
0952a73 to
1f42a46
Compare
190f3cb to
fdc9d71
Compare
1f42a46 to
d94c24d
Compare
fdc9d71 to
fc23170
Compare
0fb469a to
8237caa
Compare
fc23170 to
0d214eb
Compare
8237caa to
32552ca
Compare
0d214eb to
2a05c5e
Compare
32552ca to
bb76f49
Compare
2a05c5e to
1c32739
Compare
aa07488 to
c22861c
Compare
a9d81d9 to
f9ba127
Compare
c22861c to
55915b3
Compare
10594fe to
b4386f3
Compare
8c7ea9b to
160e62d
Compare
b4386f3 to
114dcaf
Compare
|
|
||
| func TestMain(m *testing.M) { | ||
| // No agent CLI is on PATH unless a test puts one there, so no test runs a host CLI. | ||
| agents.LookPath = func(string) (string, error) { return "", exec.ErrNotFound } |
There was a problem hiding this comment.
TestMain calls m.Run() but doesn't pass its return value to os.Exit. Without os.Exit(m.Run()), test failures in this package exit 0 and CI reports PASS. Should be os.Exit(m.Run()).
There was a problem hiding this comment.
Since Go 1.15, when TestMain returns, the test binary passes the result of m.Run() to os.Exit itself, so a failing test still exits non-zero. libs/localenv/main_test.go uses the same form. No change.
| // https://learn.chatgpt.com/docs/config-file/config-advanced and | ||
| // https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-config-dir-reference. | ||
| const ( | ||
| claudeHomeEnv = "CLAUDE_CONFIG_DIR" |
There was a problem hiding this comment.
The new claudeHomeEnv and geminiHomeEnv constants are used in claudeConfigDir, claudeMCPPath, geminiConfigDir, the MCP spec entries, and mcpHomeEnvs, but mcpHomeDefaults at line 121 still has the string literals "CLAUDE_CONFIG_DIR" and "GEMINI_CLI_HOME". Can you update the map to use the constants, as it already does for codexHomeEnv and copilotHomeEnv?
anton-107
left a comment
There was a problem hiding this comment.
Approving. I found nothing that blocks the merge.
Merge risk: easy to undo. aitools mcp is still hidden. Every write through an agent's CLI is checked by reading the file back, and the direct writer fixes any mismatch. Each agent's file stays in that agent's own valid format. Reverting this PR just sends every agent back to the direct writer.
What I checked
./cmd/aitools/...,./libs/aitools/...and./libs/process/...unit tests, plusTestAccept/cmd/aitools, all pass on 686894a.- I ran the real Claude Code CLI (2.1.325) against a throwaway HOME.
claude mcp add-json --scope userwrites the entry as expected. It also works withCLAUDECODE=1set, which is the likely real case:databricks aitools mcp addrun from inside a Claude Code session. - Gemini's argv matches upstream
packages/cli/src/commands/mcp/add.ts. There,-eisnargs: 1,populate--appends the arguments after--, and env values are split withsplit('=')keeping only two parts. So sending values that contain=to the direct writer is the right call. - Previous round: both bot comments were handled correctly. The constants were fixed in 686894a, and
os.Exit(m.Run())hasn't been needed since Go 1.15.
Non-blocking notes
- Running
claudealso applies Claude's own startup migrations to.claude.json. On a bare{}file it addedmachineID,userID,migrationVersionand similar keys. That's harmless for real users, whose file already has them, but it does go beyond writing the one entry. buildFakeAgentCLIsnow runs on every non-in-process acceptance run, even when nocmd/aitoolstest is selected. It runs in parallel so the cost is small, but it could be skipped when the selection doesn't need it.agents.LookPathis now an exported, mutable package variable, used only so tests in other packages can stub it. That's fine for now, but it's a test hook in the package's public API.
|
Thanks for running the real CLI!
|
6957247 to
edbd091
Compare
686894a to
8898daf
Compare
|
Heads-up after the restack onto main. With #7539's lock and this PR's CLI path together, |
edbd091 to
22ff1dc
Compare
One commit for the restack onto main. The individual commits, up to 686894a, are in the pull request's history. Co-authored-by: Isaac <no-reply@databricks.com>
8898daf to
03d0647
Compare
❌ Integration test reportCommit: fd2f501
521 interesting tests: 490 MISS, 10 KNOWN, 10 RECOVERED, 6 FAIL, 5 PANIC
Top 50 slowest tests (at least 2 minutes):
|
This pull request is part of the stack that adds the hidden `databricks aitools mcp` command group. It sits on databricks#7595, which adds and removes agent MCP entries through each agent's own CLI. `databricks aitools mcp add` and `remove` stopped in a terminal with "interactive selection is not available yet" when a flag was missing. This PR replaces that stopgap with pickers. Runs without a terminal keep their flags and output. ## Changes - In a terminal, `add` asks for the profile, the services and the agents that the flags do not give. The profile resolves as it does without a terminal, so the picker runs only where that stops with several profiles. - `add` has its own agent picker rather than `install`'s. Like the other `mcp` pickers, Esc cancels it, and its help says enter confirms. - At the catalog and schema prompt, Esc stops listing, as an empty answer does, and keeps the picks of the pages already shown. Ctrl-C still aborts the add. - A picker cancelled with Esc leaves nothing on screen. - A run that picks its services checks the profile and chooses the agents before it connects. A missing profile or no agent is refused without a request. A picked service whose recorded entry points at another host is refused after the pick, before anything is written. - The service picker marks services the profile already has as `(installed)`, unchecked. "Another schema..." lists another catalog and schema and keeps the checks. - The profile picker rows have no `[default]` mark. - The plan says `add`, `re-assert` (only when the write leaves the entry as it is), `update` or `replace`. An agent whose config cannot be read is shown with the reason. - `add` asks `Proceed?` only when a picker made a choice. For each sign-in-required server it installed, it asks "Sign in now?" unless the caller is already signed in. Ctrl-C stops the remaining questions and prints the login command of each server still to sign in. - In a terminal, `remove` with none of `--names`, `--profile` or `--all` shows the installed servers and asks `Proceed?` before removing the picked ones. - `add --no-login` skips the sign-in question that a terminal asks for each sign-in-required server it installed. It cannot be combined with `--login`. Without a terminal, `add` already does not sign in. - An empty pick in the service picker prints `No servers selected.` and exits 0 without writing anything. It used to fail with INVALID_FLAGS. - The pickers share `install`'s picker model, from databricks#7721: `runPicker` runs a built picker, `newPicker` takes the search func (nil for none), and the model gains an `onKey` hook and a search check that allows none. ## Why Choosing servers needs their descriptions and sign-in requirements, which the workspace knows and the flags do not show. ## Decisions to review - Agents are chosen before the services, so a missing agent or a broken profile is reported before any request. Cost: the agent question comes first. - Esc at the catalog and schema prompt stops listing instead of aborting the add, so the picks already shown are kept. Cost: an Esc meant to abort keeps those picks; Ctrl-C aborts. - `add` has its own agent picker, so the two copies can drift apart. - A picked service is checked against its recorded entry only after the pick, so one stale entry of another service does not stop the picker. - Installed services are checkable rows, not disabled rows, because the picker library has no disabled options. Checking one re-asserts its entry, as `add --names` does. - `--name` without `--names` is refused rather than offered a single-select picker. ## Tests - `mcp_pickers_test.go` covers the service and profile pickers, the agent picker's failed check and Esc, the location prompt's Esc, and the remove picker's refusal. `mcp_add_test.go` runs an interactive `add` through the prompt seams: refusals make no request, a changed host is refused after the pick, and a fully specified run asks only the sign-in question. `TestMcpAddPickerEmptyPickAddsNothing` covers the empty pick, and `TestValidateMCPAddFlags` the `--login` and `--no-login` conflict. - `mcp_remove_test.go` runs the remove picker end to end, including a declined confirmation. - `./task test-exp-aitools` passes. `go test -race ./cmd/aitools/ ./libs/aitools/... ./libs/cmdio/` and `GOOS=windows` vet pass. - Not tested: a real terminal and a live workspace. The prompts are covered through their seams. ## How to review Small edits outside the feature, about 15 lines in `libs/cmdio/` (`io.go`, `prompt.go`, `select.go`), `cmd/aitools/mcp.go`, `uninstall.go`, and the moved `multiSelectHelp` constant in `install.go`. Read closely: `cmd/aitools/mcp_pickers.go`, the `add` flow in `cmd/aitools/mcp_add.go` (agent choice at its interactive branch, before the service pick), and `cmd/aitools/mcp_remove.go`. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
This pull request is part of the stack that adds the hidden
databricks aitools mcpcommand group; its parent is #7553.aitools mcp addandremovenow change the Claude Code, Gemini CLI and Copilot CLI configs through each agent's own MCP command, when that CLI is on PATH. Codex CLI configs are always written directly.Changes
claude mcp add-json --scope user,gemini mcp add --scope user, orcopilot mcp add --tools *. A config file that does not exist yet is written directly, with mode 0600, since the agent's CLI would create it with its own permissions. Afterwards the config file holds the entry asked for, whatever the exit status: a differing entry is replaced, and a missing one is added directly. An entry that already exists is updated by the direct writer.removeruns the CLI's remove for an entry we wrote, when the agent reads that file now. A CLI failure, or an entry left behind, falls back to the direct writer.-, and Codex always use the direct writer.Why
An agent's own CLI writes its config with its own locking and layout, and follows the agent when its file format changes. Codex is the exception. With Codex CLI 0.167.1,
codex mcp addandcodex mcp removerewrote every[mcp_servers.*]table of an 11-server config: they droppedenabled = truekeys, collapsed multi-line arrays, turned10into10.0, and converted CRLF to LF.Decisions to review
.cmdshim gets only arguments without spaces, double quotes orcmd.exespecial characters. Other arguments use the direct writer.Tests
libs/aitools/agentscovers argv per agent, including a Windows path with spaces.agentconfig/cli_test.gocovers each outcome of an add,TestAgentCLIRunsWithoutTheLockthe lock, symlinks, missing config files, dash names and the timeout, and the Codex direct writer.libs/processcoversWithoutEnv, which ignores case on Windows.acceptance/cmd/aitools/mcp/add-remove-agent-cliscovers each agent's argv and the files it wrote../task test-exp-aitoolspasses withBROWSERunset and withBROWSER=none.-inprocessmode, which already fails on the base for these tests.This pull request and its description were written by Isaac.