Repository navigation
Add MCP server config writers for coding agents - #7539
Conversation
✅ Integration test reportCommit: 244c03f
|
6f07e48 to
07cc225
Compare
Add a per-agent description of where each coding agent keeps its user-scope MCP server entries, and the writers that add, remove and list the entries this CLI owns without touching anything else in the file. - libs/aitools/mcpconfig: Spec (path, format, root key, entry shape, ownership). - libs/aitools/agents: MCP spec for seven agents, nil for the others; SupportsMCP and MCPAgents. - libs/aitools/mcp/agentconfig: Upsert, Remove and Read over JSON and TOML config files. Writes are atomic, keep the file mode, write through symlinks, and skip the write when the entry already matches. Files that cannot be parsed are refused and left untouched. Entries not written by this CLI are refused unless forced. JSON keeps key order and the bytes of values it does not change. TOML is decoded and re-encoded, so comments are dropped on any rewrite. Co-authored-by: Isaac <no-reply@databricks.com>
JSON and JSONC entries are changed with hujson patches, so comments, trailing commas, a leading BOM and the bytes of everything else are kept. TOML entries are changed as text: only the lines of the server's own tables are replaced or removed, and the edited text is decoded and checked before it is written. Upsert refuses a command that is not named databricks, since such an entry could not be recognized later. A symlink whose target is missing is refused rather than replaced. OpenCode prefers an existing opencode.jsonc. Co-authored-by: Isaac <no-reply@databricks.com>
JSON: a comment on the previous member's line stays there when a member is added. An empty table gets its closing brace on its own line. A file with only comments and an empty object keeps its comments. Removing the last member keeps a strict file strict and keeps a trailing comma that was there. Inserted text uses the file's line ending. Upsert updates an entry this CLI wrote only in the keys the builders write, so keys the user added (for example disabled or enabled) survive a re-add. An entry that differs only in those keys is not rewritten. --force still replaces a foreign entry as a whole. Antigravity entries no longer write disabled: false, since the key is optional. Upsert refuses a server name outside [A-Za-z0-9_-] and says whether the command or the args are wrong. Error messages show paths with forward slashes. Tests set the agent overrides on the context instead of the process environment. Co-authored-by: Isaac <no-reply@databricks.com>
- Update sets only the launch keys (command, args, env, environment, type). Other keys are kept, and defaults such as Copilot's tools list are written only on first add. An entry whose launch keys match is not rewritten. - JSON updates patch the changed launch keys in place, so the other members keep their bytes. Numbers in user values are decoded with UseNumber. - JSON duplicate member names on the root and server path are refused as ErrConfigInvalid. - TOML table headers are found with multi-line strings protected, using localenv's helper, now exported as ProtectMultilineStrings. Quoted keys are parsed, so a quoted "genie.x" is not "genie", and a header-like line inside a string is text. - Removing a table keeps one blank line between its neighbours, and removing the first server leaves no blank first line. - Remove checks the server name as Upsert does. - The OpenCode path returns stat errors other than not-exist. - New config directories are created 0700. - Smaller fixes: top-level JSON error wording, IsUnmanaged comment, mcpconfig.IsDatabricksBinary, BOM strip via bytes.CutPrefix, ValidServerName without a package-level regexp, and a Windows-safe path assertion in toml_test. Co-authored-by: Isaac <no-reply@databricks.com>
Update only what the CLI owns in an existing entry. Launch keys are the ones the agent's builder writes, so a key the builder never writes (type, for most agents) is kept. In env tables only DATABRICKS_CONFIG_FILE is the CLI's, and user variables are kept. Compare TOML edits as TOML text, so inf and nan in an unrelated table no longer block an edit. Add blank-line layout that keeps one blank line before a new server and restores the file's own bytes when the server is removed again. Refuse OpenCode when both opencode.json and opencode.jsonc exist, and treat a dangling opencode.jsonc link as the file to edit, so the write is refused. Keep CRLF on the newlines hujson writes for a new member. Collapse the TOML set/update/remove wrappers into one put, and fix the test comments and assertions flagged in review. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
07cc225 to
f178f46
Compare
Removing the last server left empty containers behind: "mcpServers": {}
in JSON, the emptied levels of "mcp.servers", and a [mcp_servers] table in
TOML. An add to a file without them therefore did not return the file to
its earlier bytes, and an empty object was written with an odd layout.
Remove now drops each object that the removal leaves empty, up to the
root object, and drops the [mcp_servers] table when the server was its
only key. The root object stays, and it is written as {} when nothing but
whitespace is left in it. Each object that changes is also laid out with
the file's line ending.
Trade-off: an empty container that the user wrote explicitly is dropped
too, when it is emptied by this removal. It means the same for every
agent, since an empty container and a missing one are read alike. A
comment placed inside a dropped container goes with it.
Co-authored-by: Isaac <no-reply@databricks.com>
The write replaces the file by rename, which needs write permission only on the directory. A config file with mode 0444 was therefore edited and then left at 0444, so the user's read-only setting was bypassed silently. Add and remove now refuse an existing config file that cannot be opened for writing, and name the file and the agent in the error. The check runs only when the write would change the file, so an add that matches the entry already there still succeeds. A missing file is not refused. Running as root writes read-only files, so the check passes there. A read-only directory still fails in the atomic write, and the error no longer repeats the config path the way it did before. Co-authored-by: Isaac <no-reply@databricks.com>
Removing one env variable from a server used to prune the env table when it
was left empty, so Claude Code entries lost "env": {} and the next identical
write reported updated. Only removing a whole server prunes emptied
containers now; a member removal keeps its parent object.
Duplicate keys were checked only on the server object, so a duplicate
DATABRICKS_CONFIG_FILE inside env was patched in one place and read from
another. Nested objects of an entry are checked now.
Upsert accepted env variables other than the pinned one, which no later
write could remove. It refuses them now.
Pinning into an existing empty env table keeps its closing brace on its own
line.
TOML Remove checked for a server defined inline under [mcp_servers] after
adding the root table's ranges, so it deleted the table and its comments.
The check now runs first, as Upsert's does.
The Cursor MCP path cites its documentation.
Co-authored-by: Isaac <no-reply@databricks.com>
A line inside a multi-line array was read as a table header when it looked like one, such as ["c"]. That ended a server's range early, so Remove left part of an array behind. A value such as ["mcp_servers"] inside an array also passed the inline-root guard, and Upsert then wrote a table that Codex rejects. The header scans now skip lines that start inside an array. A line's brackets outside quotes and comments are counted to find that. The IsUnmanaged comment no longer says Upsert calls it. Upsert refuses any entry this CLI did not write unless force is set. Co-authored-by: Isaac <no-reply@databricks.com>
A dangling-symlink error did not wrap the stat error that caused it, and an env merge error named neither the file nor the server. Both now carry the underlying error and the context that the neighbouring errors have. The Cursor source link moves inside the MCP spec, out of the Plugin note. Co-authored-by: Isaac <no-reply@databricks.com>
Upsert named whichever unexpected env key the map iteration reached first, so the same call could report a different key on each run. The keys are now checked in sorted order. Co-authored-by: Isaac <no-reply@databricks.com>
The path of the Databricks managed MCP servers was a literal in the unmanaged check. Name it once in mcpconfig, which both agentconfig and the MCP package can import, so the two cannot drift apart. Co-authored-by: Isaac <no-reply@databricks.com>
TOML edits are decoded again and checked before the file is written, but JSON edits were not. Share the check between the two formats: the edit must change only the target server, an add or update must write the intended entry, and a removal must remove it. The check compares the rest of the document with the containers a removal empties dropped, as the TOML check does, so the drop of the last server's containers passes. The JSON comparison is made on plain values, and TOML keeps its own comparison, since JSON has no inf or nan. Co-authored-by: Isaac <no-reply@databricks.com>
A JSON edit that fails the check now names the server, as the TOML edit does. The read-only refusal keeps the cause for errors.Is. The read-only test also runs its refusal on Windows, where the mode is the read-only attribute. Co-authored-by: Isaac <no-reply@databricks.com>
The refusal named --agents, which is a flag of the MCP commands and not of the writer. The writer now says only that the file is read-only and asks for it to be made writable. A ReadOnlyError carries the path and wraps the error that opening the file returned, and it matches the new ErrReadOnly sentinel, so a command can name its own way out of the refusal. Co-authored-by: Isaac <no-reply@databricks.com>
The check that a JSON edit changed nothing but the target server decoded the whole file before and after each edit. On a large ~/.claude.json that cost about 1.8 s per add or remove. Decode only the containers on the root path and the server. Every other member is kept as its raw bytes, which hujson leaves as they were when a patch does not touch them. The layout after a member's value is left out of the comparison, since removing the last member moves the trailing comma after it. The guarantees are the same. Nothing outside the server changes, except the containers that a removal empties. An add or update writes exactly the entry, and a removal removes it. TOML keeps its own comparison. BenchmarkJSONAddBesideLargeMember adds a server beside a large member: 1 MB went from 19 ms to 1.4 ms, and 16 MB from 312 ms to 19 ms. The cost left is copying the other members' bytes, not decoding them. Co-authored-by: Isaac <no-reply@databricks.com>
anton-107
left a comment
There was a problem hiding this comment.
Requesting changes for one issue, posted inline: Claude Code does lock ~/.claude.json, so writing it without that lock loses updates in both directions.
The rest holds up. I checked the config paths against current docs for all seven agents, including OpenCode's mcp.servers form and Antigravity's ~/.gemini/config/mcp_config.json. I also ran scratch inputs through the JSON and TOML editors: comments in odd places, minified files, CRLF, multi-line strings, arrays of tables and inline roots. Every output re-parsed, and the post-edit checks refused the cases they should.
Non-blocking notes:
- OpenCode V1:
mcp.serversis the V2 form. An install still on V1 keeps its servers directly undermcp, so this write would add a server namedserversthere. If V1 installs still matter, consider refusing a file whosemcpalready holds V1-style servers. - Claude legacy config: Claude Code reads
~/.claude/.config.jsoninstead of~/.claude.jsonwhen that legacy file exists and is non-empty. In that case the entry lands in a file Claude does not read. - Cosmetic layout leftovers (outputs still valid):
- JSON: after an add and a remove,
"other": {...}, // cbecomes"other": {...} // cwith a misindented}below it. - TOML: a comment at the end of a server table, right before the blank line, is left behind by
Remove, so it ends up above the next table.
- JSON: after an add and a remove,
Claude Code locks ~/.claude.json while it saves it, by mkdir on <real path>.lock. An edit that read the file, then renamed its own version over it without the lock could drop a Claude Code update, and Claude Code could drop the entry the edit had just written. Specs that set Lock now take the same lock across load, edit and save in Upsert and Remove. The lock is the directory named after the config's real path, and an existing lock older than 10 s is taken as abandoned, as Claude Code does. A live holder keeps the lock for milliseconds, so the lock is not refreshed. A held lock is waited for up to 15 s, then the edit fails naming the lock. Only Claude Code's spec sets Lock. Co-authored-by: Isaac <no-reply@databricks.com>
OpenCode's V1 layout keeps servers directly under mcp, and V2 moves them under mcp.servers. A write to a V1 file would add a mcp.servers table beside the V1 servers, which a V1 OpenCode reads as a server named servers. Upsert and Remove now refuse such a file with an error that names the servers to move, and leave the file as it is. A member of mcp is taken as a V1 server when it is an object with a type or command key, which every documented server shape has. The V2 mcp.timeout object has neither, so a V2 file still works. The check is opt-in per spec through Legacy, and only the OpenCode spec sets it. Co-authored-by: Isaac <no-reply@databricks.com>
Claude Code reads <config dir>/.config.json instead of the global .claude.json when the legacy file is non-empty. The entry was written to .claude.json in that case, into a file Claude Code does not read. claudeMCPPath now follows Claude Code's rule. A non-empty legacy file wins. An empty one wins only if it has a write bit set and the global file is missing. The lock from the previous commit is taken on whichever file is written, since it is named after the file's real path. Claude Code's placeholder check for an empty, read-only global file on Linux and WSL is not modelled. Co-authored-by: Isaac <no-reply@databricks.com>
|
The notes:
|
anton-107
left a comment
There was a problem hiding this comment.
Round 2 (6ef0082, a fast-forward of 1f2656d).
- Legacy
.config.json: precedence matches Claude Code 2.1.325. I ran six cases against the real binary in throwaway homes: non-empty legacy with and without a global file, empty writable legacy with and without a global file, empty read-only legacy, andCLAUDE_CONFIG_DIR. In each one Claude wrote, and locked, the same fileclaudeMCPPathreturns. - OpenCode V1 refusal: looks right.
- Lock protocol: the lock is released on every path through
defer. mkdir/rmdir and the 10 s mtime rule match proper-lockfile. Claude refreshes a held lock's mtime every 5 s, so the takeover can't steal a live lock unless Claude's event loop stalls for over 10 s. - Tests and Windows: the
libs/aitools/mcp/...tests pass, andGOOS=windows go vetis clean.
The round-1 lock finding is fixed for a regular .claude.json but not for a symlinked one, which save explicitly supports; details inline. That is the only blocker, and the fix is small.
…ock Windows is still removing Claude Code locks <config path>.lock without resolving symlinks, so a symlinked config is locked next to the link. On Windows, creating a directory that another process is still removing fails with access denied; that is retried until the lock timeout. Co-authored-by: Isaac <no-reply@databricks.com>
anton-107
left a comment
There was a problem hiding this comment.
Round 3 on 244c03f: the symlink case is fixed, approving.
- Claude Code 2.1.325 under strace still locks
$HOME/.claude.json.locknext to the link, which is the nameconfigLocknow returns. I raced 12 realclaude mcp addruns againstUpserton a symlinked~/.claude.json, 5 times: nothing lost on either side. The same race on 6ef0082 lost one entry each way. - The Windows retry applies only on Windows and only to access denied from the mkdir. It shares the 15 s deadline, and after it fails with the lock path and the OS error.
TestLockedEditLosesNoUpdatesnow passes on both Windows jobs.
…abricks#7538) This is part of a stack that adds the hidden `databricks aitools mcp` command group, on top of databricks#7539. This PR adds the helpers that group needs to find Databricks managed MCP services, name their servers and build their URLs. No command calls them yet. ## Changes - `Discover` lists the services in a `catalog.schema` location with `ListMcpServicesAll` and `view=FULL`, so each source connection comes back. Other locations are refused before a request. Services with a deleted source connection are dropped. - `Service.SignInRequired()` is true when the source connection has an `oauth_provider` option. The schema and leaf come from the name the API returns, which Unity Catalog lowercases. - `system.ai` services get a label, summary and note from a static table of 11 services. Notes say which services need the Enhanced Unity Gateway preview, and the GitHub note names the GovCloud app. Other services carry only the API comment. - `ServerName` returns `<leaf>-<profile>` in `system.ai` and `<catalog>-<schema>-<leaf>-<profile>` elsewhere. Parts are sanitized to `[a-z0-9_-]` with `_` runs collapsed, so no name contains `__`, which agents split tool names on. A leaf with no usable characters returns `ErrNoServerName`. - A name over 32 characters is cut and ends in 6 hex characters of the sha256 of the full name. `UniqueName` appends `-2`, `-3`, ... and stays within the cap. - `ServiceURL` builds the endpoint from `mcpconfig.MCPServicesPathPrefix`. `SignInURL` refuses an empty or `none` workspace ID, passes a numeric ID as `o=` and any other ID as `w=`. Its path and the schema parent prefix are the `signInPath` and `schemaParentPrefix` constants. - Listing failures return `*ListError` with kinds `MCP_LOCATION_NOT_FOUND`, `MCP_PERMISSION_DENIED` and `RATE_LIMITED`. Each keeps the server's message, since the SDK maps every 404, and a private-link 403, to the same sentinels. The 403 hint names `USE_CATALOG` and `USE_SCHEMA` and says an invalid access token gets the same error. `ENDPOINT_NOT_FOUND`, `FEATURE_DISABLED` and private-link failures get no hint. - The testserver serves the list endpoint, filtering by `parent` in any casing and omitting source connections unless `view` is `FULL`. ## Why Adding, listing and removing servers all need the same service list, names and sign-in check. With `view=FULL` the list response carries `oauth_provider`, so sign-in detection needs no per-service request. Built-in service docs: https://docs.databricks.com/aws/en/agents/mcp-tools/built-in-mcp-services ## Decisions to review - Discovery does not retry on HTTP 429. The SDK already retries for up to five minutes, so another retry would double the wait. - The known table holds labels and notes only. Sign-in comes from the API, so a table entry cannot hide a service. `Known` is set only for `system.ai`, so a customer schema's `github` service gets no table entry. - `UniqueName` takes a `taken` function, so this package does not depend on the state file. ## Tests - Discovery tests run against an `httptest` server: one call per page, pagination, sign-in detection, deleted connections, mixed-case locations, non-`catalog.schema` locations (no request), and 404 and 403 mapping with their messages. - `TestKnownSystemAITable`, `TestSanitize`, `TestServerName*`, `TestUniqueName*`, `TestServiceURLEscapesTheName` and `TestSignInURL*` cover naming, URLs and the workspace ID cases. Testserver tests cover `FULL` and `BASIC` output and `parent` filtering. - `go test` on both packages, golangci-lint and `GOOS=windows go vet` pass. - A read-only listing of `schemas/System.AI` on a real workspace returned the same 11 services, with lowercase names, as `schemas/system.ai`. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
…tabricks#7595) This pull request is part of the stack that adds the hidden `databricks aitools mcp` command group; its parent is databricks#7553. `aitools mcp add` and `remove` now change the Claude Code, Gemini CLI and Copilot CLI configs through each agent's own MCP command, when that CLI is on PATH. Codex CLI configs are always written directly. ## Changes - A new entry runs the agent's add command when its config file exists: `claude mcp add-json --scope user`, `gemini mcp add --scope user`, or `copilot mcp add --tools *`. A config file that does not exist yet is written directly, with mode 0600, since the agent's CLI would create it with its own permissions. Afterwards the config file holds the entry asked for, whatever the exit status: a differing entry is replaced, and a missing one is added directly. An entry that already exists is updated by the direct writer. - `remove` runs the CLI's remove for an entry we wrote, when the agent reads that file now. A CLI failure, or an entry left behind, falls back to the direct writer. - Symlinked configs, names that start with `-`, and Codex always use the direct writer. - The CLI runs in an empty directory with a 30-second timeout, in the foreground process group so Ctrl-C reaches it. - For an agent that locks its config (Claude Code, see databricks#7539), only the direct writer takes the lock. It is never held while the agent's CLI runs, since that CLI takes the same lock. ## Why An agent's own CLI writes its config with its own locking and layout, and follows the agent when its file format changes. Codex is the exception. With Codex CLI 0.167.1, `codex mcp add` and `codex mcp remove` rewrote every `[mcp_servers.*]` table of an 11-server config: they dropped `enabled = true` keys, collapsed multi-line arrays, turned `10` into `10.0`, and converted CRLF to LF. ## Decisions to review - The file decides, not the exit status. Cost: a CLI that writes another file keeps its entry there, and ours is also added to the agent's file. - A differing entry is replaced, not refused. Cost: our launch keys win over the CLI's choices. - Existing entries use the direct writer, since the add commands replace whole entries. Cost: two paths write one file. - Codex always uses the direct writer. Cost: comments inside a Databricks server's own tables are lost on either path. - A hung CLI costs the 30-second timeout for each server. - On Windows, a `.cmd` shim gets only arguments without spaces, double quotes or `cmd.exe` special characters. Other arguments use the direct writer. ## Tests - `libs/aitools/agents` covers argv per agent, including a Windows path with spaces. `agentconfig/cli_test.go` covers each outcome of an add, `TestAgentCLIRunsWithoutTheLock` the lock, symlinks, missing config files, dash names and the timeout, and the Codex direct writer. `libs/process` covers `WithoutEnv`, which ignores case on Windows. - `acceptance/cmd/aitools/mcp/add-remove-agent-clis` covers each agent's argv and the files it wrote. - `./task test-exp-aitools` passes with `BROWSER` unset and with `BROWSER=none`. - Not tested: the Copilot and Gemini binaries, which are not installed here, so their argv follows their docs and source. Windows runs, which CI covers. `-inprocess` mode, which already fails on the base for these tests. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
This pull request is part of a stack that adds the hidden
databricks aitools mcpcommand group. It adds the layer that writes one agent MCP server entry and leaves the rest of the agent's config file unchanged. No command uses it yet.Changes
libs/aitools/mcp/agentconfigaddsUpsert,RemoveandRead. Writes are atomic, keep the file mode, and are skipped when the entry already matches.Upsertrefuses an entry this CLI did not write unlessforceis set, a server name outside[A-Za-z0-9_-], a command that is not the databricks binary, and env variables other thanDATABRICKS_CONFIG_FILE.agentconfig.ErrReadOnly, reported as<file> is read-only; make it writable.UpsertandRemovehold the lock Claude Code takes while it saves its config, from read to rename: the directory<path>.lock, named after the path as given so a symlinked config is locked next to the link, made with mkdir and stale after 10 s. A live lock is waited for up to 15 s; then the edit fails and names the lock.<config dir>/.config.json(legacy) wins over.claude.json, and so does an empty writable one when.claude.jsonis missing.mcpstill holds servers in the V1 layout (an object withtypeorcommandoutsidemcp.servers) is refused, naming the servers to move.libs/aitools/agents: Claude Code, Codex, Cursor, Copilot CLI, Gemini CLI, OpenCode and Antigravity.mcpconfig.MCPServicesPathPrefixnames the managed-service URL path, whichIsUnmanageduses.Why
Each agent keeps MCP servers in a different file and shape. The writer keeps comments and layout outside the entry it changes, and refuses any file it cannot parse, so a user's other agent settings survive.
Decisions to review
"mcpServers": {}behind after every add and remove.toolslist they narrowed.opencode.jsonandopencode.jsoncpresent is refused. The entry uses the current docs' form:mcp.servers,environment, noenabled.mcp.servers.Tests
TestRoundTripPerAgentwrites, rewrites and removes an entry for each agent, in LF and CRLF files.TestTOMLAddThenRemoveRestoresBytescovers the byte round trip,TestJSONCheckEditComparesOtherMembersAsBytesthe post-edit check,TestWriteRefusesReadOnlyFilethe read-only refusal.TestLockedEditLosesNoUpdatesruns a Claude-style locked writer against 100 upserts and loses no update in either direction; with the lock disabled it loses most of the writer's saves.TestOpenCodeV1LayoutIsRefusedandTestClaudeMCPPathFollowsLegacyConfigcover the layout and the path.go test ./libs/aitools/... ./libs/localenv/...,TestAccept/cmd/aitools,GOOS=windows go vet, golangci-lint and./task checkspass. CI passes on Linux, macOS and Windows.mcp addlater in the stack, Claude Code, Codex and Cursor list and connect to the entries this writes. Not tested: Copilot CLI, Gemini CLI, OpenCode and Antigravity reading them.How to review
Read
writer.go(Upsert,keepUserKeys,mergeEnv,checkWritable,checkEdit,lockConfig) andtoml.go(put,dropTable,appendBlock) closely.json.gois patch plumbing over hujson. TheProtectMultilineStringsrename inlibs/localenvis mechanical.This pull request and its description were written by Isaac.