Repository navigation
Add sign-in challenges and the authenticate tool to aitools mcp serve - #7553
Conversation
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/cmd/aitools/mcp/serve-authenticate-tool: branch ✅ | main (4285bf9) ❌ | latest (v1.27.0) ❌main (4285bf9): latest (v1.27.0): |
✅ Integration test reportCommit: 22ff1dc
Top 36 slowest tests (at least 2 minutes):
|
62d20d0 to
d7c9aec
Compare
38cdd13 to
1d8e6b3
Compare
d7c9aec to
d2dd977
Compare
1d8e6b3 to
7b98424
Compare
d2dd977 to
2a52223
Compare
7b98424 to
cb80384
Compare
0aa7987 to
a08bc42
Compare
1c6aee6 to
7a9dca9
Compare
a08bc42 to
4a8c08c
Compare
1abf7d0 to
6b41591
Compare
4a8c08c to
8ef469f
Compare
6b41591 to
48208cf
Compare
8ef469f to
d779ae1
Compare
922dad8 to
7e7f6c7
Compare
43bb551 to
e441c85
Compare
7e7f6c7 to
865e2f5
Compare
e441c85 to
60bf035
Compare
0952a73 to
1f42a46
Compare
60bf035 to
109bc73
Compare
1f42a46 to
d94c24d
Compare
661f21e to
af69ee8
Compare
14af751 to
9abcc11
Compare
8c7ea9b to
160e62d
Compare
9abcc11 to
79d56db
Compare
|
|
||
| // A call that the client cancelled leaves the logout answer to the next call. | ||
| h.stateMu.Lock() | ||
| stillChallenged := h.stillChallenged && ctx.Err() == nil |
There was a problem hiding this comment.
After stillChallenged is cleared (line 489) and its answer is returned (line 496), the code falls through to check ctx.Err() (line 498). But between line 487 (h.stateMu.Lock()) and line 496, if ctx.Err() != nil the stillChallenged flag was left as-is and we return the logout-answer tool result. That's fine. However on the non-stillChallenged path, when ctx.Err() != nil at line 498, h.stillChallenged was already cleared at line 489 and never restored. A cancelled call that acquired the turn and found stillChallenged true will clear the flag, write no answer, yet the next authenticate won't see stillChallenged either — so the logout answer is lost. The description says "A call that the client cancelled leaves the logout answer to the next call," but the flag is cleared unconditionally. Should the h.stillChallenged = false (line 490) be inside the if stillChallenged block only when ctx.Err() == nil, i.e. move the clearing to where the tool result is returned?
There was a problem hiding this comment.
The flag is cleared only while the call's context is live: stillChallenged is computed as h.stillChallenged && ctx.Err() == nil, and the field is cleared only inside if stillChallenged. A cancelled call sees stillChallenged as false and leaves the field set, so the next call still gets the logout answer. No change.
| }, | ||
| Stderr: h.stderr, | ||
| }) | ||
| h.pageCancelled = urlShown && err != nil && ctx.Err() != nil |
There was a problem hiding this comment.
h.pageCancelled is read and written under the turn semaphore but not under stateMu. Meanwhile the doc comment on turn says pageCancelled is read and written under it, which is correct. Just flagging: if pageCancelled is ever accessed outside of the turn (it isn't today), this would race. Looks good as-is, but the field lives on a struct whose other fields are guarded by stateMu — a future reader might add a read of pageCancelled under stateMu by mistake. Would a one-line comment on the field itself ("guarded by turn, not stateMu") help?
There was a problem hiding this comment.
Added it to the field's own comment in b68daa5.
| return serveFailure(cmd, err, 1) | ||
| } | ||
| } | ||
| challenge, err := serveChallenge(ctx, serveTarget{w: w, entry: entry, profile: profileName, service: f.service, pinnedFile: pinnedFile}, cmd.ErrOrStderr()) |
There was a problem hiding this comment.
When entry == nil && !entry.SignInRequired is false (i.e. entry != nil && !entry.SignInRequired), w is nil and serveChallenge is called with t.w == nil. That path returns nil, nil (line 374), so t.w is never dereferenced — but this is subtle. A future change to serveChallenge that reads t.w on the !signInRequired path would panic. Is it worth a comment on serveTarget.w saying it may be nil when the entry says no sign-in is needed?
| timer := time.AfterFunc(notificationWait, release) | ||
| defer timer.Stop() | ||
| } | ||
| if c.answerLocally(ctx, line) { |
There was a problem hiding this comment.
answerLocally runs after the notification gate timer is started (line 310) but before the idle watchdog (line 317). A locally-answered request that blocks in signInHooks.authenticate for minutes will fire the notification timer, which releases the gate of the next notification. The gate release is a no-op for the locally-handled request itself, but it means the next notification in the queue proceeds immediately instead of waiting notificationWait after its predecessor. Is that intended? The bridge doc says notificationWait is so the upstream finishes its SSE before a notification is sent. If the local answer runs instead of the upstream, the upstream was never started, so releasing the gate early is arguably correct — but it could also let a notification race ahead of a still-running sign-in's list_changed. Can you confirm that the ordering is safe here?
There was a problem hiding this comment.
This case can't happen. release is only non-nil for notifications (handle passes nil for requests), and Local only handles the authenticate tools/call request. So a request answered locally never starts the notification timer and never holds a gate, and the next notification waits only for the notification before it. No change.
anton-107
left a comment
There was a problem hiding this comment.
Reviewed only what this PR adds on top of aitools-mcp/07-login. I found nothing that blocks the merge. The change is a two-way door: the command group is hidden, no persisted state or config format changes, and the shared refactors (signIn's options, the rpcResponse envelope, browser/exec getting a no-stdin opener) leave existing callers' behavior as it was.
Verified locally at b68daa5: go test -race on libs/aitools/mcp/..., cmd/aitools/..., libs/browser, libs/exec, libs/testserver, plus TestAccept/cmd/aitools/mcp, all pass.
Last round: the four bot comments are all resolved. The pageCancelled and serveTarget.w comments were added in b68daa5. The stillChallenged and notification-gate concerns were correctly rebutted: the flag is cleared only inside if stillChallenged, which requires a live ctx, and release is nil for requests.
Non-blocking notes:
- The description says "The proxy never deletes a credential". The authenticate tool runs
mcp.SignIn, though, and that deletes a FAILED credential before showing the page (libs/aitools/mcp/login.go, theProvisioningInfoStateFailedcase). The behavior is fine; "never deletes an active credential" would be more accurate. - After an "Already signed in" answer, a later call challenge sets
stillChallenged. Thetools/callanswer (signInRequired) still says "Call the authenticate tool, or run:<mcp login>". In that state,mcp loginjust reports "Already signed in" again, and onlyauthenticategives the logout command. Naming the logout command insignInRequiredwhilestillChallengedis set would save a round trip. serveSignInresolves the workspace ID withauth.ResolveWorkspaceIDeven when the install record already hasentry.WorkspaceID. It runs only once per session, so this is minor.
|
Thanks!
|
6957247 to
edbd091
Compare
57552af to
e6ae53a
Compare
One commit for the restack onto main. The individual commits, up to 6957247, are in the pull request's history. Co-authored-by: Isaac <no-reply@databricks.com>
edbd091 to
22ff1dc
Compare
❌ Integration test reportCommit: de394ed
1168 interesting tests: 1134 MISS, 14 KNOWN, 9 RECOVERED, 6 FAIL, 5 PANIC
Top 50 slowest tests (at least 2 minutes):
|
…tabricks#7595) This pull request is part of the stack that adds the hidden `databricks aitools mcp` command group; its parent is databricks#7553. `aitools mcp add` and `remove` now change the Claude Code, Gemini CLI and Copilot CLI configs through each agent's own MCP command, when that CLI is on PATH. Codex CLI configs are always written directly. ## Changes - A new entry runs the agent's add command when its config file exists: `claude mcp add-json --scope user`, `gemini mcp add --scope user`, or `copilot mcp add --tools *`. A config file that does not exist yet is written directly, with mode 0600, since the agent's CLI would create it with its own permissions. Afterwards the config file holds the entry asked for, whatever the exit status: a differing entry is replaced, and a missing one is added directly. An entry that already exists is updated by the direct writer. - `remove` runs the CLI's remove for an entry we wrote, when the agent reads that file now. A CLI failure, or an entry left behind, falls back to the direct writer. - Symlinked configs, names that start with `-`, and Codex always use the direct writer. - The CLI runs in an empty directory with a 30-second timeout, in the foreground process group so Ctrl-C reaches it. - For an agent that locks its config (Claude Code, see databricks#7539), only the direct writer takes the lock. It is never held while the agent's CLI runs, since that CLI takes the same lock. ## Why An agent's own CLI writes its config with its own locking and layout, and follows the agent when its file format changes. Codex is the exception. With Codex CLI 0.167.1, `codex mcp add` and `codex mcp remove` rewrote every `[mcp_servers.*]` table of an 11-server config: they dropped `enabled = true` keys, collapsed multi-line arrays, turned `10` into `10.0`, and converted CRLF to LF. ## Decisions to review - The file decides, not the exit status. Cost: a CLI that writes another file keeps its entry there, and ours is also added to the agent's file. - A differing entry is replaced, not refused. Cost: our launch keys win over the CLI's choices. - Existing entries use the direct writer, since the add commands replace whole entries. Cost: two paths write one file. - Codex always uses the direct writer. Cost: comments inside a Databricks server's own tables are lost on either path. - A hung CLI costs the 30-second timeout for each server. - On Windows, a `.cmd` shim gets only arguments without spaces, double quotes or `cmd.exe` special characters. Other arguments use the direct writer. ## Tests - `libs/aitools/agents` covers argv per agent, including a Windows path with spaces. `agentconfig/cli_test.go` covers each outcome of an add, `TestAgentCLIRunsWithoutTheLock` the lock, symlinks, missing config files, dash names and the timeout, and the Codex direct writer. `libs/process` covers `WithoutEnv`, which ignores case on Windows. - `acceptance/cmd/aitools/mcp/add-remove-agent-clis` covers each agent's argv and the files it wrote. - `./task test-exp-aitools` passes with `BROWSER` unset and with `BROWSER=none`. - Not tested: the Copilot and Gemini binaries, which are not installed here, so their argv follows their docs and source. Windows runs, which CI covers. `-inprocess` mode, which already fails on the base for these tests. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
This pull request is part of a stack that adds the hidden
databricks aitools mcpcommand group, on top of #7551. A service that needs provider sign-in rejectstools/listuntil the user signs in, so the agent has no tool to start it. The proxy now offers oneauthenticatetool that runs the sign-in.Changes
tools/listreturn onlyauthenticate. A challenge is a JSON-RPC-32006, a-32042answer with a sign-in URL, or an error result that names a sign-in URL. A-32042URL is accepted on the profile's host, or on any host whoseo=orw=query names the bound workspace.databricks auth loginhint. A server that needs no sign-in has no such handling, so its 401 passes through unchanged.tools/callnamesauthenticateand the login command. Iftools/liststill passes,authenticateis added to its first page until a call succeeds or the sign-in completes.tools/callis answered with themcp logoutcommand to run first, since neitherauthenticatenorlogincan fix it then.authenticaterunslogin's sign-in, sends progress naming the page, and sendstools/list_changedwhen signed in. A timeout names the URL, the last failed check and the login command.mcp add, bymcp loginand by the authenticate tool's sign-in.mcp listdoes not read it.SignIndeletes a FAILED or DEGRADED one before it shows the page.Why
Discovery fails before sign-in, so the agent can only start a sign-in through a tool that exists before it. A rejected Databricks token is not fixed by a provider sign-in, so it gets the login hint instead.
Decisions to review
mcp logout.Tests
libs/aitools/mcp/proxy/challenge_test.gocovers the challenge bodies, 401s, call-only challenges, the logout answer, page reopening, queued and cancelled calls, and progress.TestSignInRequiredNamesTheLogoutWhileStillChallengedcovers the logout answer after "Already signed in".TestServeSignInUsesTheRecordedWorkspaceIDcovers the workspace ID taken from the record.serve-authenticate-toolruns the stdio flow withBROWSER=none.go test -race ./cmd/aitools/ ./libs/aitools/mcp/..., theTestAccept/cmd/aitools/mcpacceptance tests andGOOS=windows go veton the same packages pass.-32042challenge, and a real browser.How to review
Refactors that keep behavior: the sign-in path that
mcp loginand the proxy share (libs/aitools/mcp/login.go,cmd/aitools/mcp_serve.go), and the shared JSON-RPC response envelope inlibs/aitools/mcp/proxy/. Read closely:libs/aitools/mcp/proxy/challenge.go(new), itsResultmethod and the 401 rule, andserveChallengeincmd/aitools/mcp_serve.go.This pull request and its description were written by Isaac.