Repository navigation
Add aitools mcp serve, the stdio proxy for managed MCP servers - #7550
Conversation
694ba3c to
c3af024
Compare
|
Commit: Regression test reportSelected 2 changed tests (limit=20, 0 not selected) 2 tests: regression=2 unreleased=0 coverage=0 failing=0
TestAccept/cmd/aitools/mcp/serve-binding-errors: branch ✅ | main (65f3314) ❌ | latest (v1.27.0) ❌main (65f3314): latest (v1.27.0): TestAccept/cmd/aitools/mcp/serve-passthrough: branch ✅ | main (65f3314) ❌ | latest (v1.27.0) ❌main (65f3314): latest (v1.27.0): |
✅ Integration test reportCommit: ad9ad88
Top 36 slowest tests (at least 2 minutes):
|
4ecc28f to
2f43e8a
Compare
eec9d1a to
5b1d279
Compare
2f43e8a to
3c43783
Compare
5b1d279 to
9187003
Compare
4688f64 to
ed4190f
Compare
9187003 to
48fac6f
Compare
6634449 to
5c42df0
Compare
7064bbc to
10450c9
Compare
5c42df0 to
ad029dd
Compare
10450c9 to
9ec98a0
Compare
ad029dd to
851225d
Compare
9ec98a0 to
1d09e53
Compare
851225d to
fd9cdef
Compare
1d09e53 to
01b4249
Compare
fd9cdef to
6594052
Compare
11e6e00 to
2a9b99f
Compare
6594052 to
1174f67
Compare
2a9b99f to
c7d75cb
Compare
2eafa10 to
0ca1f43
Compare
0b9a961 to
e653041
Compare
f2e0cd4 to
e41d98d
Compare
3294173 to
752ff9c
Compare
anton-107
left a comment
There was a problem hiding this comment.
Requesting changes for one blocking issue, posted inline: a profile with no usable credential (config.ErrCannotConfigureDefault) gets a "temporary failure, try again" message with exit 1 and no login hint. I reproduced it with this branch's binary.
Verified on c5aa08b in a separate worktree: unit tests for cmd/aitools, cmd/root, libs/aitools/mcp/... and libs/testserver pass, TestAccept/cmd/aitools/mcp passes, and golangci-lint on the changed packages reports 0 issues.
Merge danger: low, a two-way door. serve is hidden, the whole stack is unmerged, and the cmd/root additions (the StdoutCarriesProtocol annotation, ExitStatusError, the logger restore) are internal. The flags and exit codes do become a contract once a release writes entries that call serve, so settle them before the stack ships.
Non-blocking notes:
rebindingAuthenticatere-reads.databrickscfgon every request, whileauth loginandconfigurerewrite it in place (ini.File.SaveTousesos.WriteFile, which truncates and then writes). A read that lands in that window sees no profile. That givesBindProfileNotFound,bindingChangedreturns true, and serve stops for good with "the binding changed". The window is microseconds, but a harmless login to another profile can then kill the server. Retrying the check once before stopping would close it.- On the no-install-record path,
bindingFormakes up an entry. A later drift or missing-profile error then saysremove the server with: databricks aitools mcp remove --names system.ai.github, but there is no record to remove. Bindpasses the native path toLoginCommand, while the runtime hints usefilepath.ToSlash. On Windows the startup and runtime hints spell the same file differently.
|
The non-blocking notes are in too:
|
anton-107
left a comment
There was a problem hiding this comment.
Round 2 (c5aa08b..029c1be): the blocking finding is fixed.
I re-ran round 1's case with this head's binary in an isolated HOME. A [legacy] profile with only host = https://legacy.test and no cached token now exits 2 with Run: databricks auth login --profile legacy. At c5aa08b the same profile exited 1 with "a temporary failure ... try again". The auth_type = databricks-cli control exits 2 on both heads.
The claim that the sentinel can't come from a later request holds. ErrCannotConfigureDefault only comes from Credentials.Configure in the SDK's authenticateIfNeeded. Bind runs that on the same cfg that serve then uses for every request, and the provider it caches there is never reset. A later failure comes from the token source.
The 50 ms retry only runs after a check has already failed as a changed binding, so the normal path adds no latency. A re-point that persists still stops serve after one more read; TestServeProxyRepointedProfileSendsNoToken still passes.
go test -count=1 ./cmd/aitools/... ./libs/aitools/mcp/... and TestAccept/cmd/aitools/mcp/serve* pass locally.
986009a to
f9157d1
Compare
029c1be to
171374b
Compare
f9157d1 to
088569a
Compare
One commit for the restack onto main. The individual commits, up to 029c1be, are in the pull request's history. Co-authored-by: Isaac <no-reply@databricks.com>
171374b to
ad9ad88
Compare
❌ Integration test reportCommit: cb94103
1226 interesting tests: 1203 MISS, 9 RECOVERED, 6 PANIC, 6 KNOWN, 2 FAIL
Top 50 slowest tests (at least 2 minutes):
|
Part of a stack that adds the hidden `databricks aitools mcp` command group, on databricks#7550 (`aitools mcp serve`). It adds provider sign-in for external MCP services such as GitHub, which each user signs in to before the service's tools work. ## Changes - `mcp login <service>` prints the sign-in URL, opens it through `libs/browser`, and waits up to 5 minutes. Server and network errors while polling are retried until the timeout, which names the last one. An already signed-in caller gets "Already signed in" and no URL. - `mcp logout <service>` deletes the caller's credential, even when its service is no longer visible. `login` and `logout` refuse a changed profile before any request. - A FAILED or DEGRADED credential is deleted before the sign-in page opens. - Service names are refused only when they contain `/` or a control character. Other URL syntax, such as `?`, `#` and `%`, is escaped in the request and is allowed. - `mcp add --login` signs in to each sign-in-required server that reached an agent. Its JSON gains a per-server `sign_in` object, given only to servers that reached an agent; a server that does not need sign-in gets `NOT_REQUIRED`. A failed sign-in is reported in both `servers[].sign_in.error` and the top-level `error`, which is intended, so consumers can read either. The command exits 1 and keeps the installs. - `mcp list --check` checks each profile's token, then each server. A drifted server or one that cannot be read is skipped, and says why. Each check request ends after 5 seconds. A needed sign-in names the login command. - Hints for a server pinned to a config file set `DATABRICKS_CONFIG_FILE`. `libs/browser` gains `OpenOutputToStderr`, and its default opener no longer waits for a browser its launcher leaves running. ## Why A sign-in is judged on the attempt that runs, so a FAILED or DEGRADED credential left by an earlier attempt is deleted before the page opens. A failed poll does not mean the user has left the provider's page, so it does not end the wait. One unreachable workspace held `list --check` for the SDK's five-minute retry budget. ## Decisions to review - `SignIn` returns `(alreadySignedIn, urlShown bool, err error)`, not a state. The command derives the JSON state and URL from them. - Polls retry 5xx and network errors. Cost: a longer wait while the error persists, up to the timeout. - Service names with `?`, `#` or `%` are accepted, rather than refused locally. Cost: such names reach the API. - `list --check` sets `DATABRICKS_CONFIG_FILE` in the process environment while it checks a profile, because the token store reads the process environment. - `logout` sends the DELETE before it looks up the service, so a credential for a deleted service can be removed. Cost: a mistyped name costs one request. ## Tests - `TestOpenDoesNotWaitForLauncherChild` in `libs/browser` failed before the fix. `cmd/aitools`: `TestAddChecksBoundsAProfileThatNeverAnswers` hung past five minutes before the fix. - Polls through server errors and the timeout's last error are covered in `libs/aitools/mcp`. - `TestSignInDeletesDegradedCredential` covers the DEGRADED deletion. `TestPathSafeName` and `TestServiceNameAllowsEscapedURLSyntax` cover the name check. `TestSignInInstalledRecordsTheSignInOfEachServer` covers `sign_in` for a server that reached no agent. - Acceptance `list-check` and `login-flips-active` pass. - `go test -race ./cmd/aitools/ ./libs/aitools/mcp/...` and `GOOS=windows go vet` on the same packages pass. - Not tested: the 5-minute timeout through the CLI (unit tests use milliseconds), and a real provider sign-in. ## How to review Refactors that keep behavior: the sign-in helpers shared by `login`, `logout` and `add --login` (`libs/aitools/mcp/login.go`, `cmd/aitools/mcp_login.go`). Read closely: the poll loop in `libs/aitools/mcp/login.go`, the bounded checks in `cmd/aitools/mcp_list.go`, and the default opener in `libs/browser/browser.go`, which changes how `browser.Open` waits. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
This pull request is part of a stack that adds the hidden
databricks aitools mcpcommand group. It builds on #7540 and addsaitools mcp serve, the stdio command that each installed MCP entry runs.Changes
aitools mcp serve --profile <p> --service <full name> [--agent <name>], hidden. It runs the binding checks, then bridges stdin and stdout to the service endpoint with the profile's credentials.--outputand a log file on stdout are refused. Binding failures exit 2, except a token failure at startup that is not a rejection, such as a network error or a 5xx. That one says the failure is temporary, gives no login hint, and exits 1.mcp addstill exits 1 on a binding failure.auth_typeand no cached token, gets the login hint and exits 2, as a rejected token does.DATABRICKS_CONFIG_FILE, with forward slashes at startup as at runtime.mcp.AuthenticateBoundtakes that file, so the bind checks the same file the hint names.databricks auth login --profile P --host B, fails that request before any token is fetched, with the binding error. A changed binding is checked again 50 ms later before the request fails, since a write that replaces.databrickscfgcan hide the profile for a moment. Serve then stops the same way as after a rejected token: later requests fail at once, and it exits 2.removeto suggest. Its per-token check holds the profile to the host and workspace serve bound to at start. A missing profile gets the startup setup message, and a host change says to restart the server, which binds the current host.cmd/root:Executecloses the log file and restores the default logger on return, also after a panic.Why
Each entry that
mcp addwrites runsaitools mcp serve. It binds to one profile before it reads stdin, so a profile that points elsewhere at start fails with a message instead of sending a token to another host. The token cache is keyed by profile name and records no host, so the binding is checked again before each token fetch, not only at start.Decisions to review
invalid_grantorinvalid_client, or the CLI's U2M client reporting an invalid refresh token. Everything else fails only its request, including anything the check cannot classify. The reason is a laptop that wakes before the network is back: stopping there leaves the agent with a dead server until the entry restarts. Cost: an unclassified error keeps failing requests instead of stopping serve.--log-file stdoutworks on a Unix terminal. Windows refuses it, because a console is not a regular file there.Tests
cmd/aitools: a rejected token stops the proxy with exit 2, the hint, and no later token request. A transient failure fails only its request, and the next request succeeds.TestCredentialRejectedchecks each error shape.TestRefreshGuardStopsOnBindingChangeandTestServeProxyRepointedProfileSendsNoTokencheck that a re-pointed profile fails before any token fetch and stops serve.TestCheckBindingWithRetrycovers the second check.TestServeBindFailureTokenStatuscovers the startup exit codes, including a missing credential, which exits 2 with the hint.TestServeBindHostOnlyProfileNeedsLoginruns the real credential chain on such a profile.TestBindingCheckWithoutRecordChecksStartHostandTestCheckUnrecordedBindingWordsChangesWithoutRecordcover the messages for a server with no install record.cmd/root: the stdout refusal, and logger restore after a panic and a normal return.serve-passthroughchecks each stdout line is one JSON-RPC message;serve-binding-errorschecks exit 2 and the stderr text.go test ./cmd/aitools/ ./libs/aitools/mcp/...and theTestAccept/cmd/aitools/mcp/serveacceptance tests pass.go vetand a test build ran, so the console behavior is from reading the code.How to review
Refactors that keep behavior:
servetakesadd's flag names frommcpconfig, and its tests and helpers were reshaped incmd/aitools/mcp_serve_test.goandlibs/testserver. Read closely:cmd/root/root.goandcmd/root/stdout_protocol*.gofor the stdout refusal,libs/aitools/mcp/proxy/bridge.goandrun.gofor the token failure paths, andbindingCheck,checkBindingWithRetry, and the exit status incmd/aitools/mcp_serve.go.This pull request and its description were written by Isaac.