Skip to content

Add streamable HTTP bridge for the aitools MCP proxy - #7540

Merged
simonfaltum merged 1 commit into
mainfrom
aitools-mcp/06a-bridge
Nov 19, 2026
Merged

simonfaltum merged 1 commit into
mainfrom
aitools-mcp/06a-bridge

Conversation

@simonfaltum

@simonfaltum simonfaltum commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

This is part of a stack, on top of #7549, that adds the hidden databricks aitools mcp command group. This PR adds the stdio-to-HTTP bridge that serve will use. Nothing calls it yet.

Changes

  • The new Bridge type's Run POSTs each stdin message's exact bytes to the endpoint and writes each answer to stdout as one line. Requests run concurrently.
  • A message waits for the notification before it, up to 2 s (notificationWait). Each stalled notification can hold the next message for one such wait, so a chain of stalled notifications adds up to 2 s per notification.
  • A request without a matching response gets a -32000 error. A non-2xx body passes through when it is that request's response, such as -32006 under 401.
  • A 404 for a request with a session id starts a new session. initialize and notifications/initialized are replayed silently, and the old session is replaced only after the replay succeeds.
  • An initialize that opens a session but whose stream ends, goes idle or is cut before the client gets its answer is followed by a DELETE for that session. Without this, the server kept the session open until it expired. The session is adopted only when the answer has a result.
  • SSE accepts LF, CR and CRLF line endings and one leading BOM, and limits an event's data to 64 MiB.
  • A hook's output is checked before any of it is written. Answers to notifications and client responses are not forwarded.
  • Bridge.Authenticate is required and must not be nil. Bridge.Transport is optional. Nil means http.DefaultTransport.

Why

serve relays MCP responses over stdio. The package is separate so an in-process fake endpoint can test it. A probe of a Databricks managed MCP endpoint returned Mcp-Session-Id on initialize, and tools/list worked with or without the session headers. The bridge sends both, as the MCP transports specification requires. The transport is injectable so that a profile's TLS settings reach the upstream client.

Decisions to review

  • The 10-minute idle watchdog starts before the POST, so it also bounds the wait for response headers. Cost: a plain JSON answer slower than 10 minutes is cut. Starting it after the headers would let a server that never answers hold the request forever.
  • A request waits up to 2 s for the notification before it, then goes out anyway. Cost: a chain of slow notifications delays the stream by 2 s each.
  • The replay restores only initialize and notifications/initialized. Logging levels and subscriptions set later are lost on a session restart.

Tests

  • Session recovery and framing: TestRunAnswersPingDuringReplay, TestRunDeletesSessionReplacedByInitialize, TestReadEvents (CR, BOM), TestReadEventsLimitsEventData.
  • Unanswered initialize: TestRunDeletesSessionOfUnansweredInitialize covers a stream that ends, goes idle, or is cut.
  • Ordering: TestRunHoldsRequestUntilNotificationWaitFromItsPost, TestRunWritesNothingWhenHookOutputIsInvalid.
  • go test -race -count=5 ./libs/aitools/mcp/proxy/..., GOOS=windows go vet and golangci-lint pass on the package.
  • Not tested: sign-in challenges and -32042, which come in a later change. The TLS settings are not exercised, since serve does not build the transport yet. No acceptance test, because no command is wired.

This pull request and its description were written by Isaac.

@simonfaltum
simonfaltum changed the base branch from main to aitools-mcp/04-add November 12, 2026 12:12
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from beafb3d to ad50549 Compare November 12, 2026 12:12
@simonfaltum
simonfaltum changed the base branch from aitools-mcp/04-add to aitools-mcp/05-list-remove November 12, 2026 12:25
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from ad50549 to 4ecc28f Compare November 12, 2026 12:25
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch from 7db8c31 to 9967822 Compare November 12, 2026 14:35
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from 4ecc28f to 2f43e8a Compare November 12, 2026 14:36
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch from 9967822 to 3d361ee Compare November 12, 2026 17:07
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from 2f43e8a to 3c43783 Compare November 12, 2026 17:08
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch 2 times, most recently from 6e86c42 to 30b949b Compare November 12, 2026 20:05
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch 2 times, most recently from 4688f64 to ed4190f Compare November 12, 2026 22:00
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch from eb0b033 to 95c5e54 Compare November 13, 2026 15:05
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from 6634449 to 5c42df0 Compare November 13, 2026 15:05
@pietern pietern added the aitools AI tools (databricks aitools) related changes label Nov 16, 2026
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from 5c42df0 to ad029dd Compare November 16, 2026 12:47
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch from 789b8ae to f61d0f6 Compare November 16, 2026 13:19
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch 2 times, most recently from 851225d to fd9cdef Compare November 16, 2026 13:32
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch from f61d0f6 to 2427a96 Compare November 16, 2026 13:32
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from fd9cdef to 6594052 Compare November 16, 2026 14:49
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch 2 times, most recently from 451d8a7 to 12cb417 Compare November 16, 2026 15:14
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from 6594052 to 1174f67 Compare November 16, 2026 15:14
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch from 12cb417 to 59de987 Compare November 16, 2026 15:53
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from 1174f67 to 2eafa10 Compare November 16, 2026 15:53
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch from 59de987 to 406ef75 Compare November 17, 2026 10:52
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from 2eafa10 to 0ca1f43 Compare November 17, 2026 10:52
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch from 220a20c to 607c6d0 Compare November 17, 2026 15:11
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch 4 times, most recently from 3294173 to 752ff9c Compare November 17, 2026 19:58
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 752ff9c

Run: 44164731626

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 28 12:35
🔄​ aws-windows-latest-4core-16gb 1 1 271 26 9:29
💚​ azure linux-2core-8gb 1 269 28 12:36
💚​ azure-windows-latest-4core-16gb 1 271 26 8:02
🔄​ gcp linux-2core-8gb 1 1 269 28 12:39
💚​ gcp-windows-latest-4core-16gb 1 272 26 8:44
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestSyncFullFileSync ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
🔄​ TestSyncIncrementalSyncPythonNotebookToFile ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
Top 17 slowest tests (at least 2 minutes):
duration env testname
4:15 gcp-windows-latest-4core-16gb TestImportDir
3:56 aws linux-2core-8gb TestAccept
3:53 gcp linux-2core-8gb TestAccept
3:49 azure linux-2core-8gb TestAccept
3:23 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
3:00 azure-windows-latest-4core-16gb TestAccept
2:55 aws-windows-latest-4core-16gb TestLock
2:55 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsDelete
2:53 gcp-windows-latest-4core-16gb TestAccept
2:50 aws-windows-latest-4core-16gb TestAccept
2:23 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:22 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:21 gcp linux-2core-8gb TestFilerRecursiveDelete/workspace_files
2:18 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:17 azure linux-2core-8gb TestFilerWorkspaceNotebook/scalaJupyterNb.ipynb
2:08 aws-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files_extensions
2:03 gcp-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files

@simonfaltum
simonfaltum marked this pull request as ready for review November 18, 2026 16:20
@simonfaltum
simonfaltum requested review from a team and lennartkats-db as code owners November 18, 2026 16:20

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me — no issues found.

@anton-107 anton-107 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I didn't find anything I'd block on. This is a two-way door: the package has no callers yet, keeps no persisted state, and sits behind a hidden command.

What I checked: go test -race -count=3 on the package and GOOS=windows go vet both pass. I also fuzzed eventLines/readEvents against a reference SSE parser with random chunk boundaries (LF, CR, CRLF, leading BOM, comments, other event types; 20k inputs) and numberKey against big.Rat equality (50k pairs). No mismatches. I read the rejoin and replay path, the provisional session, the gates and the shutdown ordering end to end and found no races or lost answers.

Non-blocking notes:

  1. Unadopted initialize session never ends. If an initialize gets a 2xx with Mcp-Session-Id but its stream ends before the result, or idle or shutdown cuts it, that session is never DELETEd. The DELETE deferred in deliverInitialize runs only once an answer arrives. Probe: a 200 SSE with Mcp-Session-Id: s-new and only a comment event. The client gets the -32000 and no DELETE is ever sent for s-new. Server expiry cleans it up, but forward could end x.session when !x.answered.
  2. Authenticate is required, but the field doesn't say so. A nil value panics in newRequest (bridge.go:365), while Transport documents its nil default. One sentence on the field would make the contract clear before serve wires it.
  3. SSE event left open at EOF. One without a closing blank line is still delivered (sse.go:54). The SSE spec discards it. This is lenient and documented, so I'm only noting it.

@simonfaltum

Copy link
Copy Markdown
Member Author

Thanks!

  1. Fixed in 2df1750: a session that an unanswered initialize opened is DELETEd when its stream ends, goes idle or is cut. TestRunDeletesSessionOfUnansweredInitialize runs your probe for all three.
  2. The field says it's required now (986009a).
  3. Yes, that leniency is deliberate, so I've left it.

@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from 986009a to f9157d1 Compare November 19, 2026 11:42
@simonfaltum
simonfaltum force-pushed the aitools-mcp/05-list-remove branch 2 times, most recently from 270baac to 710f6db Compare November 19, 2026 13:39
@simonfaltum
simonfaltum requested a review from a team as a code owner November 19, 2026 13:39
Base automatically changed from aitools-mcp/05-list-remove to main November 19, 2026 14:15
One commit for the restack onto main. The individual commits, up to 986009a, are in the pull request's history.

Co-authored-by: Isaac <no-reply@databricks.com>
@simonfaltum
simonfaltum force-pushed the aitools-mcp/06a-bridge branch from f9157d1 to 088569a Compare November 19, 2026 14:16
@simonfaltum
simonfaltum added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit 65f3314 Nov 19, 2026
23 checks passed
@simonfaltum
simonfaltum deleted the aitools-mcp/06a-bridge branch November 19, 2026 14:27
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: 65f3314

Run: 44544119334

Env 💥​PANIC ❌​FAIL 🟨​KNOWN 🤯​MISS 💚​RECOVERED ✅​pass 🙈​skip Time
🤯​ aws linux-2core-8gb 715 15 900 1407 320:00
💥​ aws-windows-latest-4core-16gb 1 705 15 825 1435 320:00
💥​ azure linux-2core-8gb 1 2 708 13 886 1415 320:00
💥​ azure-windows-latest-4core-16gb 1 2 6 559 14 944 1443 320:00
💥​ gcp linux-2core-8gb 1 912 15 689 1413 320:00
💥​ gcp-windows-latest-4core-16gb 1 559 17 956 1441 320:00
1333 interesting tests: 1314 MISS, 6 KNOWN, 6 RECOVERED, 5 PANIC, 2 FAIL
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🤯​ TestAccept 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_json/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/target_override/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info 🤯​M 🤯​M 🤯​M ✅​p 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info/DMS= 🤯​M 🤯​M 🤯​M ✅​p 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info/DMS=true 🤯​M 🤯​M 🤯​M ✅​p 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M ✅​p
🤯​ TestAccept/bundle/deploy/immutable/basic 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/basic/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/breakglass 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/breakglass/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/can-manage 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/can-manage/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/destroy-redeploy 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/destroy-redeploy/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/dev-mode 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/dev-mode/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/legacy-state 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/legacy-state/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/missing-snapshot 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/missing-snapshot/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/permissions-change 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/permissions-change/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/select 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/select/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=true/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=true/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
Top 50 slowest tests (at least 2 minutes):
duration env testname
17:13 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
15:20 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
14:09 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
14:05 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
14:03 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
13:57 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
13:47 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
13:34 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
12:54 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
12:17 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
12:04 azure-windows-latest-4core-16gb TestAccept/bundle/resources/materialization_sets/immutable/DMS=
11:52 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
11:39 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:56 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
10:52 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=
10:51 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true
10:50 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
10:31 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=true
10:29 azure-windows-latest-4core-16gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=true
10:26 azure-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/integration_classic/DMS=/UV_PYTHON=3.10
10:25 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
10:04 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:04 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/materialization_sets/batch/DMS=
10:04 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
9:56 azure-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=no/DMS=/NBOOK=yes/PY=no/READPLAN=
9:50 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
9:43 azure-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=no/DMS=/NBOOK=no/PY=yes/READPLAN=
9:40 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
9:34 azure linux-2core-8gb TestAccept/bundle/templates/default-python/combinations/classic/DLT=yes/DMS=/NBOOK=yes/PY=yes/READPLAN=
9:33 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:26 azure linux-2core-8gb TestAccept/bundle/resources/materialization_sets/immutable/DMS=
9:10 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
9:00 azure-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=yes/DMS=/NBOOK=no/PY=no/READPLAN=
8:58 gcp linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
8:41 gcp-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/integration_classic/DMS=/UV_PYTHON=3.11
8:34 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
8:31 aws-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=yes/DMS=/NBOOK=yes/PY=no/READPLAN=1
8:29 azure-windows-latest-4core-16gb TestAccept/bundle/resources/materialization_sets/batch/DMS=
8:25 azure linux-2core-8gb TestAccept/bundle/templates/default-python/combinations/classic/DLT=no/DMS=/NBOOK=yes/PY=yes/READPLAN=
8:22 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
8:20 gcp-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=no/DMS=/NBOOK=yes/PY=no/READPLAN=
8:20 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
8:17 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:12 azure linux-2core-8gb TestAccept/bundle/templates/default-python/combinations/classic/DLT=yes/DMS=/NBOOK=no/PY=yes/READPLAN=1
8:10 gcp-windows-latest-4core-16gb TestAccept/bundle/quiet-levels/DMS=true
8:06 aws-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/integration_classic/DMS=/UV_PYTHON=3.11
8:03 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
8:02 azure linux-2core-8gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=no/DMS=/NBOOK=yes/PY=yes/READPLAN=
8:01 azure linux-2core-8gb TestAccept/bundle/templates/default-python/integration_classic/DMS=/UV_PYTHON=3.12
8:00 azure-windows-latest-4core-16gb TestAccept/bundle/resources/materialization_sets/batch/DMS=true
(1185 table rows omitted to keep the report under 60000 bytes)

idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 20, 2026
…ricks#7550)

This pull request is part of a stack that adds the hidden `databricks
aitools mcp` command group. It builds on databricks#7540 and adds `aitools mcp
serve`, the stdio command that each installed MCP entry runs.

## Changes
- `aitools mcp serve --profile <p> --service <full name> [--agent
<name>]`, hidden. It runs the binding checks, then bridges stdin and
stdout to the service endpoint with the profile's credentials.
- Stdout carries only JSON-RPC, so `--output` and a log file on stdout
are refused. Binding failures exit 2, except a token failure at startup
that is not a rejection, such as a network error or a 5xx. That one says
the failure is temporary, gives no login hint, and exits 1. `mcp add`
still exits 1 on a binding failure.
- A profile with no usable credential at startup, such as one with a
host, no `auth_type` and no cached token, gets the login hint and exits
2, as a rejected token does.
- Login hints name the config file of a pinned profile through
`DATABRICKS_CONFIG_FILE`, with forward slashes at startup as at runtime.
`mcp.AuthenticateBound` takes that file, so the bind checks the same
file the hint names.
- A rejected token fails that request with the login hint, and serve
stops with exit 2. A transient token failure (a network error, a
timeout, a 5xx from the token endpoint) fails only that request, with no
login hint, and serve keeps running.
- Each token fetch first checks that the profile still names the bound
host and workspace, in the config file it was bound to. A profile
re-pointed during the session, for example by `databricks auth login
--profile P --host B`, fails that request before any token is fetched,
with the binding error. A changed binding is checked again 50 ms later
before the request fails, since a write that replaces `.databrickscfg`
can hide the profile for a moment. Serve then stops the same way as
after a rejected token: later requests fail at once, and it exits 2.
- A server with no install record has no `remove` to suggest. Its
per-token check holds the profile to the host and workspace serve bound
to at start. A missing profile gets the startup setup message, and a
host change says to restart the server, which binds the current host.
- `cmd/root`: `Execute` closes the log file and restores the default
logger on return, also after a panic.

## Why
Each entry that `mcp add` writes runs `aitools mcp serve`. It binds to
one profile before it reads stdin, so a profile that points elsewhere at
start fails with a message instead of sending a token to another host.
The token cache is keyed by profile name and records no host, so the
binding is checked again before each token fetch, not only at start.

## Decisions to review
- Only a rejected token or a changed binding stops the proxy: a 400 or
401 from the token endpoint with `invalid_grant` or `invalid_client`, or
the CLI's U2M client reporting an invalid refresh token. Everything else
fails only its request, including anything the check cannot classify.
The reason is a laptop that wakes before the network is back: stopping
there leaves the agent with a dead server until the entry restarts.
Cost: an unclassified error keeps failing requests instead of stopping
serve.
- A changed binding is checked twice, 50 ms apart, before serve stops.
Cost: a real re-point stops serve 50 ms later.
- A log file that is also stderr is allowed, so `--log-file stdout`
works on a Unix terminal. Windows refuses it, because a console is not a
regular file there.

## Tests
- `cmd/aitools`: a rejected token stops the proxy with exit 2, the hint,
and no later token request. A transient failure fails only its request,
and the next request succeeds. `TestCredentialRejected` checks each
error shape.
- `TestRefreshGuardStopsOnBindingChange` and
`TestServeProxyRepointedProfileSendsNoToken` check that a re-pointed
profile fails before any token fetch and stops serve.
`TestCheckBindingWithRetry` covers the second check.
- `TestServeBindFailureTokenStatus` covers the startup exit codes,
including a missing credential, which exits 2 with the hint.
`TestServeBindHostOnlyProfileNeedsLogin` runs the real credential chain
on such a profile.
- `TestBindingCheckWithoutRecordChecksStartHost` and
`TestCheckUnrecordedBindingWordsChangesWithoutRecord` cover the messages
for a server with no install record.
- `cmd/root`: the stdout refusal, and logger restore after a panic and a
normal return.
- Acceptance `serve-passthrough` checks each stdout line is one JSON-RPC
message; `serve-binding-errors` checks exit 2 and the stderr text.
- `go test ./cmd/aitools/ ./libs/aitools/mcp/...` and the
`TestAccept/cmd/aitools/mcp/serve` acceptance tests pass.
- Not tested: a real token refresh end to end, because the fake cannot
fail one. Windows: only `go vet` and a test build ran, so the console
behavior is from reading the code.

## How to review
Refactors that keep behavior: `serve` takes `add`'s flag names from
`mcpconfig`, and its tests and helpers were reshaped in
`cmd/aitools/mcp_serve_test.go` and `libs/testserver`. Read closely:
`cmd/root/root.go` and `cmd/root/stdout_protocol*.go` for the stdout
refusal, `libs/aitools/mcp/proxy/bridge.go` and `run.go` for the token
failure paths, and `bindingCheck`, `checkBindingWithRetry`, and the exit
status in `cmd/aitools/mcp_serve.go`.

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aitools AI tools (databricks aitools) related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants