Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions examples/codex/databricks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Databricks sandbox executor

Run an agent driven by this SDK with its `exec_command` / `write_stdin` /
`apply_patch` tools executing inside a Databricks sandbox.
`agents.sandbox.DatabricksStdioEnvironment` is the first-class `Environment` for
this - the Databricks counterpart of `agents.sandbox.DockerGateway`, following
the SDK's "remote execution" pattern (`docs/sdk/execution.md`).

## Usage example

```python
from databricks.sdk import WorkspaceClient
from agents import Agent, NativeProvider
from agents.sandbox import DatabricksStdioEnvironment

w = WorkspaceClient(profile="my-workspace")
# Provision "my-sandbox" (create_sandbox + poll until running); see runner.py.

environment = DatabricksStdioEnvironment.for_sandbox(
workspace_client=w,
sandbox_name="my-sandbox",
cwd="/home/sandbox-agent/work",
)
async with NativeProvider(
base_url="https://<workspace-host>/ai-gateway/codex/v1",
api_key=w.config.authenticate()["Authorization"].removeprefix("Bearer "),
supports_websockets=False,
) as provider:
agent = Agent(model="gpt-5.5", provider=provider, instructions="...")
async with agent.session(environment=environment) as session:
result = await (await session.submit("Inspect the workspace")).get_result()
print(result.final_output)
```

## Install and run the example

The SDK is distributed as a GitHub preview wheel (off PyPI). It ships a compiled
native runtime, so it needs a supported host: **Linux x86_64 with glibc 2.28 or newer**,
macOS arm64, or Windows x64.

```sh
pip install -r examples/codex/databricks/requirements.txt
```

Provide Databricks auth (a `.databrickscfg` profile), then run the example script
`runner.py` from the repo root. The model runs through the workspace's Databricks
AI Gateway by default, using the profile's own token, so no OpenAI key is needed:

```sh
SANDBOX_NAME=my-sandbox DATABRICKS_CONFIG_PROFILE=my-workspace \
PYTHONPATH=. python -m examples.codex.databricks.runner "Inspect the workspace"
```

`SANDBOX_NAME` accepts the bare id or the `sandboxes/<id>` resource name. Set
`SANDBOX_CREATE=1` (with `SANDBOX_IMAGE=...`) to provision the sandbox first. The
launcher starts `exec-server` on each session; it is killed when the sandbox
stops and relaunched on the next session.

To run the model through OpenAI directly instead of the Databricks AI Gateway,
set `OPENAI_BASE_URL` and `OPENAI_API_KEY` together (set both or neither), plus an
OpenAI model id your account can use:

```sh
SANDBOX_NAME=my-sandbox DATABRICKS_CONFIG_PROFILE=my-workspace \
OPENAI_BASE_URL=https://api.openai.com/v1 OPENAI_API_KEY=sk-... \
OPENAI_MODEL=gpt-5.3-codex \
PYTHONPATH=. python -m examples.codex.databricks.runner "Inspect the workspace"
```

## Prerequisites

1. **A running sandbox.** The application creates the sandbox (or reuses one),
starts it if stopped, and polls its status until running before opening the
environment - the exec-server is reached through `execute_command`, which
needs a running sandbox.
2. **A `codex` binary in the sandbox image.** The launcher runs
`codex exec-server --listen stdio` in the sandbox. The default sandbox image
already contains a compatible `codex` binary; for a
[user-supplied custom image](https://docs.databricks.com/aws/en/compute/serverless/sandbox-custom-image),
ensure `codex` is installed.

## Credentials and session lifetime

`DatabricksStdioEnvironment.for_sandbox` captures the workspace token into the
environment. With short-lived (OAuth / U2M) credentials that token expires (about
an hour), so build a fresh environment per session rather than reusing one. The
launcher forwards the exec-server's stderr for diagnostics and returns its exit
code, and closes the exec-server's stdin on teardown so it shuts down.

## References

- [Databricks Sandbox documentation](https://docs.databricks.com/aws/en/compute/serverless/sandbox)
- [Sandbox SDK on databricks-sdk for Python](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/sandbox/sandbox.html)
1 change: 1 addition & 0 deletions examples/codex/databricks/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
databricks-sdk[sandbox]>=0.153.0
162 changes: 162 additions & 0 deletions examples/codex/databricks/runner.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
"""Run an agent against a Databricks sandbox through the native executor.

Prerequisites (see README.md):
- the SDK preview wheel plus this example's requirements.txt
- a Databricks sandbox whose image carries the ``codex`` exec-server
- Databricks auth (a .databrickscfg profile)

Environment variables:
DATABRICKS_CONFIG_PROFILE .databrickscfg profile name (default: DEFAULT)
SANDBOX_NAME sandbox id or ``sandboxes/<id>`` resource name (required)
SANDBOX_CWD remote workspace path (default: /home/sandbox-agent/work)
SANDBOX_CREATE "1" to create the sandbox before running (default: off)
SANDBOX_IMAGE image with the codex exec-server (used when creating)
SANDBOX_INACTIVITY_TIMEOUT_S auto-stop timeout in seconds (default: 900)
OPENAI_MODEL model name (default: gpt-5.5)
OPENAI_BASE_URL model-provider base URL, set together with
OPENAI_API_KEY (default: the Databricks AI Gateway
codex endpoint)
OPENAI_API_KEY model-provider key, set together with OPENAI_BASE_URL
(default: a token minted from the Databricks profile)

With SANDBOX_CREATE=1 the runner provisions the sandbox. In all cases it then
confirms the sandbox is running (starting it if stopped and polling until ready)
and ensures the workspace directory exists before building the environment with
``DatabricksStdioEnvironment.for_sandbox``. When the session opens, the native
runtime spawns the launcher, which reaches the exec-server through
``execute_command`` (needs a running sandbox); no separate exec-server start is
needed here. A sandbox that auto-stops is started again before the next session.

By default the model runs through the sandbox workspace's Databricks AI Gateway
codex endpoint, authenticated with the profile's own token (PAT or OAuth), so no
OpenAI key is required. To use a different provider (for example OpenAI directly),
set OPENAI_BASE_URL and OPENAI_API_KEY together; setting only one is rejected.

Run:
SANDBOX_NAME=my-sandbox DATABRICKS_CONFIG_PROFILE=my-workspace \
PYTHONPATH=. python -m examples.codex.databricks.runner "Inspect the project"
"""

import asyncio
import os
import sys
import time

from databricks.sdk import WorkspaceClient

from agents import Agent, NativeProvider
from agents.sandbox import DatabricksStdioEnvironment


def _resource_name(name: str) -> str:
"""Return the ``sandboxes/<id>`` resource name the sandbox SDK APIs expect."""
return name if name.startswith("sandboxes/") else f"sandboxes/{name}"


def _ensure_sandbox(workspace_client: WorkspaceClient, sandbox_id: str) -> str:
"""Provision the sandbox and return its ``sandboxes/<id>`` resource name.

Mirrors the Databricks sandbox cookbook. Confirm the spec types and their
import path against your installed ``databricks-sdk`` - the sandbox service
module layout may differ by SDK version.
"""
# databricks-sdk re-exports Duration without __all__, so strict mypy flags it.
from databricks.sdk.service.sandbox import ( # type: ignore[attr-defined]
ComputeSpec,
Duration,
EnvironmentSpec,
Sandbox,
SandboxSpec,
)

timeout_s = int(os.getenv("SANDBOX_INACTIVITY_TIMEOUT_S", "900"))
spec = SandboxSpec(compute=ComputeSpec(inactivity_timeout=Duration(seconds=timeout_s)))
image = os.getenv("SANDBOX_IMAGE")
if image is not None:
spec.environment = EnvironmentSpec(image_uri=image)
created = workspace_client.sandbox.create_sandbox(Sandbox(spec=spec), sandbox_id=sandbox_id)
return created.name or _resource_name(sandbox_id)


def _wait_until_running(
workspace_client: WorkspaceClient, sandbox_name: str, timeout_s: int = 300
) -> None:
"""Confirm the sandbox is running before the exec-server is launched.

Starts a stopped sandbox, then polls until it reports running. Mirrors the
managed-API pattern of polling status before launching the exec-server.
"""
from databricks.sdk.service.sandbox import SandboxState

deadline = time.monotonic() + timeout_s
started = False
while True:
sandbox = workspace_client.sandbox.get_sandbox(sandbox_name)
state = sandbox.status.state if sandbox.status is not None else None
if state == SandboxState.SANDBOX_STATE_RUNNING:
return
if state == SandboxState.SANDBOX_STATE_STOPPED and not started:
workspace_client.sandbox.start_sandbox(sandbox_name)
started = True
if time.monotonic() > deadline:
raise TimeoutError(f"sandbox {sandbox_name!r} not running (state={state})")
time.sleep(2)


def _model_credentials(workspace_client: WorkspaceClient) -> tuple[str, str]:
"""Return ``(base_url, api_key)`` for ``NativeProvider``.

With neither OPENAI_BASE_URL nor OPENAI_API_KEY set, defaults to the
workspace's Databricks AI Gateway codex endpoint with a token minted from the
Databricks profile (PAT or OAuth) - no OpenAI key needed. To use another
provider (for example OpenAI directly), set both. Setting only one is rejected
so a key is never sent to the wrong endpoint.
"""
base_url = os.getenv("OPENAI_BASE_URL")
api_key = os.getenv("OPENAI_API_KEY")
if bool(base_url) != bool(api_key):
raise ValueError("set OPENAI_BASE_URL and OPENAI_API_KEY together, or neither")
if base_url and api_key:
return base_url, api_key
host = (workspace_client.config.host or "").rstrip("/")
authorization = workspace_client.config.authenticate().get("Authorization", "")
return f"{host}/ai-gateway/codex/v1", authorization.removeprefix("Bearer ").strip()


async def main(task: str) -> None:
workspace_client = WorkspaceClient(profile=os.getenv("DATABRICKS_CONFIG_PROFILE", "DEFAULT"))
requested = os.environ["SANDBOX_NAME"]
if os.getenv("SANDBOX_CREATE") == "1":
sandbox_name = _ensure_sandbox(workspace_client, requested.removeprefix("sandboxes/"))
else:
sandbox_name = _resource_name(requested)
_wait_until_running(workspace_client, sandbox_name)

cwd = os.getenv("SANDBOX_CWD", "/home/sandbox-agent/work")
workspace_client.sandbox.execute_command_sync(
sandbox_name, "/bin/bash", args=["-lc", f"mkdir -p {cwd}"]
)

environment = DatabricksStdioEnvironment.for_sandbox(
workspace_client=workspace_client,
sandbox_name=sandbox_name,
cwd=cwd,
)
base_url, api_key = _model_credentials(workspace_client)
async with NativeProvider(
base_url=base_url,
api_key=api_key,
supports_websockets=False,
) as provider:
agent = Agent(
instructions="Inspect the assigned workspace and report findings.",
model=os.getenv("OPENAI_MODEL", "gpt-5.5"),
provider=provider,
)
async with agent.session(environment=environment) as session:
result = await (await session.submit(task)).get_result()
print(result.final_output)


if __name__ == "__main__":
asyncio.run(main(" ".join(sys.argv[1:]) or "Inspect the assigned workspace"))
56 changes: 54 additions & 2 deletions packages/codex/rust/environment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -837,8 +837,8 @@ impl EnvironmentOptions {
ensure!(
self.url
.as_deref()
.is_some_and(|url| url.starts_with("docker://")),
"stdio executor requires a current SDK Docker source"
.is_some_and(|url| url.starts_with("docker://") || url.starts_with("stdio://")),
"stdio executor requires a current SDK Docker source or a stdio:// host-owned command"
);
Ok(RemoteTransport::Stdio(
StdioExecServerCommand {
Expand Down Expand Up @@ -2838,6 +2838,58 @@ mod tests {
);
}

#[test]
fn stdio_registration_accepts_host_owned_stdio_scheme() {
let options = serde_json::from_value::<super::EnvironmentOptions>(json!({
"url": "stdio://host-owned/executor",
"cwd": "file:///workspace",
"headers": {},
"stdio_command": {
"program": "codex-missing-exec-server-for-test", "args": [], "env": {}, "cwd": null
}
}))
.expect("decode host options");
let manager = codex_core_api::EnvironmentManager::without_environments(
codex_http_client::HttpClientFactory::new(
codex_http_client::OutboundProxyPolicy::ReqwestDefault,
),
);
options
.register(&manager)
.expect("register stdio:// host-owned command");
assert!(
manager
.get_environment(codex_exec_server::REMOTE_ENVIRONMENT_ID)
.is_some()
);
}

#[test]
fn stdio_registration_rejects_unknown_source_scheme() {
let options = serde_json::from_value::<super::EnvironmentOptions>(json!({
"url": "https://executor.example",
"cwd": "file:///workspace",
"headers": {},
"stdio_command": {
"program": "must-not-spawn", "args": [], "env": {}, "cwd": null
}
}))
.expect("decode host options");
let manager = codex_core_api::EnvironmentManager::without_environments(
codex_http_client::HttpClientFactory::new(
codex_http_client::OutboundProxyPolicy::ReqwestDefault,
),
);
let error = options
.register(&manager)
.err()
.expect("reject unknown stdio source scheme");
assert_eq!(
error.to_string(),
"stdio executor requires a current SDK Docker source or a stdio:// host-owned command"
);
}

#[test]
fn runtime_decoder_accepts_direct_environment_and_rejects_retired_fields() {
let direct = json!({
Expand Down
Loading