add template for claude agent sdk with long running tasks

agent-claude-sdk-advanced/.agents/skills/add-tools/SKILL.md

@@ -0,0 +1,124 @@
+---
+name: add-tools
+description: "Add tools to your agent and grant required permissions in databricks.yml. Use when: (1) Adding MCP servers, Genie spaces, vector search, or UC functions to agent, (2) Permission errors at runtime, (3) User says 'add tool', 'connect to', 'grant permission', (4) Configuring databricks.yml resources."
+---
+
+# Add Tools & Grant Permissions
+
+> **Profile reminder:** All `databricks` CLI commands must include the profile from `.env`: `databricks <command> --profile <profile>`
+
+> Don't have the resource yet? See **create-tools** skill first.
+
+**After adding any MCP server to your agent, you MUST grant the app access in `databricks.yml`.**
+
+Without this, you'll get permission errors when the agent tries to use the resource.
+
+## Workflow
+
+**Step 1:** Add MCP server in `agent_server/agent.py`:
+```python
+from databricks_langchain import DatabricksMCPServer, DatabricksMultiServerMCPClient
+
+genie_server = DatabricksMCPServer(
+    url=f"{host}/api/2.0/mcp/genie/01234567-89ab-cdef",
+    name="my genie space",
+)
+
+mcp_client = DatabricksMultiServerMCPClient([genie_server])
+tools = await mcp_client.get_tools()
+```
+
+**Step 2:** Grant access in `databricks.yml`:
+```yaml
+resources:
+  apps:
+    agent_langgraph:
+      resources:
+        - name: 'my_genie_space'
+          genie_space:
+            name: 'My Genie Space'
+            space_id: '01234567-89ab-cdef'
+            permission: 'CAN_RUN'
+```
+
+**Step 3:** Deploy and run:
+```bash
+databricks bundle deploy
+databricks bundle run agent_langgraph  # Required to start app with new code!
+```
+
+See **deploy** skill for more details.
+
+## Resource Type Examples
+
+See the `examples/` directory for complete YAML snippets:
+
+| File | Resource Type | When to Use |
+|------|--------------|-------------|
+| `uc-function.yaml` | Unity Catalog function | UC functions via MCP |
+| `uc-connection.yaml` | UC connection | External MCP servers |
+| `vector-search.yaml` | Vector search index | RAG applications |
+| `sql-warehouse.yaml` | SQL warehouse | SQL execution |
+| `serving-endpoint.yaml` | Model serving endpoint | Model inference |
+| `genie-space.yaml` | Genie space | Natural language data |
+| `lakebase.yaml` | Lakebase database | Agent memory storage (provisioned) |
+| `lakebase-autoscaling.yaml` | Lakebase autoscaling postgres | Agent memory storage (autoscaling) |
+| `experiment.yaml` | MLflow experiment | Tracing (already configured) |
+| `app.yaml` | Databricks App (app-to-app) | Custom MCP servers hosted as Apps |
+| `custom-mcp-server.md` | Custom MCP apps | Apps starting with `mcp-*` |
+
+## Custom MCP Servers (Databricks Apps)
+
+Declare the target app as an `app` resource in `databricks.yml` — the bundle grants `CAN_USE` on deploy. Requires Databricks CLI **v0.298.0+**.
+
+```yaml
+resources:
+  apps:
+    agent_langgraph:
+      resources:
+        - name: 'mcp_server'
+          app:
+            name: 'mcp-my-server'
+            permission: CAN_USE
+```
+
+See `examples/custom-mcp-server.md` for the full flow (agent code + YAML + deploy).
+
+## value_from Pattern
+
+**IMPORTANT**: Make sure all `value_from` references in `databricks.yml` `config.env` reference an existing key in the `databricks.yml` `resources` list.
+Some resources need environment variables in your app. Use `value_from` in `databricks.yml` `config.env` to reference resources defined in `databricks.yml`:
+
+```yaml
+# In databricks.yml, under apps.<app>.config.env:
+env:
+  - name: MLFLOW_EXPERIMENT_ID
+    value_from: "experiment"        # References resources.apps.<app>.resources[name='experiment']
+  - name: LAKEBASE_INSTANCE_NAME
+    value_from: "database"   # References resources.apps.<app>.resources[name='database']
+```
+
+**Critical:** Every `value_from` value must match a `name` field in `databricks.yml` resources.
+
+## MCP Error Handling
+
+MCP tool calls can fail (network issues, permission errors, timeouts). Use `handle_tool_error` on MCP servers to catch errors and return them to the LLM instead of crashing the agent:
+
+```python
+DatabricksMCPServer(
+    name="genie",
+    url=f"{host}/api/2.0/mcp/genie/{space_id}",
+    handle_tool_error=True,   # Return error messages to LLM instead of raising
+    timeout=60.0,             # Increase timeout for slow tools like Genie
+)
+```
+
+For local function tools defined with `@tool`, see `create-tools` skill > `examples/local-python-tools.md` for the `ToolException` + `handle_tool_error` pattern.
+
+## Important Notes
+
+- **MLflow experiment**: Already configured in template, no action needed
+- **Multiple resources**: Add multiple entries under `resources:` list
+- **Permission types vary**: Each resource type has specific permission values
+- **Deploy + Run after changes**: Run both `databricks bundle deploy` AND `databricks bundle run agent_langgraph_advanced`
+- **value_from matching**: Ensure `config.env` `value_from` values match `databricks.yml` resource `name` values

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/app.yaml

@@ -0,0 +1,9 @@
+# Databricks App (for custom MCP servers hosted as Apps)
+# Use for: Granting CAN_USE on another Databricks App (e.g., an mcp-* server app)
+# Requires: CLI v0.298.0+
+
+# In databricks.yml - add to resources.apps.<app>.resources:
+- name: 'mcp_server'
+  app:
+    name: '<target-app-name>'
+    permission: CAN_USE

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/custom-mcp-server.md

@@ -0,0 +1,51 @@
+# Custom MCP Server (Databricks App)
+
+Custom MCP servers are Databricks Apps with names starting with `mcp-*`.
+
+Declare the target app as an `app` resource in `databricks.yml` and the bundle will grant `CAN_USE` on deploy. Requires Databricks CLI **v0.298.0+**.
+
+## Steps
+
+### 1. Add MCP server in `agent_server/agent.py`
+
+```python
+from databricks_langchain import DatabricksMCPServer, DatabricksMultiServerMCPClient
+
+custom_mcp = DatabricksMCPServer(
+    url="https://mcp-my-server.cloud.databricks.com/mcp",
+    name="my custom mcp server",
+)
+
+mcp_client = DatabricksMultiServerMCPClient([custom_mcp])
+tools = await mcp_client.get_tools()
+```
+
+### 2. Grant access in `databricks.yml`
+
+Add the target app as a resource:
+
+```yaml
+resources:
+  apps:
+    agent_langgraph:
+      resources:
+        - name: 'mcp_server'
+          app:
+            name: 'mcp-my-server'
+            permission: CAN_USE
+```
+
+### 3. Deploy
+
+```bash
+databricks bundle deploy
+databricks bundle run agent_langgraph
+```
+
+The bundle grants `CAN_USE` on the target app automatically — no manual permission steps needed.
+
+## Notes
+
+- Requires CLI v0.298.0+ (earlier versions will warn `unknown field: name` on `app.name`)
+- The only supported permission is `CAN_USE`
+- Subsequent `databricks bundle deploy` commands preserve the `app` resource

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/experiment.yaml

@@ -0,0 +1,8 @@
+# MLflow Experiment
+# Use for: Tracing and model logging
+# Note: Already configured in template's databricks.yml
+
+- name: 'my_experiment'
+  experiment:
+    experiment_id: '12349876'
+    permission: 'CAN_MANAGE'

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/genie-space.yaml

@@ -0,0 +1,9 @@
+# Genie Space
+# Use for: Natural language interface to data
+# MCP URL: {host}/api/2.0/mcp/genie/{space_id}
+
+- name: 'my_genie_space'
+  genie_space:
+    name: 'My Genie Space'
+    space_id: '01234567-89ab-cdef'
+    permission: 'CAN_RUN'

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/lakebase-autoscaling.yaml

@@ -0,0 +1,21 @@
+# Lakebase Autoscaling Postgres (for agent memory)
+# Use for: Short-term or long-term memory storage with autoscaling Lakebase
+
+# In databricks.yml - add to resources.apps.<app>.resources:
+- name: 'postgres'
+  postgres:
+    branch: "projects/<project-name>/branches/<branch-name>"
+    database: "projects/<project-name>/branches/<branch-name>/databases/<database-id>"
+    permission: CAN_CONNECT_AND_CREATE
+
+# In databricks.yml config block - add to env:
+# - name: LAKEBASE_AUTOSCALING_PROJECT
+#   value: "<project-name>"
+# - name: LAKEBASE_AUTOSCALING_BRANCH
+#   value: "<branch-name>"
+
+# How to find the values:
+# databricks api get /api/2.0/postgres/projects
+# databricks api get /api/2.0/postgres/projects/<project-name>/branches
+# databricks api get /api/2.0/postgres/projects/<project-name>/branches/<branch-name>/databases
+# The database-id is the internal ID (e.g., db-xxxx-xxxxxxxxxx), NOT "databricks_postgres"

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/lakebase.yaml

@@ -0,0 +1,18 @@
+# Lakebase Database (for agent memory)
+# Use for: Long-term memory storage via AsyncDatabricksStore
+# Requires: value_from reference in databricks.yml config block
+
+# In databricks.yml - add to resources.apps.<app>.resources:
+- name: 'database'
+  database:
+    instance_name: '<your-lakebase-instance-name>'
+    database_name: 'databricks_postgres'
+    permission: 'CAN_CONNECT_AND_CREATE'
+
+# In databricks.yml config block - add to env:
+# - name: LAKEBASE_INSTANCE_NAME
+#   value_from: "database"
+# - name: EMBEDDING_ENDPOINT
+#   value: "databricks-gte-large-en"
+# - name: EMBEDDING_DIMS
+#   value: "1024"

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/serving-endpoint.yaml

@@ -0,0 +1,7 @@
+# Model Serving Endpoint
+# Use for: Model inference endpoints
+
+- name: 'my_endpoint'
+  serving_endpoint:
+    name: 'my_endpoint'
+    permission: 'CAN_QUERY'

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/sql-warehouse.yaml

@@ -0,0 +1,7 @@
+# SQL Warehouse
+# Use for: SQL query execution
+
+- name: 'my_warehouse'
+  sql_warehouse:
+    sql_warehouse_id: 'abc123def456'
+    permission: 'CAN_USE'

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/uc-connection.yaml

@@ -0,0 +1,9 @@
+# Unity Catalog Connection
+# Use for: External MCP servers via UC connections
+# MCP URL: {host}/api/2.0/mcp/external/{connection_name}
+
+- name: 'my_connection'
+  uc_securable:
+    securable_full_name: 'my-connection-name'
+    securable_type: 'CONNECTION'
+    permission: 'USE_CONNECTION'

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/uc-function.yaml

@@ -0,0 +1,9 @@
+# Unity Catalog Function
+# Use for: UC functions accessed via MCP server
+# MCP URL: {host}/api/2.0/mcp/functions/{catalog}/{schema}/{function_name}
+
+- name: 'my_uc_function'
+  uc_securable:
+    securable_full_name: 'catalog.schema.function_name'
+    securable_type: 'FUNCTION'
+    permission: 'EXECUTE'

agent-claude-sdk-advanced/.agents/skills/add-tools/examples/vector-search.yaml

@@ -0,0 +1,9 @@
+# Vector Search Index
+# Use for: RAG applications with unstructured data
+# MCP URL: {host}/api/2.0/mcp/vector-search/{catalog}/{schema}/{index_name}
+
+- name: 'my_vector_index'
+  uc_securable:
+    securable_full_name: 'catalog.schema.index_name'
+    securable_type: 'TABLE'
+    permission: 'SELECT'