Repository navigation
Release v1.63.0 - #7011
Release v1.63.0#7011
Conversation
Co-authored-by: openhands <openhands@all-hands.dev>
|
AI agent (OpenHands) acting on behalf of Graham Neubig: I triggered the official Prepare Release workflow to propagate the merged filelock remediation (#6811) into a released SDK and downstream automation image. I am reviewing this generated release PR and its integration, behavior, example, security, and compatibility checks before approval or merge. No checks will be bypassed. |
REST API breakage checks (OpenAPI) — ✅ PASSEDResult: ✅ PASSED |
|
Hi! I started running the behavior tests on your PR. You will receive a comment with the results shortly. |
|
Hi! I started running the integration tests on your PR. You will receive a comment with the results shortly. |
🔒 Release Security Scan🔒 Approval drift (time-of-check vs time-of-use)❌ 29 finding(s) Baseline:
Audited 127 PR(s): 98 clean, 29 flagged, 0 un-auditable. 📦 Supply-chain dependency diff✅ no findings Baseline: OSV: no known vulns across 35 new/bumped dep(s). Added dependencies
Bumped dependencies
Internal `openhands-*` bumps
Deterministic scanners: approval-drift + supply-chain dependency diff. Read-only; no PR code executed. |
Agent-server conversation start benchmarkBase
How this is measuredEach create is a Canvas-shaped
A timing is flagged when the 95% CIs do not overlap and the medians differ by at least 50 ms, or when a row with at least 8 samples per side moves by at least 150 ms and 25%. A create stalled the server when its slowest |
|
AI agent (OpenHands) acting on behalf of Graham Neubig: release is on hold. The current-head Release Security Scan failed its approval-drift gate: 29 of 127 merged PRs differ from their approved head; the dependency-diff check passed. Independent review confirms the generated version/schema/lock updates and inclusion of #6811 (filelock 4.2.9) and #6925, but recommends HOLD pending gate reconciliation and completion of integration/behavior/example checks. I have not approved or merged this release and will not waive the gate. I am classifying post-approval deltas versus main-merge/rebase history to provide actionable evidence rather than assuming all findings represent unsafe code. |
|
AI agent (OpenHands) acting on behalf of Graham Neubig: blocker diagnosis is now narrower. Both integration and behavior matrices fail before model calls with |
🔄 Running Examples with
|
| Example | Status | Duration | Cost |
|---|---|---|---|
| 01_standalone_sdk/02_custom_tools.py | ✅ PASS | 32.8s | $0.03 |
| 01_standalone_sdk/03_activate_skill.py | ✅ PASS | 21.2s | $0.03 |
| 01_standalone_sdk/05_use_llm_registry.py | ✅ PASS | 8.2s | $0.00 |
| 01_standalone_sdk/07_mcp_integration.py | ✅ PASS | 30.4s | $0.02 |
| 01_standalone_sdk/09_pause_example.py | ✅ PASS | 16.4s | $0.01 |
| 01_standalone_sdk/10_persistence.py | ✅ PASS | 26.0s | $0.04 |
| 01_standalone_sdk/11_async.py | ✅ PASS | 25.9s | $0.03 |
| 01_standalone_sdk/12_custom_secrets.py | ✅ PASS | 14.0s | $0.01 |
| 01_standalone_sdk/13_get_llm_metrics.py | ✅ PASS | 24.6s | $0.03 |
| 01_standalone_sdk/14_context_condenser.py | ✅ PASS | 4m 32s | $0.00 |
| 01_standalone_sdk/17_image_input.py | ✅ PASS | 22.5s | $0.02 |
| 01_standalone_sdk/18_send_message_while_processing.py | ✅ PASS | 17.4s | $0.01 |
| 01_standalone_sdk/19_llm_routing.py | ✅ PASS | 18.1s | $0.02 |
| 01_standalone_sdk/20_stuck_detector.py | ✅ PASS | 14.8s | $0.02 |
| 01_standalone_sdk/21_generate_extraneous_conversation_costs.py | ✅ PASS | 10.9s | $0.01 |
| 01_standalone_sdk/22_anthropic_thinking.py | ✅ PASS | 16.7s | $0.01 |
| 01_standalone_sdk/23_responses_reasoning.py | ✅ PASS | 43.9s | $0.01 |
| 01_standalone_sdk/24_planning_agent_workflow.py | ✅ PASS | 4m 34s | $0.26 |
| 01_standalone_sdk/25_agent_delegation.py | ✅ PASS | 1m 10s | $0.08 |
| 01_standalone_sdk/26_custom_visualizer.py | ✅ PASS | 20.0s | $0.02 |
| 01_standalone_sdk/28_ask_agent_example.py | ✅ PASS | 33.1s | $0.04 |
| 01_standalone_sdk/29_llm_streaming.py | ✅ PASS | 29.0s | $0.02 |
| 01_standalone_sdk/30_tom_agent.py | ✅ PASS | 34.7s | $0.01 |
| 01_standalone_sdk/31_iterative_refinement.py | ✅ PASS | 1m 34s | $0.07 |
| 01_standalone_sdk/32_configurable_security_policy.py | ✅ PASS | 18.0s | $0.02 |
| 01_standalone_sdk/33_hooks/main.py | ✅ PASS | 31.4s | $0.04 |
| 01_standalone_sdk/34_critic_example.py | ✅ PASS | 25.8s | $0.02 |
| 01_standalone_sdk/36_event_json_to_openai_messages.py | ✅ PASS | 16.4s | $0.00 |
| 01_standalone_sdk/37_llm_profile_store/main.py | ✅ PASS | 6.8s | $0.00 |
| 01_standalone_sdk/38_browser_session_recording.py | ✅ PASS | 50.1s | $0.03 |
| 01_standalone_sdk/39_llm_fallback.py | ✅ PASS | 21.2s | $0.01 |
| 01_standalone_sdk/40_acp_agent_example.py | ✅ PASS | 41.6s | $0.20 |
| 01_standalone_sdk/41_task_tool_set.py | ✅ PASS | 36.1s | $0.03 |
| 01_standalone_sdk/42_file_based_subagents.py | ✅ PASS | 30.4s | $0.03 |
| 01_standalone_sdk/44_model_switching_in_convo.py | ✅ PASS | 11.2s | $0.01 |
| 01_standalone_sdk/45_parallel_tool_execution.py | ✅ PASS | 3m 17s | $0.40 |
| 01_standalone_sdk/46_agent_settings.py | ✅ PASS | 12.0s | $0.01 |
| 01_standalone_sdk/47_defense_in_depth_security.py | ✅ PASS | 3.0s | $0.00 |
| 01_standalone_sdk/48_conversation_fork.py | ✅ PASS | 18.7s | $0.01 |
| 01_standalone_sdk/49_switch_llm_tool.py | ✅ PASS | 9.9s | $0.04 |
| 01_standalone_sdk/50_async_cancellation.py | ✅ PASS | 12.8s | $0.00 |
| 01_standalone_sdk/51_agent_hooks/main.py | ✅ PASS | 38.8s | $0.04 |
| 01_standalone_sdk/52_dynamic_workflow.py | ✅ PASS | 2m 29s | $0.12 |
| 01_standalone_sdk/53_client_defined_tools.py | ✅ PASS | 14.3s | $0.01 |
| 01_standalone_sdk/54_goal_completion_loop.py | ✅ PASS | 29.4s | $0.03 |
| 01_standalone_sdk/55_persistent_memory.py | ✅ PASS | 17.1s | $0.02 |
| 01_standalone_sdk/56_structured_output.py | ✅ PASS | 51.2s | $0.07 |
| 01_standalone_sdk/57_prompt_hooks/main.py | ✅ PASS | 14.8s | $0.00 |
| 01_standalone_sdk/58_ask_oracle_tool/main.py | ✅ PASS | 13.6s | $0.01 |
| 01_standalone_sdk/59_route_task_to_model.py | ✅ PASS | 14.3s | $0.04 |
| 02_remote_agent_server/01_convo_with_local_agent_server.py | ✅ PASS | 43.1s | $0.02 |
| 02_remote_agent_server/02_convo_with_docker_sandboxed_server.py | ✅ PASS | 1m 15s | $0.04 |
| 02_remote_agent_server/03_browser_use_with_docker_sandboxed_server.py | ✅ PASS | 2m 49s | $0.11 |
| 02_remote_agent_server/04_convo_with_api_sandboxed_server.py | ❌ FAIL Exit code 1 |
8.5s | -- |
| 02_remote_agent_server/06_custom_tool/main.py | ✅ PASS | 5m 14s | $0.06 |
| 02_remote_agent_server/07_convo_with_cloud_workspace.py | ✅ PASS | 40.9s | $0.02 |
| 02_remote_agent_server/08_convo_with_apptainer_sandboxed_server.py | ✅ PASS | 4m 26s | $0.03 |
| 02_remote_agent_server/09_acp_agent_with_remote_runtime.py | ❌ FAIL Exit code 1 |
6.0s | -- |
| 02_remote_agent_server/10_cloud_workspace_share_credentials.py | ✅ PASS | 38.8s | $0.00 |
| 02_remote_agent_server/11_conversation_fork.py | ✅ PASS | 41.5s | $0.00 |
| 02_remote_agent_server/12_settings_and_secrets_api.py | ✅ PASS | 2m 32s | $0.02 |
| 02_remote_agent_server/13_workspace_get_llm.py | ✅ PASS | 44.6s | $0.01 |
| 02_remote_agent_server/14_client_defined_tools.py | ✅ PASS | 36.9s | $0.02 |
| 02_remote_agent_server/15_openai_compatible_gateway.py | ✅ PASS | 21.1s | $0.04 |
| 02_remote_agent_server/16_deferred_init.py | ✅ PASS | 20.6s | $0.01 |
| 04_llm_specific_tools/01_gpt5_apply_patch_preset.py | ✅ PASS | 19.9s | $0.02 |
| 04_llm_specific_tools/02_gemini_file_tools.py | ✅ PASS | 36.4s | $0.11 |
| 05_skills_and_plugins/01_loading_agentskills/main.py | ✅ PASS | 13.9s | $0.02 |
| 05_skills_and_plugins/02_loading_plugins/main.py | ✅ PASS | 19.4s | $0.03 |
| 05_skills_and_plugins/04_mixed_marketplace_skills/main.py | ✅ PASS | 4.6s | $0.00 |
❌ Some tests failed
Total: 70 | Passed: 68 | Failed: 2 | Total Cost: $2.55
Failed examples:
- examples/02_remote_agent_server/04_convo_with_api_sandboxed_server.py: Exit code 1
- examples/02_remote_agent_server/09_acp_agent_with_remote_runtime.py: Exit code 1
|
AI agent (OpenHands) acting on behalf of Graham Neubig: pushed the integration-runner fix in 010400e. Workflow entrypoints now use |
|
AI agent (OpenHands) acting on behalf of Graham Neubig. Confirmed cause of integration/behavior runner startup failureThe LiteLLM upgrade in #6811 ( In LiteLLM 1.105.1, I reproduced this using official PyPI wheels, holding the SDK code, interpreter, and other installed dependencies constant and selecting only the LiteLLM version:
A traced import also confirmed the old proxy CLI appends the working directory; the new version does not. This isolates the trigger rather than attributing it to model behavior or a transient runner failure. The fix already pushed in 010400e uses module invocation, eliminating reliance on this third-party side effect while retaining the security upgrade. All four workflow-entrypoint regression tests pass locally. These are startup-only tests with no LLM calls; full integration/behavior validation on the fixed head is still needed. This diagnosis does not resolve the separate approval-drift security gate. |
Release v1.63.0
This PR prepares the release for version 1.63.0.
Started by: @neubig
Release Checklist
.pr/artifacts removedintegration-test)behavior-test)test-examples)security-scan)release-note-requiredPRs are accurately called out in the final release notesUpdating this branch
If
mainadvances, refresh this branch as described in.github/workflows/README-RELEASE.mdinstead of using Update branch, so new.pr/artifacts are removed in the same update.What happens on merge
When this PR is merged, the
create-release.ymlworkflow will automatically:v1.63.0and auto-generated notes, plus an explicit preamble for mergedrelease-note-requiredPRsv1.63.0version-bump-prs.ymlafter successful PyPI publication🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdkpython-node-runtimepython-node-runtimepython-node-runtimegolang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:010400e-pythonRun
All tags pushed for this build
About Multi-Architecture Support
010400e-python) is a multi-arch manifest supporting both amd64 and arm64010400e-python-amd64) are also available if needed