Skip to content

docs(verify): map maintenance 2026-11-14 - #6847

Merged
enyst merged 1 commit into
mainfrom
cloud/blissful-tesla-l6vzev
Nov 14, 2026
Merged

enyst merged 1 commit into
mainfrom
cloud/blissful-tesla-l6vzev

Conversation

@enyst

@enyst enyst commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

HUMAN:
Engel Nyst (@enyst) set up a daily automation that keeps the verify-agent-server and verify-sdk feature maps current. This PR is its run for main@3c66af1, after PR #6753.


AGENT:
I'm an AI agent helping Engel Nyst (@enyst) with project work.

Why

25 commits merged between the maps' baseline (fb75ae9) and main@3c66af1. map owners listed 33 of 35 agent-server families and sdk-map owners 18 of 21 SDK families (shared LLM and conversation code, plus a pydantic bump), so both maps got a full pass instead of a delta: every family replayed live on a fresh run, then every edited family replayed again by an independent agent.

The pass found five product defects that had no issue. Each was reproduced a second time by an independent agent, filed, and marked in the maps; four have their own PR.

Summary

Agent-server map: full pass, fb75ae9..3c66af1, 990 → 1002 sub-features.

SDK map: full pass, fb75ae9..3c66af1, 464 → 476 sub-features.

Merged PRs covered (intent / result):

PR Intent Result
#6690 LLM usage sessions documented pass; changed what an ask records (map corrected); exposed #6844
#6780 domain errors to HTTP once documented pass; statuses and bodies unchanged on driven routes
#6702 one ServerContext documented pass; failed init rolls back env (new bullet)
#6573 LLM span content documented pass; #6838 asks what the filter covers
#6757 task sub-agent secrets documented pass on SDK and server
#6040 capability resolution documented pass, except the native DeepSeek guard (#6840)
#6670 hung browser documented pass
#4870 mask thinking blocks contradictory: summary says masked blocks lose the signature, code strips all masking passes; #6837
#5776 persist state during model wait documented pass
#6794 MCP auto negotiation documented, with a consumer-visible cost pass; on stdio and streamable HTTP with server/discover, per-session server state is not kept between calls and a tools/list_changed sent outside a request is not delivered (the PR moved those tests to SSE)
#5815 plugin hooks loaded twice documented pass
#6549 gateway direct events documented pass; #6842 found next to it
#5353 Debian snapshot, #6753 previous map PR, 11 dependency bumps (pydantic 2.14.3, sqlalchemy, cachetools, boto3 ×2, posthog, joserfc, openrouter ×2, two CI actions) no consumer change expected full replay shows none

Route diff fb75ae9 → 3c66af1: no route added, removed or deprecated.

Twins. Main has no Server: / Same promise: links yet (#6728 is open), so no link was edited. Pairs driven on both surfaces by ID: task-delegate secrets (S11.task-secrets / F08.secrets-task-delegate), masking of an agent message (S13.mask-thinking / F08.secrets-agent-message), ask usage (S04.ask-agent-usage-saved / F10.ask-agent-usage-saved), completion logs (S02.log-completions / F33.completion-logs). None disagreed: the ask-usage pair fails the same way on both, which puts #6844 in the SDK, not at the seam.

Issues filed by this pass (each reproduced twice, by different agents, on fresh runs):

Issue What Regression? PR
#6837 text and reasoning-only replies lose every thinking signature, so the thinking is not sent back yes, from #4870 #6839
#6840 native DeepSeek image-loss rejection is retried and ends as LLMServiceUnavailableError (about 120 s with default retries) new behavior from #6040 #6841
#6842 gateway turn on a paused or errored conversation answers 409/500 while the turn runs no, also on fb75ae9 #6843
#6844 ask_agent usage is not saved: REST and a restarted server miss what the socket reported no; #6690 made the socket right, which exposed it #6845
#6838 the tracing filter covers the llm.<model> span only no none: needs a maintainer decision

The four PRs are green except "Validate PR description", which waits for triage to label the new issues.

Cross-repo. No merged PR in the range needs a Canvas change. Canvas pins Agent Server 1.62.0 (config/defaults.json), released before these commits, so released Canvas does not have them yet. #6844 shows in Canvas as the usage meter dropping back after a reload (screenshots below); it is a server defect, so no OpenHands/OpenHands issue was filed.

Next baselines: both maps move to 3c66af1d14d0a7fbe74b8dc4218412ec7a877b79.

Issue Number

#5751

How to Test

Static checks, from the repository root with the two scripts/ directories on PATH:

control-agent-server map check && control-agent-server map coverage && sdk-map check
uv run pytest tests/cross/test_verify_agent_server_skill.py tests/cross/test_verify_sdk_skill.py tests/cross/test_control_sdk.py   # 98 passed

Replay an edited family (needs DEEPSEEK_API_KEY for the model-backed bullets):

control-agent-server map run --file F10 --fresh --record     # also F03 F07 F08 F16 F19 F30 F31 F33 F34
sdk-map run --file S02                                       # also S01 S03 S04 S09 S10 S11 S13 S15 S20 S21

Saved transcripts, one folder per sub-feature (all):

Limits: Docker, a real code-server binary, GitHub and ChatGPT credentials were not available (the blocked rows). The Canvas check ran on one desktop viewport with DeepSeek, so thinking_blocks[].signature was not exercised in the UI.

Video/Screenshots

Canvas (OpenHands/OpenHands@830c045) on Agent Server 3c66af1.

#6690, Canvas F05.slash-btw: after a /btw side question the usage meter rises without a reload (28,326 input tokens, $0.0026) and the main thread gets no message.

[Usage meter after a side question](https://raw.githubusercontent.com/enyst/openhands-evidence/6c6225b5edd5a364893a18f923ca13d0a059ddf4/software-agent-sdk/map-maintenance/2026-11-14/ui/F05.slash-btw-02-after-btw-live.png)

#6844, same conversation after a page reload: the meter is back at 14,736 tokens and $0.0015, because REST does not have the ask's usage.

[Usage meter after reload](https://raw.githubusercontent.com/enyst/openhands-evidence/6c6225b5edd5a364893a18f923ca13d0a059ddf4/software-agent-sdk/map-maintenance/2026-11-14/ui/F05.slash-btw-03-after-reload.png)

#6549, Canvas F05.queued-message: a message sent while the agent runs a tool call is still accepted and shown.

[Message sent while running](https://raw.githubusercontent.com/enyst/openhands-evidence/6c6225b5edd5a364893a18f923ca13d0a059ddf4/software-agent-sdk/map-maintenance/2026-11-14/ui/F05.queued-message-02-finished.png)

#6757, Canvas F14.agent-access: the parent's terminal shows the saved secret masked, and the task sub-agent reports the variable present (len=13).

[Secret reaches the sub-agent](https://raw.githubusercontent.com/enyst/openhands-evidence/6c6225b5edd5a364893a18f923ca13d0a059ddf4/software-agent-sdk/map-maintenance/2026-11-14/ui/F14.agent-access-subagent-secret-child.png)

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

  • Only files under .agents/skills/verify-agent-server/ and .agents/skills/verify-sdk/ change (map families, the two indexes, SDK recipe programs). No product code, no scripts/, no .pr/.
  • Harness gaps seen, not changed here: the CLI's vscode capability probe still looks for openvscode-server, so Needs: vscode bullets stay blocked where code-server is installed.
  • feat: enable MCP protocol auto negotiation #6794's two consumer-visible losses (table above) are documented in the PR itself; flagged for a maintainer who relies on stateful or progressive-disclosure MCP servers.

Generated by Claude Code


🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python-slim amd64, arm64 python-node-runtime Link
python-minimal amd64, arm64 python-node-runtime Link
python amd64, arm64 python-node-runtime Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:c0869e4-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-c0869e4-python \
  ghcr.io/openhands/agent-server:c0869e4-python

All tags pushed for this build

ghcr.io/openhands/agent-server:c0869e4-golang-amd64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-golang-amd64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-golang-amd64
ghcr.io/openhands/agent-server:c0869e4-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:c0869e4-golang-arm64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-golang-arm64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-golang-arm64
ghcr.io/openhands/agent-server:c0869e4-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:c0869e4-java-amd64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-java-amd64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-java-amd64
ghcr.io/openhands/agent-server:c0869e4-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:c0869e4-java-arm64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-java-arm64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-java-arm64
ghcr.io/openhands/agent-server:c0869e4-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:c0869e4-python-amd64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python-amd64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python-amd64
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime-amd64
ghcr.io/openhands/agent-server:c0869e4-python-arm64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python-arm64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python-arm64
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime-arm64
ghcr.io/openhands/agent-server:c0869e4-python-minimal-amd64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python-minimal-amd64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python-minimal-amd64
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime-minimal-amd64
ghcr.io/openhands/agent-server:c0869e4-python-minimal-arm64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python-minimal-arm64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python-minimal-arm64
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime-minimal-arm64
ghcr.io/openhands/agent-server:c0869e4-python-slim-amd64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python-slim-amd64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python-slim-amd64
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime-slim-amd64
ghcr.io/openhands/agent-server:c0869e4-python-slim-arm64
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python-slim-arm64
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python-slim-arm64
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime-slim-arm64
ghcr.io/openhands/agent-server:c0869e4-golang
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-golang
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-golang
ghcr.io/openhands/agent-server:c0869e4-golang_tag_1.21-bookworm
ghcr.io/openhands/agent-server:c0869e4-java
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-java
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-java
ghcr.io/openhands/agent-server:c0869e4-eclipse-temurin_tag_17-jdk
ghcr.io/openhands/agent-server:c0869e4-python-minimal
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python-minimal
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python-minimal
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime-minimal
ghcr.io/openhands/agent-server:c0869e4-python-slim
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python-slim
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python-slim
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime-slim
ghcr.io/openhands/agent-server:c0869e4-python
ghcr.io/openhands/agent-server:c0869e4e158533e9a27e46b7868ffb04253a0d3d-python
ghcr.io/openhands/agent-server:cloud-blissful-tesla-l6vzev-python
ghcr.io/openhands/agent-server:c0869e4-python-node-runtime

About Multi-Architecture Support

  • Each variant tag (e.g., c0869e4-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., c0869e4-python-amd64) are also available if needed

Full pass of both feature maps over fb75ae9..3c66af1.

Agent-server map: the ask_agent bullets (F10, F33, F34) expect the stats
event an ask records; new bullets for the gateway's 409s (F30), the
all-or-nothing failed init (F03), and secrets reaching a task delegate and
an agent message (F08); known-bug bullets for #6803, #6808, #6814, #6826,
#6828, #6842 and #6844; F16 names code-server; F33 uses
log_completions_to_file.

SDK map: new bullets for the image request guard, the default output
budget, thinking masking, LLM span content, usage sessions, arun pause
persistence, MCP protocol negotiation and plugin hook reloads; Known
failure bullets for #6809, #6837, #6840 and #6844; S09's restart bullet
states what each connection kind does.

Both baselines move to 3c66af1.

Co-authored-by: openhands <openhands@all-hands.dev>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwKPsuVAvmEWqYbQREH
@github-actions

Copy link
Copy Markdown
Contributor

REST API breakage checks (OpenAPI) — ✅ PASSED

Result: ✅ PASSED

Action log

@enyst
enyst requested a review from all-hands-bot November 14, 2026 11:09

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was posted by an AI agent (OpenHands).

Scope gate: all 40 changed files sit under .agents/skills/verify-agent-server/ and .agents/skills/verify-sdk/ (feature maps, the two indexes, SDK recipe programs). Per the repo's contribution guidance, the verify skills are first-class, actively maintained content in this repository with dedicated cross tests, so this map-maintenance pass belongs here. No product code, no scripts/, no CI workflows are touched, and the route diff (fb75ae9 → 3c66af1) is empty, which the map's route-coverage check confirms.

Verification on head c0869e4:

  • Static checks pass exactly as claimed: map check reports 35 agent-server families / 1002 sub-features and 21 SDK families / 476 sub-features with no problems; route coverage reports 215 routes, 205 owned, 205 driven, no unowned / not-driven / stale exclusions. uv run pytest tests/cross/test_verify_agent_server_skill.py tests/cross/test_verify_sdk_skill.py tests/cross/test_control_sdk.py → 98 passed, matching the "How to Test" claim.
  • The PR's most consequential map corrections were independently checked against the source at this head. The F10/F33/F34 ask corrections match the code: ask_agent installs an ask-agent-llm registry LLM (ASK_AGENT_LLM_USAGE_ID), its call routes through session_for(call_context), whose telemetry fires notify_stats_update(), and the server's stats_callback emits one stats-keyed ConversationStateUpdateEvent that is not a MessageEvent — so "one stats event, no message" is accurate, and the #6844 xfail (broadcast-only, nothing persisted for REST/restart) is consistent with that path.
  • S02.log-completions' deprecation claim matches llm.py exactly (log_completions deprecated since v1.63.0, removal scheduled for v1.68.0; log_completions_to_file is the real flag).
  • All ~30 cited unit-test paths/symbols exist in this tree (e.g. test_capability_wire.py::test_native_deepseek_adapter_must_deliver_or_reject_images, test_llm.py::test_advertised_output_limit_is_not_a_request_default, test_astep_releases_state_lock.py::test_pause_during_llm_call_is_persisted_at_once, test_response_dispatch.py::test_thinking_blocks_mask_registered_secret, test_acp_agent.py::test_warm_npx_cache_passes_every_pinned_package), including the new S02 program's dependency on DEFAULT_MAX_OUTPUT_TOKENS_CAP = 16384.
  • The new/updated SDK programs run live against this head: context-guard reproduces the documented 16384 cap and the advertised-limit behavior, image-guard reproduces LLMBadRequestError before any HTTP request with requests: 0 plus the override/vision paths, and image-adapter-loss reproduces the #6840 failure signature exactly as documented (retries: 2, LLMServiceUnavailableError, zero requests sent).
  • Every issue and fix-PR the maps cite exists at the time of review (#6837/#6839, #6840/#6841, #6842/#6843, #6844/#6845, #6838, #6814, #6803, #6828, #6808, #6826, #6809, #6785/#6786, #6657, #6745, #6647), and the linked issue #5751 is closed. The blocked rows (Docker, code-server, GitHub/ChatGPT credentials) are accurately scoped as environment limits rather than map failures.

No material findings: the diff is documentation and verification content only, its corrections match the code at this head, its programs execute cleanly, and CI is green on the head (53/53 checks).

✅ APPROVED

@enyst
enyst merged commit 7435f4b into main Nov 14, 2026
53 checks passed
@enyst
enyst deleted the cloud/blissful-tesla-l6vzev branch November 14, 2026 22:30
@enyst enyst mentioned this pull request Nov 15, 2026
1 of 5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants