Skip to content

fix(sdk): save secrets to base_state.json as soon as they change - #6762

Open
enyst wants to merge 19 commits into
mainfrom
claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave
Open

enyst wants to merge 19 commits into
mainfrom
claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave

Conversation

@enyst

@enyst enyst commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

HUMAN:
Engel Nyst asked me to find the fork PR that removes the two sources of truth for a conversation's settings (meta.json in the agent-server and base_state.json in the SDK), which is #4813 by @BSmick6; to put its commits on branches in this repo, split so that each concern can be reviewed on its own; and to open new PRs that credit the author and say that we do the final review and tweaks. Then to review each PR, address what the review finds, and prove that the changes work by running the SDK and the agent-server the way a program or an agent does, with the repo's verify-sdk and verify-agent-server skills. The principle behind the request: the agent-server is a REST interface to the SDK. It should not be a different thing, and it should not repeat what the SDK already does, unless a particular case gives a special reason.

This is part 1 of 3: the SDK saves secrets to base_state.json the moment they change.


AGENT:

I'm Claude (Fable 5.2), an AI agent working for Engel Nyst. I brought #4813 up to date with main, split it into a stack of three PRs, reviewed each, and verified them against real servers. The fix in this PR is @BSmick6's, from #4813; the ACP commit comes from Engel Nyst's earlier follow-up on that PR. What this final pass added is at the end of the summary.

The stack: 1. this PR → 2. #6763 (a fork keeps the source's policy, analyzer and secrets) → 3. #6764 (the agent-server stops keeping its own copy). #4813's own commits, merged with main at 56d559f6, are on the branch claude/openhands-sdk-dual-source-truth-qp1x6r for reference.

Why

LocalConversation.update_secrets() changed the secret registry in place. ConversationState saves itself only when one of its fields is assigned, so the new secret stayed in memory: base_state.json got it only if something else happened to save the state later. On main that shows up three ways:

  • a secret given at creation (Conversation(secrets=...), POST /api/conversations) is not in base_state.json, and the next GET does not list it;
  • a secret posted at runtime is not listed until some other state change;
  • a restart, an idle eviction or a sandbox pause right after posting a secret loses it.

This is the SDK half of the issue, and the ground the rest of the stack stands on: once meta.json stops carrying secrets (#6764), base_state.json has to have them every time.

Summary

Added in the final review:

  • A state-update event now prints its value through the serializer. Assigning the registry publishes a secret_registry state update, and str(event) printed a LookupSecret's secret headers in clear. JSON output (the socket, the event log) was already masked; this closes the str() path, with a test.
  • A resume that passes no agent (what the agent-server does) under a key that cannot decrypt the state still does not save it on its own. On main that resume writes nothing. With the update_secrets change alone it would: the constructor fills the registry from the agent's context secrets through update_secrets, and under the wrong key (a changed OH_SECRET_KEY without the previous one listed) that save would write the undecrypted state over base_state.json, so the agent's LLM key would stay lost after the right key was restored. A context secret of the agent loaded from base_state.json that is still cipher text, or a StaticSecret without its value, is therefore not filled in. A caller's own agent is taken as given. The limits of this are in the notes. (A resume that passes an agent, or another iteration limit, saves on main and here alike: that assignment is a state change.)
  • An SDK regression test (carried over from fix(agent-server): persist secret, policy and analyzer changes in base_state #5788, which makes the same SDK change), five new checks in the existing verify-sdk S04 secrets recipe, whose program now reads base_state.json right after update_secrets and right after Conversation(secrets=...), and the removal of three verify-agent-server known-bug markers that the proof below retires.
  • Two tests from the last pass, after main gained fix(sdk): persist state changes made while a step waits on the model #5776. A secret posted from another thread while a step waits on its model is on disk before the call returns. And a registry object held from before an update still masks what is learned after it: a running step's stream masker and the Codex credential lifecycle keep the object they started with, which is correct only because update_secrets replaces the registry with model_copy (shared masking cache) and not with a fresh object.

Issue Number

Part of #4810. The last PR of the stack closes it.

How to Test

uv run pytest tests/sdk/conversation/test_conversation_secrets_constructor.py tests/sdk/agent/test_acp_agent.py \
  tests/sdk/agent/test_astep_releases_state_lock.py -q

Each regression test fails without its change and passes with it. The first two rows are main; the others are this branch with the named change reverted:

Test without the change with it
test_update_secrets_saves_base_state_before_returning (on main) fails: assert set() == {'START_TOKEN'} passes
test_secret_posted_during_llm_call_is_persisted_at_once (on main) fails: assert [] == ['POSTED_MID_STEP'] passes
test_live_token_masking_follows_update_secrets_reassignment (new update_secrets, old masker) fails: the token is streamed in clear passes
test_update_secrets_keeps_masking_shared_with_a_registry_held_from_before (a fresh registry object in update_secrets) fails: assert 'leak: runtime-value' == 'leak: <secret-hidden>' passes
test_secrets_state_update_event_never_carries_a_secret_value (old __str__) fails: the header value is in str(event) passes
test_load_under_a_key_that_cannot_decrypt_leaves_the_saved_state_alone (fill not skipped) fails: the agent's key is None once the right key is back passes
test_callers_own_context_secret_is_registered_whatever_it_looks_like (skip applied to every agent) fails: the secret is not registered passes

In this section main is main at 3c66af1d, the commit this branch is merged with. Live, with the repo skills. For the three server bullets every attempt is its own freshly launched agent-server (control-agent-server map run --fresh --repeat 2), two attempts from a main checkout, then two from this branch. The last row is one sdk-map run per side, with no server:

Bullet main this branch
F04.create-secrets-get xfail, xfail xpass, xpass
F08.secrets-visible xfail, xfail xpass, xpass
F08.secrets-unsaved-restart xfail, xfail xpass, xpass
S04.secrets-update, S04.secrets-constructor (sdk-map run, new checks on base_state.json) fail, 5 checks: saved_after_update is [], nothing saved after the constructor either pass (scripted, and example 12 with a real model)

xfail means the bullet failed with exactly the failure its known-bug marker documents; xpass means the documented bug no longer reproduces. The three server bullets were replayed with main's copy of the map, which still carries their markers, against a server launched from main and then from this branch (--checkout). This PR removes the three markers, as the skill asks. With this branch's map the same bullets would then report fail on main and pass here, since a bullet without a marker is a plain fail or pass; I did not run that combination separately. The S04 row is this branch's recipe, with its new checks on base_state.json, run against main's SDK and against this branch; from a main checkout that bullet passes, because main's recipe does not read base_state.json.

# from a main checkout, whose map still has the markers; ../part1 is a checkout of this branch with its own `uv sync --dev`
export PATH="$PWD/.agents/skills/verify-agent-server/scripts:$PATH"
control-agent-server map run --file F08 --only F08.secrets-visible,F08.secrets-unsaved-restart --fresh --repeat 2                        # main: xfail
control-agent-server map run --file F08 --only F08.secrets-visible,F08.secrets-unsaved-restart --fresh --repeat 2 --checkout ../part1   # this branch: xpass
control-agent-server map run --file F04 --only F04.create-secrets-get --fresh --repeat 2                                                 # main: xfail
control-agent-server map run --file F04 --only F04.create-secrets-get --fresh --repeat 2 --checkout ../part1                            # this branch: xpass

# from this branch
export PATH="$PWD/.agents/skills/verify-sdk/scripts:$PATH"
sdk-map run --file S04 --only S04.secrets-update,S04.secrets-constructor

On this branch's final commit (ad341922), the two SDK map families:

Family Result
S04 local conversation (SDK map) 25 of 26 pass; S04.factory labelled blocked in the map
S13 security and confirmation (SDK map) 8 of 9 pass; S13.confirm-approve-reject, S13.llm-analyzer labelled blocked in the map

The whole agent-server families (F04, F08, F10, F12, F31) and the client entry points were replayed on the top of the stack, which contains this commit; that table is in #6764.

GitHub CI on ad341922: all 52 checks that ran are green. One job, agent-server-tests, needed its one re-run: a timing test this PR does not touch failed once while the runner stalled (the comment on this PR has the details). Two checks are skipped by design: the macOS matrix placeholder, and the description check, which waits for the PR to leave draft.

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

  • On the agent-server, a secrets change now reaches the events socket as a state update with key secret_registry, as policy and analyzer changes already do. I captured the frame on a live server from this branch's final code: names and kinds, with values and secret-looking LookupSecret headers masked as **********. GET lists the same names and kinds, with the value as null and without the secret-looking header. No plaintext and no cipher text in either.
  • "Saved before the call returns" holds for a secret posted from another request or thread, in the middle of a step too. Until fix(sdk): persist state changes made while a step waits on the model #5776 a run kept one deferred save open while it waited on its model, so such a secret reached the file only when that step ended. fix(sdk): persist state changes made while a step waits on the model #5776 is in main now and this branch is merged with it; the second row of the table above is the test for it: a secret posted from another thread during the model call is read back from base_state.json before the call returns. One case is still deferred, as for every other state change: a call made on the run loop's own thread inside a step (an event callback, or a tool under a synchronous run()) is written when that step ends.
  • Under a key that cannot decrypt the state, a resume that passes secrets= and a later update_secrets now save, where on main they did not (any other state change saves there too, and loses what could not be decrypted). The agent-server passes its meta.json secrets on every load until fix(agent-server): one owner for confirmation_policy, security_analyzer and secrets (base_state.json) #6764, which removes that path. Cipher text is recognized by its gAAAAA prefix, so on a resume without an agent a real context secret that starts that way is skipped as well; a registry saved with this PR already holds such a secret, so this only shows on a conversation that main created and has not saved since.
  • With this PR alone, a runtime secret with a new name survives a reload. A new value for a name the conversation was created with still goes back to the creation-time value, because the server re-applies its meta.json copy on load. fix(agent-server): one owner for confirmation_policy, security_analyzer and secrets (base_state.json) #6764 removes that copy.
  • fix(agent-server): persist secret, policy and analyzer changes in base_state #5788 contains the same SDK change with a different agent-server half (it keeps the meta.json copies as creation-time seeds). This stack removes those copies instead, so fix(agent-server): persist secret, policy and analyzer changes in base_state #5788's server half is not needed if the stack lands; its SDK test and its feature-map wording are carried here.

🤖 Generated with Claude Code

https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY


🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python-slim amd64, arm64 python-node-runtime Link
python-minimal amd64, arm64 python-node-runtime Link
python amd64, arm64 python-node-runtime Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:6d9b169-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-6d9b169-python \
  ghcr.io/openhands/agent-server:6d9b169-python

All tags pushed for this build

ghcr.io/openhands/agent-server:6d9b169-golang-amd64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-golang-amd64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-golang-amd64
ghcr.io/openhands/agent-server:6d9b169-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:6d9b169-golang-arm64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-golang-arm64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-golang-arm64
ghcr.io/openhands/agent-server:6d9b169-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:6d9b169-java-amd64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-java-amd64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-java-amd64
ghcr.io/openhands/agent-server:6d9b169-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:6d9b169-java-arm64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-java-arm64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-java-arm64
ghcr.io/openhands/agent-server:6d9b169-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:6d9b169-python-amd64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python-amd64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python-amd64
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime-amd64
ghcr.io/openhands/agent-server:6d9b169-python-arm64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python-arm64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python-arm64
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime-arm64
ghcr.io/openhands/agent-server:6d9b169-python-minimal-amd64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python-minimal-amd64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python-minimal-amd64
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime-minimal-amd64
ghcr.io/openhands/agent-server:6d9b169-python-minimal-arm64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python-minimal-arm64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python-minimal-arm64
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime-minimal-arm64
ghcr.io/openhands/agent-server:6d9b169-python-slim-amd64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python-slim-amd64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python-slim-amd64
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime-slim-amd64
ghcr.io/openhands/agent-server:6d9b169-python-slim-arm64
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python-slim-arm64
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python-slim-arm64
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime-slim-arm64
ghcr.io/openhands/agent-server:6d9b169-golang
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-golang
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-golang
ghcr.io/openhands/agent-server:6d9b169-golang_tag_1.21-bookworm
ghcr.io/openhands/agent-server:6d9b169-java
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-java
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-java
ghcr.io/openhands/agent-server:6d9b169-eclipse-temurin_tag_17-jdk
ghcr.io/openhands/agent-server:6d9b169-python-minimal
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python-minimal
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python-minimal
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime-minimal
ghcr.io/openhands/agent-server:6d9b169-python-slim
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python-slim
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python-slim
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime-slim
ghcr.io/openhands/agent-server:6d9b169-python
ghcr.io/openhands/agent-server:6d9b169d15654cbe4b3bc2ab69c3d23a109cde09-python
ghcr.io/openhands/agent-server:claude-openhands-sdk-dual-source-truth-qp1x6r-1-secrets-autosave-python
ghcr.io/openhands/agent-server:6d9b169-python-node-runtime

About Multi-Architecture Support

  • Each variant tag (e.g., 6d9b169-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 6d9b169-python-amd64) are also available if needed

BSmick6 and others added 5 commits November 11, 2026 18:07
update_secrets() mutated state.secret_registry.secret_sources in place.
ConversationState autosaves only on field assignment, so the new secret
stayed in memory: base_state.json did not have it until some other state
change happened to save, and a restart or idle eviction before that lost
it. Readers of the saved state (GET /api/conversations/{id}) could not
see it either.

Update a copy of the registry and assign it, under the state lock, so
the change is saved before update_secrets returns.

Split out of #4813 (commit 30f79f0), where it shipped together with the
agent-server changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
update_secrets() now reassigns state.secret_registry to a fresh object so
the change autosaves to base_state.json. The ACP bridge had captured
state.secret_registry.mask_secrets_in_output once at session start, so a
secret added later was still streamed unmasked through on_token (only the
persisted event was masked). Read the registry on each call instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W9qxyUBKhzmbLrTK2wi7D7
Reads base_state.json right after the constructor and after
update_secrets, then resumes from disk: start-time and runtime secrets
are both there, and a runtime override of a start-time secret is the
value that comes back.

Fails on main, where the registry was mutated in place and never saved.

Carried over from #5788 (commit 864a1e5), which fixes the same bug the
same way.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
The S04 secrets program now reads base_state.json right after
update_secrets and right after Conversation(secrets=...), before any
message or run, and the recipe checks the names are there.

On main both lists are empty; with the fix the recipe passes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
F04.create-secrets-get, F08.secrets-visible and
F08.secrets-unsaved-restart report xfail twice on main and xpass twice
on this branch (`map run --fresh --repeat 2`, each attempt on its own
server), so their markers go and their text describes what happens now.

The wording is carried over from #5788, which proved the same three
bullets for the same SDK change.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

REST API breakage checks (OpenAPI) — ✅ PASSED

Result: ✅ PASSED

Action log

The comment explained what LocalConversation.update_secrets does and
pointed at another method. It now says the one thing the reader of this
code needs: the registry is looked up on every call because the state's
registry object is replaced when secrets change.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
str(ConversationStateUpdateEvent) formatted the live value. For the
`secret_registry` key that is the registry object, whose LookupSecret
headers are a plain dict, so an Authorization header printed in clear.
Nothing in the repo prints these events today, but with secrets saved on
every change this event is now routine on the agent-server, and one
`logger.debug("%s", event)` would have leaked a credential.

Print the value through the serializer, which masks it. The new test
pins the whole contract for a secrets change: one state update, names
and masked values only, in the event, in its printed form and in the
saved event file, with and without a cipher; an unchanged update emits
nothing.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
The secret bullets hold for a conversation that is not in the middle of
a step. A change posted while a run waits on its model is live at once
and saved when the step ends; #5776 is the fix for that window.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands-sdk/openhands/sdk/agent
   acp_agent.py183516291%343–344, 493–494, 521, 559, 638, 643, 776, 803, 809, 894, 935, 1200–1201, 1258–1259, 1343, 1363, 1606–1607, 1657, 1659, 1663, 1667, 1690, 1716, 1719–1722, 1788–1789, 1794, 1810, 1818, 1876, 1883, 1892, 1901, 2078, 2192, 2204, 2217, 2290, 2293–2294, 2324, 2474, 2479, 2529–2530, 2532, 2674, 2812–2815, 2820, 2964, 3063, 3089–3090, 3106–3107, 3393–3394, 3398–3400, 3403–3408, 3412–3414, 3418, 3422–3423, 3426–3427, 3430, 3771, 3786–3787, 3790–3792, 3800, 3804, 3808–3809, 3815–3816, 3828–3829, 3832, 3880, 3885–3887, 3891–3892, 3924, 3961–3962, 4013, 4241–4243, 4246–4247, 4307, 4491, 4499–4500, 4503, 4546–4547, 4550, 4558–4562, 4569, 4571, 4575, 4578, 4587–4591, 4598, 4634–4635, 4653–4656, 4659, 4663–4665, 4667, 4671–4672, 4900, 4902, 4937, 4940–4941, 5022–5023, 5037–5038, 5094–5095
openhands-sdk/openhands/sdk/conversation/impl
   local_conversation.py14309693%204, 257, 461, 529, 565–568, 570, 830–831, 955, 1417, 1419, 1490, 1614, 1646, 1781–1782, 1788, 1797–1800, 1847, 1860, 1884–1885, 1891–1893, 1917, 1924–1927, 1980, 2001–2003, 2052, 2184, 2523–2524, 2528, 2533, 2536, 2544–2545, 2549–2551, 2573–2577, 2580, 2584–2585, 2595–2597, 2603, 2607–2608, 2708, 2905, 2908–2909, 3129, 3166, 3170, 3174, 3195, 3247, 3251, 3285, 3292–3293, 3301, 3305–3306, 3310, 3317, 3636–3637, 3655–3656, 3691, 3907, 3911, 3932, 3983, 3990–3991
openhands-sdk/openhands/sdk/event
   conversation_state.py39295%41, 63
TOTAL57780933784% 

…oad keeps today

The comment records that model_copy keeps the masking cache shared with
the previous registry object, which ACP credential and stream masking can
still hold. The feature-map note now separates a runtime secret with a new
name, which survives a reload, from a new value for a creation-time name,
which the load still replaces with the copy in meta.json.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…aved

A start-time secret and a posted one are in base_state.json and in GET at
once since update_secrets saves the registry; F08's summary line and an
F21 gotcha still said they wait for the next save.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
@datadog-official

This comment has been minimized.

On a resume the constructor fills the registry from the agent's context
secrets, through update_secrets. That call now saves the state. When the
conversation was opened under a key that cannot decrypt it (a changed
OH_SECRET_KEY without the previous one listed), the context secrets are
still cipher text and every other credential reads as unset, so the save
wrote that over base_state.json: the agent's LLM key was gone for good,
also after the right key was restored. On main the same load wrote
nothing.

A context secret that is still cipher text, or a StaticSecret left
without its value, is no longer filled in. It was never a usable value,
and without the fill the load does not save.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
S21.masking covers a secret added after the ACP session started, with its
unit test. F08.secrets-restart no longer speaks of a secret 'that reached
base_state.json': every runtime secret does.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…iled decryption

The check that keeps an undecrypted context secret out of the registry ran
on every construction. A new conversation whose agent carries a secret
that happens to start like cipher text, or a StaticSecret without a value,
would no longer have registered it. Only an agent loaded from
base_state.json can hold values that failed to decrypt, so the check now
applies to that case alone, and a caller's own agent is taken as given.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
main now writes a state change made while a step waits on its model to
base_state.json immediately, so the note that such a change waits for
the end of the step no longer holds.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
With #5776 in main a run no longer keeps a deferred save open across the
model call, so update_secrets() writes base_state.json before it returns
in the middle of a step too. On main the secret is changed in place and
the test fails with an empty saved registry.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
… texts

update_secrets replaces the registry object. The stream masker of a
running step and the Codex credential lifecycle keep the object they
started with, and stay correct only because model_copy shares the
masking cache. A test now holds that: with a fresh SecretRegistry in
update_secrets it fails with the rotated value printed in clear.

The comment on the ACP bridge's masker said it was set per turn to the
registry's bound method; it is bound once and reads the current
registry. The S04 map says when a secrets change is written if the call
is made on the run loop's own thread. mapping.md's example of a
known-bug label named a bullet that is no longer one.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
@enyst

enyst commented Nov 14, 2026

Copy link
Copy Markdown
Member Author

agent-server-tests failed on ad341922 with one test out of 3211: tests/agent_server/test_pub_sub.py::TestPubSubConcurrentDispatch::test_slow_subscriber_does_not_block_others (assert 0.989 < 0.3). It does not look like this PR's:

I re-ran the failed job once.

A possible fix for the test, as a separate change: assert the order of delivery (fast before slow) instead of bounding the wall-clock time. As written, the bound cannot tell concurrent from sequential dispatch either, since 0.2 s + 0 s is 0.2 s both ways.


Generated by Claude Code

@enyst
enyst requested a review from all-hands-bot November 19, 2026 23:45
…dual-source-truth-qp1x6r-1-secrets-autosave

# Conflicts:
#	openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py
@all-hands-bot

Copy link
Copy Markdown
Collaborator

⏳ Review waiting on checks

The current head 6d9b169d1565 still has checks that have not finished, so no review conversation was started:

  • Check OpenAPI Schema
  • agent-server-tests
  • build-binary-and-test (ubuntu-latest)
  • cross-tests
  • pre-commit
  • sdk-tests
  • tools-tests

No action is needed. The scheduled scan retries once every check on the head reports a conclusion.

This is an automated check - no AI was used to generate this comment.

@github-actions

Copy link
Copy Markdown
Contributor

Agent-server conversation start benchmark

Base main@3d638f2ee096d71d9948f4d8eb0ed99eeb9006b0 vs this PR merge 3e67b09405, measured in the same job on the same runner, interleaved: 5 server starts per side and scenario. Medians with 95% bootstrap CIs. The LLM is a mock that answers instantly, so this is agent-server overhead only.

base this PR change
Create conversation (warm) 0.105 s [0.097, 0.180] 0.155 s [0.129, 0.187] +48% ≈ same
↳ first agent event 0.148 s [0.143, 0.242] 0.217 s [0.188, 0.255] +46% ≈ same
First create right after server start 0.509 s [0.275, 1.033] 0.420 s [0.266, 0.519] -17% ≈ same
↳ first agent event 0.899 s [0.500, 1.326] 0.761 s [0.450, 0.918] -15% ≈ same
First create 3 s after server start 0.202 s [0.119, 0.258] 0.155 s [0.096, 0.382] -23% ≈ same
↳ first agent event 0.597 s [0.286, 0.703] 0.431 s [0.285, 0.776] -28% ≈ same
Create in a new git worktree (warm, 0.5 s git fetch) 0.717 s [0.651, 0.778] 0.726 s [0.663, 0.789] +1% ≈ same
↳ first agent event 0.794 s [0.707, 0.850] 0.785 s [0.720, 0.860] -1% ≈ same
Worktree creates that stalled the server > 20 ms 3/20 (15%) 2/20 (10%) -5 pts ≈ same
Create with one stdio MCP server (warm, 3 s startup) 1.168 s [1.127, 1.225] 1.147 s [1.121, 1.204] -2% ≈ same
↳ first agent event 4.011 s [3.854, 4.175] 3.937 s [3.818, 4.114] -2% ≈ same
Creates that stalled the server > 20 ms 4/35 (11%) 3/35 (9%) -3 pts ≈ same
Server startup (/alive) 5.662 s [4.087, 6.345] 4.777 s [4.121, 6.209] -16% ≈ same
How this is measured

Each create is a Canvas-shaped POST /api/conversations (agent profile, worktree: true, first message included, so the request waits for agent startup). A side task polls /alive every 20 ms to catch event-loop stalls. Scenarios, each on its own server starts:

  • default: a fresh directory per create, 8 creates per start.
  • pause: the first create waits 3 s after startup, as a user opening the UI would.
  • worktree: every create branches a new worktree from a local repository whose origin answers fetches after 0.5 s.
  • mcp: the agent profile enables one stdio MCP server that starts after 3 s, like an npx/uvx launch.

A timing is flagged when the 95% CIs do not overlap and the medians differ by at least 50 ms, or when a row with at least 8 samples per side moves by at least 150 ms and 25%. A create stalled the server when its slowest /alive took over 20 ms; stall rows are flagged when the share of stalled creates moves by at least 20 points with a Fisher exact p below 0.01. The first-create rows have one sample per server start, so they are flagged only on a clear change. A "↳ first agent event" row times the create above it until the agent's first event arrives on a socket opened after the create returns, as Agent Canvas does, so it includes the connect and replay. Runner: blacksmith-2vcpu-ubuntu-2404-3ktlelpkblyuqcxm, 178 environment variables. Run it locally with python scripts/agent_server_benchmarks/conversation_start.py --help.

@enyst
enyst marked this pull request as ready for review November 20, 2026 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants