Repository navigation
fix(sdk): a fork keeps the source's confirmation policy, security analyzer and secrets - #6763
Conversation
LocalConversation.fork() built the fork without the source's cipher, so the fork's base_state.json stored the LLM key redacted. The agent-server resumes every fork from that file, so the fork's first turn failed with LLMAuthenticationError. Forward the cipher, and copy the source's conversation secrets into the fork's registry so they persist too. The verify-agent-server map bullets F10.fork-keeps-credentials and F10.fork-run-memory now pass; their known-bug markers are removed. Fixes #5465 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014MKndVj7XWvRQmvW9w2bDg
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
must carry, and it does the registry copy better than #4813 did: a fresh SecretRegistry with deep-copied sources, so the fork does not share the source's masking cache or its lock. Merging it here instead of re-implementing it keeps one version of that code, with its authors and its tests (kyletser's restart test included), whichever PR lands first. Conflict: the feature-map index count. main's 955 stands (`map check` agrees); #5765 removes two known-bug markers and adds no sub-feature. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…ersation.fork() These fields are part of ConversationState but fork() did not copy them, so the fork's base_state.json was written with the NeverConfirm / None defaults and the source conversation's policy was silently lost: a fork of a conversation under AlwaysConfirm ran its actions unasked. Split out of #4813 (commit 61b865e). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
… kept The fork shared the source's analyzer object, so a stateful analyzer's history (e.g. ToolShieldLLMSecurityAnalyzer) mixed two conversations. Give the fork its own instance through a JSON round-trip with expose_secrets, so the fork's cipher re-encrypts nested credentials (the guardrail LLM api_key) instead of writing them redacted. Tests: the fork owns its analyzer, a ciphered analyzer key survives a resume of the fork, and secrets passed at start reach the fork (the registry copy itself comes from #5765). Split out of #4813 (commit f6ade80). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
The policy copy had no test. This one checks the fork's live state and its base_state.json, which is what the agent-server resumes a fork from. It fails on main, where the fork starts with NeverConfirm. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
New sub-feature S04.fork-keeps-state with a program that forks a ciphered conversation carrying ConfirmRisky(MEDIUM), a ToolShield analyzer with its own key and a runtime secret, then changes the fork and loads it again from disk. The recipe checks the fork's memory, its base_state.json and the reload. On main fifteen of its checks fail: the fork starts with NeverConfirm, no analyzer and no secrets, and its saved key is the placeholder. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
Review of the fork changes found three small things:
- Secret sources were deep-copied. A custom source that holds a lock or
a client made fork() raise ("cannot pickle '_thread.lock' object"),
after the fork's directory had already been written. They are now
copied like the agent and the analyzer, through a dump/validate round
trip (`_own_copy`), and the analyzer and the secrets are copied before
the fork is constructed, so a copy that fails leaves nothing behind.
- `invoked_skills` was not copied, next to two skill lists that are.
- The analyzer test passed with a shallow copy. It now uses a
ToolShield analyzer and checks that the guardrail LLM is the fork's
own object too.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
Coverage Report •
|
||||||||||||||||||||||||||||||
fork() copied the events but not the hook verdicts recorded against them. A pending action that a PreToolUse hook had denied was executed when the fork ran, and a user message that a UserPromptSubmit hook had blocked was sent to the model, while the source did neither. This is on main too; it is the same gap as the confirmation policy, a safety decision that a fork silently dropped, so it is closed here. The fork takes over `blocked_actions` and `blocked_messages` for the events it copies, and `last_user_message_id` (the last user message it actually has, for a fork of an earlier branch point). Found in review of this stack. Each new test fails without the change: the denied command runs, the blocked prompt reaches the model. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
`api_key` was a bare SecretStr with no serializer, so a saved conversation state always held the redaction placeholder for it, even with a cipher, and loading turned that placeholder into the key: the analyzer sent `Authorization: Bearer **********`. Only the GRAYSWAN_API_KEY environment variable hid this. The field now goes through `serialize_secret` / `validate_secret`: with a cipher the key is encrypted and comes back; without one it loads as no key, and the analyzer reports UNKNOWN as it does when none is set. This matters for this stack because a fork, and from the last PR every reload, takes the analyzer from base_state.json alone. Found in review of this stack. Both new tests fail without the change. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
Brings in the review fixes made on part 1: the shorter ACP masker comment, the masked string form of state update events with its test, and the feature-map note about changes made during a step. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…eps its secrets Two properties of the registry copy that had no test: - Replacing a secret on the fork must not change what the source masks. With a registry copied by `model_copy`, as #4813 first did, the two share one masking cache and the source stops masking its own value once the fork has resolved the new one. The test fails on that copy. - `fork(agent=...)` with an agent that brings secrets in its context has them next to the ones copied from the source. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
The helper that copies a model through its own dump and validate, with credentials kept, moves from local_conversation.py to utils/pydantic_secrets.py as copy_keeping_secrets. The agent-server makes the same kind of copy for a child conversation and can import it instead of spelling the round trip out again. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state # Conflicts: # openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
|
Heads-up for the stack: the #5465 part of this PR (cipher + secret registry on fork) comes from #5765, which is being driven to merge on its own now (merged with current This comment/PR was created by an AI agent (OpenHands) on behalf of Graham Neubig. |
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
Part of the fork's state was the hooks' verdicts on the events it copies, but not the hooks. A pending action the source's PreToolUse hook had denied stayed denied in the fork, and the next action the fork asked for ran with no hook at all. The fork is now built with the caller's own hook_config. The plugin hooks are not passed along: the fork loads the plugins itself, and the merged config would register them twice. Hooks of a plugin the source loaded at runtime are still not carried. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state # Conflicts: # .agents/skills/verify-agent-server/references/feature-map/F10-fork-navigate-condense-ask.md # .agents/skills/verify-sdk/references/feature-map/README.md
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state main at 3d638f2 brings #5765 (the fork's cipher and secret registry) and the SecretField refactor. What part 2 still adds to fork() stays on top of main's version: the policy, the analyzer copy, the hooks and their verdicts, invoked_skills, and one copy helper for the agent, the analyzer and the secret sources. The helper builds its context with client_context, and GraySwanAnalyzer.api_key is declared with SecretField instead of the deprecated validator pair. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
Agent-server conversation start benchmarkBase
How this is measuredEach create is a Canvas-shaped
A timing is flagged when the 95% CIs do not overlap and the medians differ by at least 50 ms, or when a row with at least 8 samples per side moves by at least 150 ms and 25%. A create stalled the server when its slowest |
The fork copied every secret source through a dump and validate round trip. A custom `SecretSource` that keeps its value in a private attribute or in a field the dump leaves out came out of that with no value (and a required field of that kind made `fork()` raise), where main's deep copy (#5765) keeps it. The same held for such a source in the agent's context. Secret sources are deep-copied again, as on main. A source the deep copy cannot handle (one that holds a lock or an open client, which makes `fork()` raise on main) is rebuilt from its fields instead. Tests for each case; each fails with only the other half in place. Found by the review of the merge with main. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
HUMAN:
Engel Nyst asked me to find the fork PR that removes the two sources of truth for a conversation's settings (
meta.jsonin the agent-server andbase_state.jsonin the SDK), which is #4813 by @BSmick6; to put its commits on branches in this repo, split so that each concern can be reviewed on its own; and to open new PRs that credit the author and say that we do the final review and tweaks. Then to review each PR, address what the review finds, and prove that the changes work by running the SDK and the agent-server the way a program or an agent does, with the repo'sverify-sdkandverify-agent-serverskills. The principle behind the request: the agent-server is a REST interface to the SDK. It should not be a different thing, and it should not repeat what the SDK already does, unless a particular case gives a special reason.This is part 2 of 3: a fork keeps the source's confirmation policy, security analyzer and secrets.
AGENT:
I'm Claude (Fable 5.2), an AI agent working for Engel Nyst. I brought #4813 up to date with
main, split it into a stack of three PRs, reviewed each, and verified them against real servers. The policy and analyzer copy in this PR is @BSmick6's, from #4813, with the analyzer handling from Engel Nyst's follow-up on that PR. The cipher and the secret registry come from #5765, merged in as it is. What the final review added is listed under "Final review".The stack: 1. #6762 (secrets are saved the moment they change) → 2. this PR → 3. #6764 (the agent-server stops keeping its own copy). This PR is based on part 1, so its diff shows only its own change.
Why
LocalConversation.fork()built the fork from the agent and the events. It did not copy the confirmation policy, the security analyzer or the secret registry, and it did not pass the source's cipher. Onmain:AlwaysConfirmorConfirmRiskystarts withNeverConfirmand runs its actions unasked;secrets=orupdate_secrets()) are missing on the fork; only those in the agent's own context come along with the agent;base_state.jsonholds the model key as the redaction placeholder, so a fork whose agent is loaded from that file has no model key and the provider rejects its first model call. The agent-server loads every fork that way.The agent-server hid part of this by copying the source's
meta.jsonto the fork and applying it on every load. #6764 removes that second copy, so the SDK has to carry these values itself.Summary
SecretRegistrywith copies of the source's secrets. Its tests come with it, including kyletser's restart test from fix(conversation): pass cipher to forked conversations (#5465) #5473. fix(agent-server): own confirmation_policy, security_analyzer, secrets in ConversationState #4813 copied the registry withmodel_copy, which shares the registry's private masking cache and lock between source and fork; fix(sdk): keep the source's cipher and secrets on a forked conversation #5765's fresh registry does not, so that version is the one kept.confirmation_policy(@BSmick6, fix(agent-server): own confirmation_policy, security_analyzer, secrets in ConversationState #4813).security_analyzerthrough a round trip that keeps nested credentials, so a stateful analyzer does not mix two conversations' history and a guardrail model key is re-encrypted for the fork instead of written redacted (Engel Nyst's follow-up on fix(agent-server): own confirmation_policy, security_analyzer, secrets in ConversationState #4813).Final review: what was found and changed
blocked_actions,blocked_messages,last_user_message_id) and were not copied. I reproduced it onmain: a pending action that aPreToolUsehook had denied on the source was executed when the fork ran, and a prompt that aUserPromptSubmithook had blocked was sent to the model. The fork now carries the verdicts for the events it copies, and the hooks themselves: it is built with thehook_configits source was created with. The verdicts alone would have been half a fix, since the next action the model asked for in an SDK-level fork ran without the caller'shook_confighooks; only the hooks of plugins given at construction, which the fork loads itself, still ran (a fork made through the agent-server kept explicit hooks, because the server passes its storedhook_configwhen it loads the fork). The plugin hooks are not handed over: the fork loads the plugins itself, and the merged config would register them twice. Six tests, all failing without the change.GraySwanAnalyzernever kept itsapi_keyacross a save, cipher or not: the field had no serializer, so it was written as the placeholder and read back as the literal string**********, which the analyzer then sent as its bearer token. This showed only when the key was passed as a field andGRAYSWAN_API_KEYwas not set, because that variable always replaces the field. The field now uses the same validator and serializer as every other credential: with a cipher the key is encrypted and comes back; without one it is still written as the placeholder but loads as no key, and the analyzer reportsUNKNOWNrisk instead of sending the placeholder. Two tests, both failing without the change. Found while checking what a forked analyzer has after a reload.copy_keeping_secrets, next to the SDK's other secret utilities, so the agent-server can use the same one for child conversations). Before, the same round trip was spelled out twice (agent, analyzer) while the secret sources were deep-copied.model_copy(deep=True), which fix(sdk): keep the source's cipher and secrets on a forked conversation #5765 used for them, raisesTypeErroron a custom secret source that holds a lock (cannot pickle '_thread.lock' object) or an HTTP client; the built-in sources,StaticSecretandLookupSecret, were not affected. The analyzer and the secret sources are copied under the source's lock, before the fork object exists, so a copy that fails leaves no fork directory behind. The agent's copy stays where it was onmain, before the lock is taken.invoked_skillsis copied too, next to the skill listsfork()already copied, so the fork reports the skills its source invoked (conversation.invoked_skills, andinvoked_skillsin the agent-server's conversation info) instead of an empty list.fork(agent=...)keeps its secrets. Newverify-sdksub-featureS04.fork-keeps-state, with a program that forks a ciphered conversation, changes the fork and loads it again from disk.Issue Number
Part of #4810.
How to Test
With
main'sfork()and this branch's tests, 15 of the command's 219 tests fail: eight intest_fork.py(test_fork_keeps_source_cipher_for_persisted_secrets,test_fork_keeps_confirmation_policy,test_fork_copies_secret_registry,test_fork_keeps_a_replacement_agents_own_secrets,test_fork_gives_independent_security_analyzer,test_fork_copies_a_secret_source_that_cannot_be_deep_copied,test_fork_copies_invoked_skills,test_fork_preserves_ciphered_security_analyzer_credentials), all six intest_fork_hook_blocks.py(two withassert ['drop-the-data'] == []: the command the hook denies ran), and the restart testtest_cipher_backed_fork_resumes_with_credentialsintest_conversation_service.py. On this branch all 219 pass.In this section
mainismainat3c66af1d, the commit this branch is merged with. Live, with the repo skills. The two server-level bullets run against a freshly launched agent-server per attempt, two attempts per side, andF10.fork-run-memoryuses a real model (DeepSeek flash): the fork has to finish a run and recall a codename only its source was told. The twoS04rows are onesdk-maprun per side; forS04.fork-keeps-state, whichmaindoes not have, themaincell is this branch's recipe run againstmain's SDK.mainF10.fork-keeps-credentialsF10.fork-run-memoryS04.fork-keeps-state(sdk-map run, new)fork_policy.kindisNeverConfirm,fork_analyzerisNoneType, the secrets arenull, no analyzer is saved (so no guard key either), a prompt sent to the fork is not blocked by the source's hookS04.fork(scripted, and example 48 with a real model)The two F10 bullets were replayed with
main's copy of the map, which still carries their known-bug markers, against a server launched frommainand then from this branch (--checkout). The markers themselves are removed by #5765's commits. With this branch's map the same bullets would then reportfailonmainandpasshere; without markers I ran them only on the top of the stack, where both pass.On this branch's final commit (
115395d6), the two SDK map families:S04local conversation (SDK map)S04.factorylabelled blocked in the mapS13security and confirmation (SDK map)S13.confirm-approve-reject,S13.llm-analyzerlabelled blocked in the mapThe whole agent-server families (
F04,F08,F10,F12,F31) and the client entry points were replayed on the top of the stack, which contains this commit; that table is in #6764.GitHub CI on
115395d6: all 14 checks that ran are green (the same two skipped by design as on #6762).Type
Notes
F10map edits. The policy, the analyzer, the hook verdicts,invoked_skills, the copy helper (which then replaces fix(sdk): keep the source's cipher and secrets on a forked conversation #5765's deep copy of the secret sources), the GraySwan fix, theS04.fork-keeps-staterecipe and their tests stay.meta.jsonwhen it opens the fork: the policy and analyzer are the ones the source was created with, and a secret from the start request that was changed later has its original value again. A secret added after creation is kept, with its value only if the server has a cipher. fix(agent-server): one owner for confirmation_policy, security_analyzer and secrets (base_state.json) #6764 has the server-level recipes for that (F10.fork-keeps-settings,F08.fork-after-restart).SessionStarton first use andSessionEndon close included. A fork made through the agent-server is not affected: the server never makes the SDK fork object ready, and loads the fork from disk with its stored hooks, as before.load_plugin(), or custom stuck-detection thresholds.mainand left for it:fork(reset_metrics=False)of a conversation that was resumed from disk stops counting what the fork spends, so the fork's budget cap never trips. Only an SDK fork used in-process is affected; a server fork is reloaded from disk and counts. I have a reproduction (a test file that fails onmain) and a suggested fix for an issue.maindo not run on it (among them REST contract, persisted settings, OpenAPI schema, binary and image build, TypeScript client). fix(agent-server): one owner for confirmation_policy, security_analyzer and secrets (base_state.json) #6764 lists local runs of the OpenAPI, REST-contract, persisted-settings and SDK-API checks on the top of the stack; the binary build and the TypeScript client CI were not run locally (the TypeScript client was exercised live as map familyF34).🤖 Generated with Claude Code
https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY