Repository navigation
Conversation
A turn was written once, at its end, so a process killed at its budget left only the question behind and nothing could be recovered from the attempt. Rows now reach the file as the turn produces them: before a tool call runs, and at the start of every iteration. A row whose tool call never got an answer is repaired when a transcript is read -- as an [interrupted] tool result -- rather than left for a strict provider to reject. Each row carries a stable row id. The stamps a turn files at its end, the reply's provenance and the hook observers, land after those rows are already on disk, so they travel as one turn_meta record that restates the row by id; the reader merges it in file order. A row already on disk is never written a second time, and the save boundary is read from the ids that were written rather than from a count, because a continuation can merge its row into the step it continues and leave the record shorter than it was. The turn log gained adopt_appended: a list is adopted only when it is the record grown at the end. A window pass's shrunken copy is a view of the record, not the record, and following it would drop rows the turn produced.
gloryfromca
left a comment
There was a problem hiding this comment.
Blocking: recovery scaffolding must stay transient, and interrupted tool-call repair must preserve provider-valid ordering across restarts.
I reviewed github/main...HEAD, the surrounding turn-log/session writer and provider serialization paths, relevant callers, the branch history, backward compatibility, and the repository rules in AGENTS.md, CONTEXT-MAP.md, and CONTEXT.md. I also checked that the added tests were not weakened; they pass but do not cover either failing lifecycle below.
Verification:
uv run pytest tests/test_agent_loop_session_persistence.py -q -n 0- 4 passeduv run pytest tests/test_agent_loop_broken_turn.py tests/test_agent_loop_session_stamps.py tests/test_session_manager.py tests/test_session_revision.py tests/test_session_turn_shape.py -q -n 0- 311 passeduv run pytest tests/test_agent_loop_dead_end_retry.py -q -n 0- 45 passed- Direct reproductions confirmed a
_recovery_syntheticrow is written to JSONL and a crash/resume/save/restart history reloads as assistant tool call, user, assistant, then synthetic tool result.
The first combined test attempt produced 29 setup errors because the fresh environment lacked the workspace raven_everos package. After uv sync --all-packages, the commands above passed.
| keys = patch.get("keys") | ||
| if index is not None and isinstance(keys, dict): | ||
| messages[index].update(keys) | ||
| messages = repair_dangling_tool_calls(messages) |
There was a problem hiding this comment.
Blocking: place interrupted results at the matching call boundary while preserving disk accounting.
repair_dangling_tool_calls() appends every repair to the end. That works only while the interrupted call is the file tail. In the ordinary crash recovery lifecycle, the first reload appends a synthetic result in memory, _persisted_count counts it although it is intentionally absent from disk, and the next save appends the resumed user/assistant rows without that result. A second restart then reloads assistant(tool_calls), user, assistant, tool, which strict chat providers reject because the tool result no longer immediately follows its call. The added test already constructs a later assistant row but asserts only that a tool row exists, not its provider-valid position.
gloryfromca
left a comment
There was a problem hiding this comment.
Blocking: the two previously reported persistence defects remain unchanged.
The new commit only makes the two session-row scans tolerate session-like objects without a messages attribute. I reviewed that delta and the full resulting diff, affected callers and provider serialization, branch history, backward compatibility, tests, and the repository rules in AGENTS.md, CONTEXT-MAP.md, and CONTEXT.md. It does not change incremental persistence's unfiltered log.messages() call or the end-appending interrupted-call repair, so both existing threads still stand.
Verification on this head: uv run pytest tests/test_agent_loop_session_persistence.py tests/test_agent_loop_broken_turn.py tests/test_agent_loop_session_stamps.py tests/test_agent_loop_dead_end_retry.py tests/test_agent_loop_turn_record.py tests/test_session_manager.py tests/test_session_revision.py tests/test_session_turn_shape.py -q -n 0 passed, 381 tests. The crash/resume/save/restart reproduction still reloads the rows as assistant tool call, user, assistant, then tool result.
gloryfromca
left a comment
There was a problem hiding this comment.
Blocking: interrupted tool-call repair must preserve provider-valid ordering across resume and restart.
I reviewed the new delta and the full resulting diff, affected session and provider callers, branch history, backward compatibility, test changes, and the repository rules in AGENTS.md, CONTEXT-MAP.md, and CONTEXT.md. The row-ID cleanup and added late-stamp mappings are coherent, and the changed test still verifies the metadata patch while additionally checking that the filing key stays internal.
I withdrew and resolved my recovery-scaffolding finding after refuting it against _save_turn's existing final gate. The interrupted-call finding still reproduces: after crash, resume/save, and restart, the loaded order is assistant tool call, user, assistant, then tool result. At this fourth review round I am keeping that item blocking because all three convergence conditions hold: this change introduces the read-time repair/accounting mismatch; interruption followed by resuming and later restarting is ordinary operation; and once strict providers reject the malformed history, the affected session has no in-session recovery path short of discarding or manually editing its history.
Verification: uv run pytest tests/test_agent_loop_session_persistence.py tests/test_agent_loop_broken_turn.py tests/test_agent_loop_session_stamps.py tests/test_agent_loop_dead_end_retry.py tests/test_agent_loop_turn_record.py tests/test_session_manager.py tests/test_session_revision.py tests/test_session_turn_shape.py -q -n 0 passed, 381 tests. Direct checks confirmed synthetic recovery rows are not persisted and the interrupted-call ordering failure remains.
gloryfromca
left a comment
There was a problem hiding this comment.
Blocking: the existing interrupted tool-call ordering defect remains.
This delta only defers incremental filing at picture-bearing rows until the window replaces the picture with its durable note. I reviewed that change and the full resulting diff, affected window/session/provider callers, branch history, backward compatibility, test coverage, and the repository rules in AGENTS.md, CONTEXT-MAP.md, and CONTEXT.md; I found no new high-bar issue in the image boundary. The open interrupted-call thread is untouched, its lifecycle still reproduces on this head, and the convergence conditions stated in the prior review remain satisfied.
Verification: uv run pytest tests/test_agent_loop_session_persistence.py tests/test_agent_loop_broken_turn.py tests/test_agent_loop_session_stamps.py tests/test_agent_loop_dead_end_retry.py tests/test_agent_loop_turn_record.py tests/test_agent_window_parts.py tests/test_read_file_image.py tests/test_session_manager.py tests/test_session_revision.py tests/test_session_turn_shape.py -q -n 0 passed, 520 tests, with two unrelated Pydantic warnings from Ollama TypedDict schemas.
gloryfromca
left a comment
There was a problem hiding this comment.
Blocking: the existing interrupted tool-call ordering defect remains.
This revision extracts the picture-row boundary into _incremental_upto and adds focused coverage for that helper and TurnLog.adopt_appended; those tests accurately preserve the implemented behavior. I reviewed the delta and full resulting diff, affected callers, history, backward compatibility, test changes, and the repository rules in AGENTS.md, CONTEXT-MAP.md, and CONTEXT.md. I found no new high-bar issue, but the open blocker is untouched and still reproduces on this head; the previously stated convergence conditions remain satisfied.
Verification: uv run pytest tests/test_agent_loop_session_persistence.py tests/test_agent_loop_broken_turn.py tests/test_agent_loop_session_stamps.py tests/test_agent_loop_dead_end_retry.py tests/test_agent_loop_turn_record.py tests/test_agent_window_parts.py tests/test_read_file_image.py tests/test_session_manager.py tests/test_session_revision.py tests/test_session_turn_shape.py -q -n 0 passed, 522 tests, with two unrelated Pydantic warnings from Ollama TypedDict schemas.
Summary
A turn used to reach the session file once, at its end. A process killed at its budget -- which is what a benchmark or a channel turn does when the budget runs out -- left only the question behind, and nothing could be recovered from the attempt.
This change writes the turn as it happens:
read_session_fileappends an[interrupted]tool result for it. A strict provider rejects a history whosetool_callshave no matching tool message, and the repair belongs on the way in -- the file keeps saying what actually happened. The same helper now serves the cancelled-turn path, so there is one implementation._row_id, minted by the turn log when it records the row.replied_via_tooland the hookobservers(which ACP reads as_meta, including the hold marker) land after the rows are on disk, so they are written as one{"_type": "turn_meta", "patches": [{"row_id", "keys"}]}record that the reader merges in file order. Patching by position was tried and does not work: the record can be shorter than the list that was written.adopt_appended: a list is adopted only when it is the record grown at the end. A window pass's shrunken copy is a view of the record, and following it drops rows the turn produced.The session file format gains two record types (
_row_idon rows, and theturn_metarecord). Every reader goes throughread_session_file, which applies the patches; the replay, export and ACP paths read through it.Type
Verification
uv run pytest tests/test_agent_loop_session_persistence.py tests/test_agent_loop_empty_recovery.py tests/test_session_manager.py -q-> 650 passeduv run pytest tests/test_agent_loop_session_persistence.py tests/test_agent_loop_iteration_hooks.py tests/test_agent_loop_dead_end.py tests/test_agent_loop_dead_end_retry.py tests/test_session_manager.py tests/test_acp_methods.py -q-> 421 passeduv run ruff checkanduv run ruff format --checkpass on every touched file.New tests: a tool round is on disk before the next call; an assistant call with no result is repaired when the file is read; the closing answer is filed exactly once and no row is written under two ids; a
turn_metarecord restates a row by its identifier.Relevant tests pass locally
Relevant lint / type checks pass locally
User-facing docs or screenshots are updated when needed
Risk
The session file now carries
_row_idon rows andturn_metarecords. Older readers that build history from raw rows would see the extra key (transports send an allowlist, so it does not reach a provider); this change updates the one reader,read_session_file, and everything else reads through it.A turn now appends several times instead of once. The cost is one locked append per step; it was not benchmarked here beyond the suite's timing, which did not move.
If the process is killed with SIGKILL mid-step, the last streamed text is still lost. That is the remaining gap this change narrows but does not close.
Rollback: revert the commit. Sessions written meanwhile stay readable:
turn_metais ignored by older readers and_row_idis an extra key.Security impact considered
Backward compatibility considered
Rollback path is clear for risky changes
Related Issues
N/A