Repository navigation
perf(session): keep the listing scan of an unchanged transcript - #1345
Conversation
list_sessions reads every transcript in full on every call, and a picker polls it. Cache the scan result per file, keyed by (mtime_ns, size), so an unchanged transcript is not read again: 13.2 MB over 200 sessions goes from 84 ms per call to 0.8 ms on the repeat path, with the first call unchanged.
a902082 to
3c8ccaa
Compare
Windows compatibility review —
|
The stat now happens before the read, so a path that disappears between the glob and the stat takes the empty branch rather than the old open() OSError one. Pin it.
|
Heads up on the red run at That test is untouched by this PR and passes locally in 4.02s; it looks like the PDF parse crossed the 3s idle line on a loaded runner. I could not re-run the job myself (fork contributors do not get the rerun button -
For whoever re-runs or reviews: if this test trips the ceiling again on a later run, the honest fix is the marker on that test (it is a real PDF parse, not a wait), and I would rather that land as its own small change than be smuggled into this one. |
Windows compatibility review —
|
save(), delete() and append_metadata_patch() now drop the path's listing entry instead of relying on the (mtime_ns, size) stamp alone. A same-size rewrite can land in the same mtime tick on NTFS, which would otherwise serve the previous listing. list_sessions() also copies the cached metadata dict before handing it out, and the cache docs now state the bounds and the single-thread assumption. Add tests for LRU eviction, a same-size rewrite with a newer mtime, and recovery from a cached decode failure. Co-authored-by: Claude (deepseek-flash) <noreply@anthropic.com>
Windows compatibility review —
|
The three explicit pops in save, append_metadata_patch and delete had no
test that failed without them: a same-size rewrite landing in the same
mtime tick was served from the cache, and the top-level metadata copy
was not pinned either. Each new test seeds the colliding stamp so the
pop it covers is what keeps the listing fresh.
A record without a metadata key now lists as {} again, the shape the
listing returned before the scan cache landed; the cached scan hands the
raw record back, so the listing has to apply the default itself.
Co-Authored-By: Claude Code <noreply@anthropic.com>
LivXue
left a comment
There was a problem hiding this comment.
This adds an in-process LRU cache to the session listing scan, keyed on (st_mtime_ns, size), capped at 2048 entries, so the picker stops re-parsing unchanged transcripts on every poll. Explicit invalidation was added at save(), append_metadata_patch() and delete(), so an in-process write never relies on the stamp to be noticed. Mutation testing backs the new guards: with all three pop() calls removed and the shallow copy reverted, the newly added regressions fail, whereas before they were added the full file still passed 157. Known boundaries are documented in the description: an equal-size rewrite by another process inside the same mtime tick can still return a stale row, and the metadata copy protects top-level keys only, with nested values shared (all current callers are read-only). No configuration schema, RPC contract or file format changed; rollback is one commit.
Summary
SessionManager.list_sessionsreads every*.jsonltranscript in full on every call (_scan_fileparses each line to producemessage_count,last_message_at,first_user_messageandlast_message_preview). Thesession.listRPC, the ACP listing andraven sessions listall go through it, so a picker that polls re-reads the entire transcript corpus each time even though most files have not changed._scan_listednow keeps the scan result keyed by the file's(st_mtime_ns, st_size), so an unchanged transcript is not read again. The cache is bounded (2048 entries, LRU) and process-local.Key decisions:
save,delete,append_metadata_patch) drop the path's entry explicitly. The stat key alone can miss a same-size rewrite that lands in the same mtime tick (measured on NTFS); an explicit drop makes same-process invalidation deterministic.list_sessionshands out a shallow copy of the cachedmetadatadict, so editing a top-level key of a returned record does not pollute the cache. Nested dicts and lists are still shared with the cache; that is an intentional trade-off for not deep-copying every listing, not an isolation guarantee.metadatakey lists as{}again, the shape the listing returned before this cache; the cached scan hands the raw record back, so the listing applies the default itself.Measurement
Synthetic corpus matching the on-disk layout (
sessions/<channel>/<chat_id>.jsonl), 200 sessions x 200 messages, 13.2 MB total, Python 3.12, Ubuntu 22.04, warm cache. Measured by the author:About 105x on the repeat path, which is the path a picker takes. The first call is unchanged: nothing is skipped that was being done, only repeated.
Why
(mtime_ns, size)is the right keyA transcript only changes when a turn is appended, and an append moves both fields. The stat is taken before the read, so a write that lands during a scan leaves the recorded stamp older than the file: the next poll sees a newer
mtimeand scans again. A stamp that is too stale costs a rescan; the one case it cannot see is a same-size rewrite by another process inside the same mtime tick (the residual risk noted under Risk).An unreadable (non-UTF-8) transcript is cached with its
Noneresult, so a file that keeps failing does not turn every poll into a decode error path.Tests
Eleven added to
tests/test_session_manager.py:test_list_sessions_scans_an_unchanged_transcript_once- two lists, one_scan_filecall, identical results.test_list_sessions_rescans_a_transcript_that_grew- after appending a message the listing reports the newmessage_count.test_list_sessions_skips_a_transcript_it_cannot_stat- a path that vanishes between glob and stat is left out.test_the_scan_cache_evicts_the_least_recently_used_entry- past the limit the oldest entry goes.test_list_sessions_rescans_an_equal_size_rewrite_with_a_newer_mtime- a same-size rewrite is seen through the mtime.test_list_sessions_rescans_after_an_undecodable_transcript_becomes_readable- a cached decode failure is retried once the file is valid.test_save_invalidates_the_scan_cache_when_the_stamp_would_not_change- a rewrite throughsaveof the same size with the old mtime put back is still listed fresh.test_append_metadata_patch_invalidates_the_scan_cache- a patch is visible to the next listing even when the cache entry carries the stamp the append leaves.test_a_session_recreated_after_delete_is_scanned_fresh- a same-key session written at the deleted path afterdeleteis not read from the old entry.test_list_sessions_metadata_is_a_top_level_copy- editing a top-level key of a returned record does not change the next listing.test_list_sessions_reports_empty_metadata_for_a_legacy_record- a record without ametadatakey lists as{}, notNone.Type
The commit type is
perf; this template has no Performance option, soOtheris the closest match.Verification
No user-facing docs or screenshots changed: this is an internal listing cache with no API, RPC schema, or UI change.
Environment: Windows 11, Python 3.12. Dependencies synced with
uv sync --locked --extra dev --dev(theall-extrasset does not build here:datrieneeds MSVC).Mutation check, run against the same test file:
self._scan_cache.pop(path, None)calls removed and the metadata copy reverted to the raw cached dict, the four cache regression tests fail (4 failed, 1 passed; the legacy-shape test is unaffected by this mutation).test_append_metadata_patch_invalidates_the_scan_cache, the save pop failstest_save_invalidates_the_scan_cache_when_the_stamp_would_not_change, and the delete pop failstest_a_session_recreated_after_delete_is_scanned_fresh(each1 failed, 3 passed).data.get("metadata")failstest_list_sessions_reports_empty_metadata_for_a_legacy_record(assert None == {}).cmpclean); the file then reports162 passed.The full
pytestsuite was not run on this machine; the change is confined to the session listing path.Risk
Security impact considered
Backward compatibility considered
Rollback path is clear for risky changes
Security: no new input surface, no file format or credential change; the cache holds parsed listing data in memory only.
Backward compatibility: no API, RPC schema or on-disk format change. The cache is process-local; another process's write is still noticed through
(mtime_ns, size). The residual blind spot is a same-size rewrite by another process inside the same mtime tick; normal append and truncate paths change the size. A legacy record without ametadatakey keeps its historical{}listing shape.Rollback: revert
raven/session/manager.pyandtests/test_session_manager.py; there is no migration or persistent state to undo.Related Issues
N/A