Skip to content

perf(session): keep the listing scan of an unchanged transcript - #1345

Merged
LivXue merged 4 commits into
EverMind-AI:mainfrom
Lumea-art:perf/session-list-scan-cache
Nov 18, 2026
Merged

LivXue merged 4 commits into
EverMind-AI:mainfrom
Lumea-art:perf/session-list-scan-cache

Conversation

@Lumea-art

@Lumea-art Lumea-art commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

SessionManager.list_sessions reads every *.jsonl transcript in full on every call (_scan_file parses each line to produce message_count, last_message_at, first_user_message and last_message_preview). The session.list RPC, the ACP listing and raven sessions list all go through it, so a picker that polls re-reads the entire transcript corpus each time even though most files have not changed.

_scan_listed now keeps the scan result keyed by the file's (st_mtime_ns, st_size), so an unchanged transcript is not read again. The cache is bounded (2048 entries, LRU) and process-local.

Key decisions:

  • Writes made through this manager (save, delete, append_metadata_patch) drop the path's entry explicitly. The stat key alone can miss a same-size rewrite that lands in the same mtime tick (measured on NTFS); an explicit drop makes same-process invalidation deterministic.
  • list_sessions hands out a shallow copy of the cached metadata dict, so editing a top-level key of a returned record does not pollute the cache. Nested dicts and lists are still shared with the cache; that is an intentional trade-off for not deep-copying every listing, not an isolation guarantee.
  • A legacy metadata record with no metadata key lists as {} again, the shape the listing returned before this cache; the cached scan hands the raw record back, so the listing applies the default itself.
  • The cache is documented as single-threaded (every caller runs synchronously on the event-loop thread) and as churning past 2048 distinct transcripts; an eviction costs a rescan. The stamp is the whole invalidation criterion, so a same-size rewrite by another process inside the same mtime tick can still return a stale listing -- a known residual risk, and as far as this criterion can go.

Measurement

Synthetic corpus matching the on-disk layout (sessions/<channel>/<chat_id>.jsonl), 200 sessions x 200 messages, 13.2 MB total, Python 3.12, Ubuntu 22.04, warm cache. Measured by the author:

first call repeat call (median of 5)
before 84.0 ms 84.3 ms
after 83.8 ms 0.7 to 0.8 ms

About 105x on the repeat path, which is the path a picker takes. The first call is unchanged: nothing is skipped that was being done, only repeated.

Why (mtime_ns, size) is the right key

A transcript only changes when a turn is appended, and an append moves both fields. The stat is taken before the read, so a write that lands during a scan leaves the recorded stamp older than the file: the next poll sees a newer mtime and scans again. A stamp that is too stale costs a rescan; the one case it cannot see is a same-size rewrite by another process inside the same mtime tick (the residual risk noted under Risk).

An unreadable (non-UTF-8) transcript is cached with its None result, so a file that keeps failing does not turn every poll into a decode error path.

Tests

Eleven added to tests/test_session_manager.py:

  • test_list_sessions_scans_an_unchanged_transcript_once - two lists, one _scan_file call, identical results.
  • test_list_sessions_rescans_a_transcript_that_grew - after appending a message the listing reports the new message_count.
  • test_list_sessions_skips_a_transcript_it_cannot_stat - a path that vanishes between glob and stat is left out.
  • test_the_scan_cache_evicts_the_least_recently_used_entry - past the limit the oldest entry goes.
  • test_list_sessions_rescans_an_equal_size_rewrite_with_a_newer_mtime - a same-size rewrite is seen through the mtime.
  • test_list_sessions_rescans_after_an_undecodable_transcript_becomes_readable - a cached decode failure is retried once the file is valid.
  • test_save_invalidates_the_scan_cache_when_the_stamp_would_not_change - a rewrite through save of the same size with the old mtime put back is still listed fresh.
  • test_append_metadata_patch_invalidates_the_scan_cache - a patch is visible to the next listing even when the cache entry carries the stamp the append leaves.
  • test_a_session_recreated_after_delete_is_scanned_fresh - a same-key session written at the deleted path after delete is not read from the old entry.
  • test_list_sessions_metadata_is_a_top_level_copy - editing a top-level key of a returned record does not change the next listing.
  • test_list_sessions_reports_empty_metadata_for_a_legacy_record - a record without a metadata key lists as {}, not None.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

The commit type is perf; this template has no Performance option, so Other is the closest match.

Verification

  • Relevant tests pass locally
  • Relevant lint / type checks pass locally
  • User-facing docs or screenshots are updated when needed

No user-facing docs or screenshots changed: this is an internal listing cache with no API, RPC schema, or UI change.

Environment: Windows 11, Python 3.12. Dependencies synced with uv sync --locked --extra dev --dev (the all-extras set does not build here: datrie needs MSVC).

PYTHONUTF8=1 .venv/Scripts/python.exe -m pytest -q tests/test_session_manager.py
# 162 passed in 7.63s

# each added regression test, by node id
# 5 passed

PYTHONUTF8=1 .venv/Scripts/python.exe -m ruff check raven/session/manager.py tests/test_session_manager.py
# All checks passed!

PYTHONUTF8=1 .venv/Scripts/python.exe -m ruff format --check raven/session/manager.py tests/test_session_manager.py
# 2 files already formatted

git diff --check
# no output, exit 0

Mutation check, run against the same test file:

  • With all three self._scan_cache.pop(path, None) calls removed and the metadata copy reverted to the raw cached dict, the four cache regression tests fail (4 failed, 1 passed; the legacy-shape test is unaffected by this mutation).
  • Removing one pop at a time fails exactly its test: the append pop fails test_append_metadata_patch_invalidates_the_scan_cache, the save pop fails test_save_invalidates_the_scan_cache_when_the_stamp_would_not_change, and the delete pop fails test_a_session_recreated_after_delete_is_scanned_fresh (each 1 failed, 3 passed).
  • Reverting the legacy default to data.get("metadata") fails test_list_sessions_reports_empty_metadata_for_a_legacy_record (assert None == {}).
  • Restored from the byte-for-byte backup (cmp clean); the file then reports 162 passed.

The full pytest suite was not run on this machine; the change is confined to the session listing path.

Risk

  • Security impact considered

  • Backward compatibility considered

  • Rollback path is clear for risky changes

  • Security: no new input surface, no file format or credential change; the cache holds parsed listing data in memory only.

  • Backward compatibility: no API, RPC schema or on-disk format change. The cache is process-local; another process's write is still noticed through (mtime_ns, size). The residual blind spot is a same-size rewrite by another process inside the same mtime tick; normal append and truncate paths change the size. A legacy record without a metadata key keeps its historical {} listing shape.

  • Rollback: revert raven/session/manager.py and tests/test_session_manager.py; there is no migration or persistent state to undo.

Related Issues

N/A

list_sessions reads every transcript in full on every call, and a picker polls it. Cache the scan result per file, keyed by (mtime_ns, size), so an unchanged transcript is not read again: 13.2 MB over 200 sessions goes from 84 ms per call to 0.8 ms on the repeat path, with the first call unchanged.
@Lumea-art
Lumea-art force-pushed the perf/session-list-scan-cache branch from a902082 to 3c8ccaa Compare November 17, 2026 14:10
@LivXue

LivXue commented Nov 17, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — 3c8ccaad1a

Reviewed at head 3c8ccaad1a94b1a3587c7bc3e0c7164e7cb7c180 (base a639935478; the branch is cut against a base that is not an ancestor — its own diff is the two files listed on the PR: raven/session/manager.py +27/-2 and tests/test_session_manager.py +37). Scope: _scan_listed's new _scan_cache, the _scan_file / _stamp / save paths it has to stay consistent with, and the listing caller at raven/session/manager.py:1449.

No Windows compatibility issue found.

The cache key is (st_mtime_ns, st_size), and _scan_listed re-stats before trusting it, so the two ordinary ways a transcript moves — an appended turn and a rewritten file from undo_last_turn/clear — both land on a fresh scan as long as either value moves. Reading the stat before the scan (rather than after) is the safe direction, as the comment says: a write during the scan makes the next poll see a newer stamp. Path keys are Path objects, which compare case-insensitively on Windows, so two spellings of one transcript share one entry rather than duplicating it; Path.stat() follows the file, and the rewrite path already replaces atomically, so a listing never reads a half-written file.

One observation, not a defect and not introduced as a behaviour change in spirit: the (mtime_ns, size) pair is the same freshness rule save() has used all along (_stamp at raven/session/manager.py:208), so the new cache inherits its one blind spot — a filesystem whose mtime granularity is coarser than the write interval. On NTFS the file time is system-clock driven (~15.6 ms), and on exFAT/FAT volumes mounted on Windows, or an SMB share, st_mtime_ns can move in 2-second steps. An undo_last_turn followed by a turn of the same byte length inside one such tick would leave (mtime_ns, size) unchanged while the content differs, and the picker would keep the older preview and count until the file is appended to again. Impact is cosmetic (a listing preview), the same class the save path already accepts, and nothing here makes it worse. If it is ever worth tightening, hashing the first kilobyte alongside the pair would close it.

Verification (native Windows host, Python 3.12.13, shared venv, detached worktree pinned to the head):

  • python -S runner.py tests/test_session_manager.py -q → 153 passed in 21.83s.

Coverage gaps: no live gateway picker polled the listing; the cache's behaviour on an exFAT/SMB volume was reasoned from the platform rather than measured on one.

The stat now happens before the read, so a path that disappears between the glob and the stat takes the empty branch rather than the old open() OSError one. Pin it.
@Lumea-art

Copy link
Copy Markdown
Contributor Author

Heads up on the red run at 3c8ccaa: it is not this change. Shard 4/4 passed every test (11694 passed, 123 skipped) and then the idle ceiling failed the session:

= 1 unmarked test(s) waited more than 3s (wall clock minus CPU and run-queue wait) (failing the session) =
   3.59s idle of   3.80s (0.05s queued)  tests/test_knowledge_parser_pdf.py::test_a_picture_page_with_a_caption_is_still_read_as_a_scan
mark it slow or production_timing with the reason, or take the wait out of the test

That test is untouched by this PR and passes locally in 4.02s; it looks like the PDF parse crossed the 3s idle line on a loaded runner. I could not re-run the job myself (fork contributors do not get the rerun button - Must have admin rights to Repository), so I pushed f5db619 to trigger a fresh run rather than leave the PR red.

f5db619 is not filler: moving the stat before the read gives the vanished-path case its own branch (the old open() raised OSError inside the scan, the new stat raises it before the scan), so it now has a test. That is the one branch of the new code that had no coverage.

For whoever re-runs or reviews: if this test trips the ceiling again on a later run, the honest fix is the marker on that test (it is a real PDF parse, not a wait), and I would rather that land as its own small change than be smuggled into this one.

@LivXue

LivXue commented Nov 18, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — f5db619e85

Reviewed at head f5db619e85cde6b8c8e31feae9121e626265a24c (base a639935478; this head follows 3c8ccaad1a, reviewed earlier — the only change since is one added test, test_list_sessions_skips_a_transcript_it_cannot_stat, which pins the OSError-from-stat path _scan_listed already handles). Scope: the PR's own diff (raven/session/manager.py +27/-2, tests/test_session_manager.py +57) and the cache it adds.

No Windows compatibility issue found. The earlier review's analysis stands: _scan_listed re-stats before trusting (st_mtime_ns, st_size), the stat is taken before the scan so a concurrent append makes the next poll rescan, and Path keys compare case-insensitively on Windows. The new test exercises the vanished-file path this platform produces when a session is removed between the glob and the stat, and it passes here.

Verification (native Windows host, Python 3.12.13, shared venv, detached worktree pinned to the head):

  • python -S runner.py tests/test_session_manager.py -q → 154 passed in 16.67s (153 before this head, plus the new test).

Coverage gaps: no live gateway picker polled the listing; the one observation from the earlier review (a coarse-mtime volume can hide a same-size rewrite until the next append) remains a documented property of the (mtime, size) rule the save path already uses, not a defect.

save(), delete() and append_metadata_patch() now drop the path's listing
entry instead of relying on the (mtime_ns, size) stamp alone. A same-size
rewrite can land in the same mtime tick on NTFS, which would otherwise
serve the previous listing. list_sessions() also copies the cached
metadata dict before handing it out, and the cache docs now state the
bounds and the single-thread assumption.

Add tests for LRU eviction, a same-size rewrite with a newer mtime, and
recovery from a cached decode failure.

Co-authored-by: Claude (deepseek-flash) <noreply@anthropic.com>
@LivXue

LivXue commented Nov 18, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — 2c7623c271

Reviewed at head 2c7623c271d931c9825f6b77d32764a33c30d1bd (base a6399354789). Scope: the delta over the previously reviewed f5db619e85 — raven/session/manager.py (+31/-1) and tests/test_session_manager.py (+65), read against the earlier review's observation about coarse-mtime, same-size rewrites.

No Windows compatibility issue found.

  • The new commit closes that observation in the right direction: this process's own writes (save, delete, append_metadata_patch) now drop the _scan_cache entry explicitly instead of trusting the (st_mtime_ns, st_size) stamp, whose mtime can repeat within one NTFS clock tick on a same-size rewrite. The tests pin the LRU eviction (_SCAN_CACHE_LIMIT lowered to 2) and a same-size rewrite with a newer mtime.
  • list_sessions now hands the caller dict(metadata) instead of the cache's own dict, so a caller mutating the returned metadata cannot corrupt the cached record — a real aliasing bug at the previous head, fixed here.
  • The new docstrings state the two limits honestly (LRU churn above 2048 transcripts costs a rescan, never a stale result; the get/move/evict sequence assumes the event-loop-thread-only usage), and delete() pops after session_path(key) is assigned (line 1315 before 1324), so no unbound name.

Verification, native Windows host (GitHub tarball of 2c7623c271, extracted under D:\tmp\rv1345):

  • python -S runner.py tests/test_session_manager.py -q → 157 passed (155 before + the 2 new tests)

Gaps: a same-tick, same-size rewrite from another process is still only noticed by the stamp — inherent to mtime-keyed caches and documented in the code; the tree is the commit's export rather than a git worktree.

The three explicit pops in save, append_metadata_patch and delete had no
test that failed without them: a same-size rewrite landing in the same
mtime tick was served from the cache, and the top-level metadata copy
was not pinned either. Each new test seeds the colliding stamp so the
pop it covers is what keeps the listing fresh.

A record without a metadata key now lists as {} again, the shape the
listing returned before the scan cache landed; the cached scan hands the
raw record back, so the listing has to apply the default itself.

Co-Authored-By: Claude Code <noreply@anthropic.com>

@LivXue LivXue left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds an in-process LRU cache to the session listing scan, keyed on (st_mtime_ns, size), capped at 2048 entries, so the picker stops re-parsing unchanged transcripts on every poll. Explicit invalidation was added at save(), append_metadata_patch() and delete(), so an in-process write never relies on the stamp to be noticed. Mutation testing backs the new guards: with all three pop() calls removed and the shallow copy reverted, the newly added regressions fail, whereas before they were added the full file still passed 157. Known boundaries are documented in the description: an equal-size rewrite by another process inside the same mtime tick can still return a stale row, and the metadata copy protects top-level keys only, with nested values shared (all current callers are read-only). No configuration schema, RPC contract or file format changed; rollback is one commit.

@LivXue
LivXue merged commit 90714e7 into EverMind-AI:main Nov 18, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants