Repository navigation
Freeze memtomem stores written by v0.2.0 and v0.3.0 and read them with the current readers (#271) - #317
Merged
Conversation
…t readers (#271) First part of the Memory backends family: one release, as the template for v0.1.2 and v0.2.0. tests/fixtures/persisted/memory/v0.3.0/store is a memtomem store the v0.3.0 CLI wrote (freeze.py beside it): run under every strategy, a failed run, a board run, a room apply, and last a board run SIGKILLed during its second pipeline stage, so the store is as that kill left it, with its newest rows in the WAL and a live lease on the board. release-read.json records what the v0.3.0 interpreter read from it; index.json holds the scenario definitions and a sha256 of every file. test_frozen_memory.py runs in the default suite with no memtomem engine: the stored rows against the scenario definitions, the day-file names and namespaces HEAD derives, and that chunks.source_file is the only column binding a store to where it was written. test_frozen_memory_real.py runs with SYNCMILL_TEST_MEMTOMEM_REAL=1 against real memtomem: opening the store, the adapter's reads and writes beside the released rows, a scratch purge, a reindex, SyncmillMemory, the gateway, status, the filters each strategy sends, a run and a room round carrying a released result, the store found at the project default location, and a released lease reclaimed and its scratch namespace purged. Not in this commit: the v0.1.2 and v0.2.0 stores, the persisted-contracts row, and the CI line that adds the real file to the real-memtomem job. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…271) Answers three review rounds of the v0.3.0 template. test_frozen_memory_real.py: - A HEAD strategy now runs a task whose every word is in the released strategy results. memtomem tries an AND match and falls back to OR only when that finds nothing, so a word the released rows lacked left every later phase with the run's own rows alone. Each recall must now return released rows of the kinds it asks for, none of another kind, and every agent of that phase must be handed them. - The first open of a moved, not rebound copy (the SIGKILL's WAL still beside it) reads every row and searches every namespace, not only counts. - A fresh share is checked against the exact set of released shared rows; the old set comparison could not fail. - rebind compares the directory prefix exactly, separator included, instead of a LIKE pattern in which a temporary path's `_` is a wildcard. test_frozen_memory.py: - The killed run's row is absent from the main database file alone (its bytes are taken before any connection opens the copy, since closing one checkpoints the WAL). - The store records embedding_provider none and has no vector table. - The release read searched every namespace by every kind; the memtomem version is compared as numbers. freeze.py (the v0.3.0 output is unchanged; regenerated into scratch: same 13 scenarios and files, the namespace-hash lock names aside): - Every earlier row must stay exactly as stored, ids and full source path included, after each scenario. - The killed run's row must be in the WAL only. - Databases the release wrote are read through a scratch copy, rooms too. - Output is published whole: a hidden sibling, then one rename after the target is checked again; the sibling is removed on any failure, and a symlink at the target is refused, not followed. - A stalled killed run kills its whole process group before reading output. CI's real-memtomem job now runs test_frozen_memory_real.py. Not here: the memory migration reader over these stores (`memory migration --from memtomem`), which the plan puts in Memory-2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ly (#271) tests/fixtures/persisted/memory/v0.2.0 is written by the v0.2.0 CLI with the same freeze.py as v0.3.0. Both releases lock memtomem 0.6.8, and every one of the 13 scenarios left the same rows as at v0.3.0, ids aside. The default and real-engine tests now run over both releases. persisted-contracts.md gets a "Memory (memtomem)" row (documents, readers and what is not covered); Mem0 and LangGraph stay "not yet" in a row of their own. The real smoke required before a memtomem pin bump now includes tests/test_frozen_memory_real.py (CLAUDE.md, CONTRIBUTING.md, README.md). Not here: the schema 1 stores of v0.1.2 to v0.1.4 (memtomem 0.3.x). Measured: v0.1.2 writes one mixed day file with untagged rows; v0.1.4 writes per-namespace day files whose shared copies carry only shared-from=, plus a scratch row v0.1.2 lacks; v0.1.3 is unmeasured. They follow in their own PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ock (#271) The doc row and the test docstring called the job by a name it does not have. The README command wrapped inside an inline code span, so copying it from the raw file split it into two shell lines; it is now a fenced block with continuations, run as written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #271. This is the first part of the Memory family: the memtomem stores that v0.2.0 and v0.3.0 (both memtomem 0.6.8) wrote. The issue stays open: the schema 1 stores of v0.1.2 and v0.1.4 (memtomem 0.3.x), the Mem0 and LangGraph stores with
memory migration, and the TUI jobs family remain.What
tests/fixtures/persisted/memory/<release>/storeis one memtomem store per release, written by that release's own CLI through its default backend (freeze.pybeside it, run against agit worktreeof the tag withuv sync --frozen --no-dev, fakecodexandkimion PATH, no embedding provider). Every scenario goes into the same store, one after another:rununder route, pipeline, compete, council (two critique rounds) and decompose: private rows and theirsharedcopies withtask:andshared-from=tags and every kind tag;###headings, a prompt long enough to split into two chunks;board run(itsboard-run:scratch row and theboard-result:row),room apply(room-result);board run -s pipelineSIGKILLed during its second stage, so the store is frozen with that row only in the WAL and a live lease on the board.freeze.pystops unless each scenario added exactly the rows its definition names and left every earlier row exactly as stored, and unless the killed row is in the WAL only.index.jsonholds the scenario definitions and a sha256 of every file;release-read.jsonis what the release's own interpreter read back from every row. Output is published whole (a hidden sibling, then a rename).tests/test_frozen_memory.py(default suite, no engine): the stored rows against the scenario definitions, the day-file names and namespaces HEAD derives, a shared copy naming its private row, the release read tied to the rows,chunks.source_file(and its FTS mirror) as the only column binding a store to where it was written, the killed row absent from the main database file alone,embedding_provider = none.tests/test_frozen_memory_real.py(SYNCMILL_TEST_MEMTOMEM_REAL=1, now in CI'smemory-backend-realjob): HEAD's first and second open of a moved copy, reading every row out of the WAL;MemtomemBackend.get/search/count_namespacecompared with the release's read,searchwith each later-phase kind filter,search_multi,status;share_chunkandadd_and_sharebeside released rows (a later day and the release's own day);purge_namespaceof a released scratch namespace; a reindex that changes no row;SyncmillMemory.search_context(filters, agent namespace, board-scoped view),search_shared/get_shared/shared_stats; the gatewayshared_*tools;syncmill status; each strategy's recall in a HEAD run (the filters sent, the released rows returned, each handed to every agent of that phase); a HEADrunand room round handed a released result; the store found at the project default location throughconfigure_project_memory; a released lease reclaimed and its scratch namespace purged by the board GC. Copies that write, purge or reindex are rebound first with the two statements memtomem itself uses, asserting every other cell unchanged.Docs:
persisted-contracts.mdgets a "Memory (memtomem)" row, with what is not covered; Mem0 and LangGraph stay "not yet" in their own row. The real smoke before a memtomem pin bump now includestests/test_frozen_memory_real.py(CLAUDE.md, CONTRIBUTING.md, README.md).No
src/change. No current reader misread a released store.Measured along the way
sqlite3connection checkpoints the WAL into the main file, so the killed store is opened first by the engine under test, and the main-file bytes are taken before any connection.shared-from=and a scratch row v0.1.2 lacks. It goes with v0.1.2 in the follow-up.Verification
suite-runCLEAN;ruff check .clean; CI's real-memtomem command run locally: 68 passed.search_multiandsearch_context, shared-from tags, the share link, the gateway's namespace checks, each strategy's kind string, the room round's recall, the project store binding, FTS rows lost beside a fresh share, reads that fail on a moved copy, recalled context dropped by a strategy or by a later pipeline stage. Fixture-side: a checkpointed WAL, a recorded embedding provider, and a release read missing a namespace or a kind each fail.freeze.pyregenerated v0.3.0 into a scratch directory after the review fixes: same 13 scenarios and files, and all default tests pass on it.memory migration --from memtomemreader test, declined as Memory-2 scope), post-commit SHIP with two doc nits, fixed.🤖 Generated with Claude Code