Repository navigation
Give pages permanent numbers and keep their addresses in a directory - #79
Merged
Merged
Conversation
The checkpoint took its LSN after pages.sync(). A spill can run outside the commit lock, so it could write a page after the sync and append its record before the LSN was read. The checkpoint then dropped a record whose page was never forced. Reading the LSN first means every record below it was appended after its page was written, and the sync that follows covers that page.
…n a directory A page id used to be the page's address: segment, offset and the low bits of its generation. Moving a page meant giving it a new id, and so rewriting every node on the path to it. That is most of what compaction spends its time and memory on. A page id is now a number handed out in order, and pages.dir maps each number to its segment, offset and length in 64-byte units. The directory is memory-mapped in 8 MiB chunks, so lookups cost no heap. Only PageStore deals in addresses. Page and PageRef records carry the address as well. The directory is forced only at checkpoints, and recovery writes every logged address back before it checks or replays a page, so entries a crash lost come back from the WAL. Compaction is unchanged for now; it just gets new numbers for the pages it rewrites. The storage format goes to 4. Directories written by older builds fail to open with the usual export-and-reload message.
PageDirectoryTest checks that entries and the next number survive a reopen across more than one mapped chunk, that a reserved number is never handed out, and that a file in another format is refused. PageStoreTest checks that numbers stay in order across segments, that a number with no entry reads as corrupt, and that a page can't be written past what was allocated for it. The new crash test puts pages.dir back to its last checkpointed state after a halt, which is what losing unsynced directory writes looks like, and expects every logged commit to come back and later allocations not to reuse the restored numbers. It fails if recovery stops placing addresses.
pages.md explains numbers, addresses and pages.dir, with the hex example regenerated for a numbered page and a way to follow an entry by hand. wal-and-recovery.md gets the larger Page and PageRef records, the address step in recovery and the new checkpoint order. The other pages pick up the format change and the extra file.
…blank one Mapping the new file extended it to 8 MiB, but the header was only forced at the first checkpoint. A power loss before that left a file of zeros that every later open refused as an unknown format. The header is now forced as soon as it is written, and a header that is still all zeros is treated as a new file.
# Conflicts: # docs/architecture/overview.md # docs/operations/configuration.md # engine/src/main/java/io/hstore/engine/page/PageStore.java
This was referenced Oct 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #77, since both change
PageStoreand the directory uses the same mapping approach.Why
A page id was the page's address. Moving a page gave it a new id, so every node on the path to it had to be
rewritten too. That is why a full compaction at scale 4 allocated 6 to 9 GB and ran out of a 4 GB heap: it
rebuilds large parts of every tree just to move pages out of a segment.
This PR separates the two. A page id is now a number handed out in order, and a page directory maps each number
to where the page sits. Compaction still works the old way here; the next PR changes it to move pages by
copying their bytes and updating one directory entry.
What changes
PageIdis a 48-bit index plus a 16-bit reuse count (always 0 for now).PageAddressis segment, offset andlength in 64-byte units.
PageDirectoryisdata/segments/pages.dir: a 64-byte header, then one 8-byte address per number,memory-mapped in 8 MiB chunks. Lookups are a load from the page cache and use no heap.
PageandPageRefWAL records carry the address. The directory is forced only at checkpoints, and recoverywrites every logged address back before it checks or replays a page, so an entry lost in a crash comes back
from the log.
Checkpointernow reads its LSN before syncing pages. Before, a spill running outside the commit lock couldwrite a page after the sync and log it before the LSN was read, and the checkpoint would drop a record whose
page was never forced. That's the first commit, on its own.
Testing
PageDirectoryTest: entries and the next number survive a reopen across more than one mapped chunk,reserved numbers are never handed out, and a file in another format is refused.
pages.dirback to its last checkpointed state, whichis what losing unsynced directory writes looks like. Every logged commit has to come back, and later
allocations must not reuse the restored numbers. Taking the address step out of recovery makes it fail.
examples/clinical-claims.hql, andhstore checkverified all105 pages. Then a SIGKILL partway through a 200,000-insert load: on reopen it replayed 813 commits from the log
and
checkpassed.node cache and with
--cache-mb 64(about 4,000 nodes, so most reads miss the cache and go through thedirectory). Each cell is this branch's throughput divided by Read pages through a memory map of their segment #77's, as medians of 3 runs at scale 1 and 2 at
scale 4. Runs of the same setup varied by 1 to 15% on most rows, so read anything within about 10% of 1.00 as no
change:
read.incidenceread.membersread.comembershipread.twohoplatency.readread.incidence.coldmixed.readwrite.updatelatency.commitchurn.deleterecoverThe reads that miss the cache most,
read.membersandread.twohopat 64 MiB, are where the directory lookuphappens on almost every page, and they didn't move. A few cells are lower:
read.incidenceat scale 4 with64 MiB (0.86),
latency.readat scale 1 with 64 MiB (0.86, 22% spread),read.membersat scale 4 with thedefault cache (0.89, 9 and 12% spread), and
read.incidence.churnedat scale 4 with the default cache (0.83,not in the table, 38% spread). In the first one, #77 itself ran faster with the small cache than with the full
one, so I read these as run-to-run variation, but they're worth watching when phase 2 is timed. Disk after loading is about 10 MB larger at scale 1, which is the directory
file.
What's next
Phase 2 moves pages during compaction by copying bytes and updating the directory, and removes
relocateandadopt.