Size the node cache in megabytes

benchmarks/src/main/java/io/hstore/bench/HStoreStore.java

@@ -32,21 +32,19 @@ final class HStoreStore implements Store {
     private long writtenBeforeReopen;
     private long readBeforeReopen;
 
-    static final int CACHED_NODES = 1 << 20;
+    static final long DEFAULT_CACHE_BYTES = (1L << 20) * 16 * 1024;
 
     private final int history;
 
-    private final int cachedNodes;
     private final long cacheBytes;
 
     HStoreStore(Path directory, boolean sync, int history, long cacheBytes, double liveRatio, int pagesPerSegment, boolean create) {
         this.directory = directory;
         this.history = history;
-        this.cacheBytes = cacheBytes;
+        this.cacheBytes = cacheBytes > 0 ? cacheBytes : DEFAULT_CACHE_BYTES;
         EngineOptions engine = EngineOptions.defaults();
-        this.cachedNodes = cacheBytes > 0 ? Math.toIntExact(cacheBytes / engine.pageSize()) : CACHED_NODES;
         this.options = DatabaseOptions.defaults().withEngine(engine
-                .withDurability(sync ? Durability.SYNC : Durability.ASYNC).withCachedNodes(cachedNodes).withHistoryLimit(history)
+                .withDurability(sync ? Durability.SYNC : Durability.ASYNC).withCacheBytes(this.cacheBytes).withHistoryLimit(history)
                 .withCompactionLiveRatio(liveRatio).withPagesPerSegment(pagesPerSegment > 0 ? pagesPerSegment : engine.pagesPerSegment()));
         this.database = HypergraphDatabase.open(directory, options);
         if (create) {
@@ -70,8 +68,7 @@ public String version() {
 
     @Override
     public String cache() {
-        String budget = cacheBytes > 0 ? " (%,d MiB divided by the page size)".formatted(cacheBytes >> 20) : "";
-        return "node cache of %,d decoded nodes%s and retains %d generations of history".formatted(cachedNodes, budget, history);
+        return "node cache of %,d MiB and retains %d generations of history".formatted(cacheBytes >> 20, history);
     }
 
     @Override

docs/architecture/overview.md

@@ -154,7 +154,7 @@ sequenceDiagram
 
 Notes:
 
-* Nodes are decoded once and then shared by every reader through `NodeCache`. It has 16 shards, each a `ConcurrentHashMap` plus a CLOCK (second-chance) ring, with total capacity `cache_nodes`. Lookups are lock-free; only admission takes a per-shard lock. Decoded nodes are immutable (`owner == null`), so sharing them needs no copying. See [pages.md](../storage/pages.md#caching-and-io-accounting).
+* Nodes are decoded once and then shared by every reader through `NodeCache`. It has 16 shards, each a `ConcurrentHashMap` plus a CLOCK (second-chance) ring, with a total byte budget of `cache_mb`, each node counted at the size of its page. Lookups are lock-free; only admission takes a per-shard lock. Decoded nodes are immutable (`owner == null`), so sharing them needs no copying. See [pages.md](../storage/pages.md#caching-and-io-accounting).
 * `IoTrace` is bound with `ScopedValue.where(...)` around a query. `recordRead` throws `HStoreException.limit` once page reads plus cache hits exceed the query's page budget (`query_page_budget`).
 * Historical reads use `TransactionManager.snapshotAt(generation, branch)`. It looks the generation up in the retained `history` map; see [catalog-and-generations.md](../storage/catalog-and-generations.md#history-retention-and-time-travel).
 

docs/benchmarks.md

@@ -21,7 +21,7 @@ flowchart LR
 |---|---|---|
 | Version | 0.1.0 (this repository) | 1.4 at commit `99485a1`, storage `bdb-je` on Berkeley DB JE 5.0.73 |
 | Data model used | node type `Entity` (indexed by key, property `name`); `SET_EDGE` type `Relation` | `String` atoms; `HGPlainLink` links |
-| Cache | node cache of 1,048,576 decoded nodes, or `CACHE_MB` / page size | JE cache at 30% of the heap (HyperGraphDB default), or `CACHE_MB`, plus the HyperGraphDB atom cache |
+| Cache | node cache of 16 GiB (enough for the whole dataset), or `CACHE_MB` | JE cache at 30% of the heap (HyperGraphDB default), or `CACHE_MB`, plus the HyperGraphDB atom cache |
 | `async` durability | `durability = async` | HyperGraphDB default: JE `WRITE_NO_SYNC` |
 | `sync` durability | `durability = sync`, `wal_mode = references`, group commit | JE `COMMIT_SYNC` |
 | History | `history_limit = 64` generations retained for time travel (default) | none |
@@ -251,7 +251,7 @@ not what drives HStore's footprint:
   defers the work to its cleaner. In `sync` mode group commit amortises the fsync and HStore leads (1.35×).
 * **Parallel incidence reads at scale 4 (0.46×).** Every probe batch opens a snapshot, which pins a
   generation in a shared skip-list map. At 10 threads this coordination, plus node-cache admission under
-  churn (the hot set exceeds one million decoded nodes), costs more than JE's latch-free cache hits. At scale 1
+  churn (the hot set exceeds the node cache), costs more than JE's latch-free cache hits. At scale 1
   the results are mixed: 0.66× async and 1.34× sync.
 * **Disk at scale 4 (0.54×).** Live data is comparable: about 1 GB of live node images in HStore against
   1.27 GB in JE. The difference is garbage that has not been reclaimed yet. Both engines are log-structured
@@ -279,15 +279,15 @@ run in brackets, and "Results agree" requires every run of both stores to return
 and `HEAP` override the thread count and heap size. Close other
 workloads first: the numbers are only meaningful on an otherwise idle machine.
 
-**Equal cache budgets.** By default the two caches are not the same size. HStore caches about a million decoded
-nodes, which at 16 KiB pages is enough to hold the whole dataset. HyperGraphDB gets JE's default of 30% of the
+**Equal cache budgets.** By default the two caches are not the same size. HStore's node cache gets 16 GiB, which is
+enough to hold the whole dataset. HyperGraphDB gets JE's default of 30% of the
 heap. `CACHE_MB` gives both the same budget:
 
 | HStore | HyperGraphDB |
 |---|---|
-| node cache of `CACHE_MB` / page size nodes | JE cache of exactly `CACHE_MB` |
+| node cache of `CACHE_MB`, each node counted at the size of its page | JE cache of exactly `CACHE_MB` |
 
-Decoded nodes can take more or less heap than their page, and HyperGraphDB's own atom and incidence caches can't
+Decoded nodes take more heap than their page, and HyperGraphDB's own atom and incidence caches can't
 be capped in bytes (they shrink under memory pressure). So check the retained-heap line in the report to see what
 each store actually kept. Results go to a directory ending in `-cache-<mb>`.
 

docs/operations/configuration.md

@@ -91,7 +91,7 @@ max_connections          = 200
 | `page_size` | `HSTORE_PAGE_SIZE` | `16384` | Page size in bytes. Must be a power of two from 1 KiB to 1 MiB. It is recorded in `<data>/FORMAT` at creation, and opening with a different value fails with `database uses N byte pages, options request M`. |
 | `durability` | `HSTORE_DURABILITY` | `sync` | `sync`: a commit is acknowledged after the group-commit barrier has `fsync`ed it. `async`: acknowledged once appended, so a crash can lose a suffix of acknowledged commits but never corrupts the database. |
 | `wal_mode` | `HSTORE_WAL_MODE` | `references` | `references`: the WAL logs `PageRef(pageId, length, crc32c)` records and data pages are synced before the WAL. `images`: the WAL logs full page images. See [WAL and recovery](../transactions/wal-and-recovery.md). |
-| `cache_nodes` | `HSTORE_CACHE_NODES` | `65536` | Decoded tree nodes kept in the node cache. |
+| `cache_mb` | `HSTORE_CACHE_MB` | `256` | Megabytes of decoded tree nodes kept in memory, each counted at the size of its page. |
 | `checkpoint_wal_mb` | `HSTORE_CHECKPOINT_WAL_MB` | `256` | WAL volume since the last checkpoint that triggers a background checkpoint. |
 | `history_limit` | `HSTORE_HISTORY_LIMIT` | `64` | Committed generations kept addressable for `AT GENERATION`, `AS OF`, `HISTORY` and the Studio time slider. |
 | `compaction_live_ratio` | `HSTORE_COMPACTION_LIVE_RATIO` | `0.5` | Sealed segments whose live bytes fall below this fraction of their allocated bytes are relocated by compaction: one segment (the emptiest) per background pass after every `checkpoint_wal_mb` of written node images, or all of them with `COMPACT`. Time-travel history is preserved; retired segments are deleted once `history_limit` has rolled past them. Must satisfy 0 < r < 1; other values fail at startup. |
@@ -139,9 +139,13 @@ higher write amplification per commit, because each commit rewrites one root-to-
 16 KiB suits mixed workloads. Use 4–8 KiB for update-heavy workloads with small hyperedges, and 32–64 KiB
 for analytics over giant hyperedges. The value cannot be changed after `init`.
 
-**`cache_nodes`.** The node cache holds *decoded* nodes, not raw pages. Interior nodes are touched by every
-lookup and are what you want resident. A rule of thumb: total pages × 0.02 for interior coverage, plus the
-working set of hot leaves. The Dashboard *Cache hit rate* tile and `STATS;` show whether the cache suffices.
+**`cache_mb`.** The node cache holds *decoded* nodes, not raw pages. Each node counts as the size of its page on
+disk (at most `page_size`), so the setting is a byte budget. The heap it really uses is somewhat higher, because a
+decoded node takes more memory than its page. Interior nodes are touched by every lookup and are what you want
+resident. A rule of thumb: 2% of the data size for interior coverage, plus the working set of hot leaves. The
+Dashboard *Cache hit rate* tile and `STATS;` show whether the cache suffices. The old `cache_nodes` setting, a
+node count, is no longer accepted: the server refuses to start with it, whether it comes from `hstore.conf`, the
+environment or a flag, and says to use `cache_mb`.
 
 **`checkpoint_wal_mb`.** A checkpoint writes the current generation's catalog image and truncates WAL segments
 before the checkpoint LSN. Lower values shorten recovery, since recovery replays WAL after the last checkpoint,

docs/operations/docker.md

@@ -76,7 +76,7 @@ Without a password and without `HSTORE_AUTHENTICATION=off`, the container refuse
 ### Server settings
 
 Every [setting](configuration.md) is available as `HSTORE_<SETTING>`, for example `HSTORE_DURABILITY=async`,
-`HSTORE_LOG_STATEMENT=ddl`, `HSTORE_LOG_MIN_DURATION_MS=250`, `HSTORE_CACHE_NODES=262144`,
+`HSTORE_LOG_STATEMENT=ddl`, `HSTORE_LOG_MIN_DURATION_MS=250`, `HSTORE_CACHE_MB=1024`,
 `HSTORE_STUDIO=off`, `HSTORE_EMBEDDING_PROVIDER=ollama`. Environment variables override
 `/var/lib/hstore/data/hstore.conf`. Flags appended to the `docker run` command line override both:
 

docs/operations/studio.md

@@ -141,7 +141,7 @@ Errors use one JSON shape:
 | `GET /api/graph` | yes | Hypergraph neighbourhood for visualisation; see below. |
 | `GET /api/schema` | yes | Types visible to the session: `id`, `name`, `kind`, `version`, `count`, `roles`, `properties[{name,type,indexed,required}]`, `jsonIndexes[{path,type}]`, plus `branches`. |
 | `GET /api/history` | yes | `{"current": G, "generations": [{"id","wallTime","txnId"}]}` for the retained generations. |
-| `GET /api/stats` | ADMIN | Engine counters, segments (`id`, `state`, `pages` = node images, `bytes`, `live`), recovery outcome, configuration (`pageSize`, `segmentBytes`, `durability`, `walMode`, `cachedNodes`, `historyLimit`, `checkpointWalBytes`), semantic index generation, Studio session count, branches. |
+| `GET /api/stats` | ADMIN | Engine counters, segments (`id`, `state`, `pages` = node images, `bytes`, `live`), recovery outcome, configuration (`pageSize`, `segmentBytes`, `durability`, `walMode`, `cacheBytes`, `historyLimit`, `checkpointWalBytes`), semantic index generation, Studio session count, branches. |
 
 Example `GET /api/info`:
 

docs/storage/pages.md

@@ -215,16 +215,18 @@ Most nodes fit in 4 KiB, so most reads are a single 4 KiB `pread`. A short read
 The cache stores **decoded, frozen** `Node` objects, not bytes. A hit therefore costs no decoding and no allocation. The cache has 16 shards; a node goes to shard `mix(pageId) >>> 60`. Each shard has:
 
 * a `ConcurrentHashMap<Long, Slot>` from page id to `Slot(pageId, node, volatile referenced)`;
-* a `Slot[] ring` of capacity `max(16, cache_nodes / 16)` with a CLOCK hand;
+* a byte budget of `max(64 KiB, cache_mb / 16)` and a circular, doubly linked ring of slots with a CLOCK hand. Each slot weighs its page: header plus payload when it was read, the image size when the commit wrote it;
 * a `ReentrantLock` taken only for admission.
 
 **Reads are lock-free.** `get` is a `ConcurrentHashMap.get`, plus setting `referenced = true` if it was clear. The check avoids a redundant volatile write on hot entries.
 
 **Admission (`put`) uses CLOCK second-chance eviction under the shard lock:**
 
 * An entry already holding the same node is left alone.
-* An entry for the same page with a different node is replaced in its ring slot.
-* Otherwise the hand advances, clearing `referenced` bits, until it finds an empty slot or an unreferenced one. That victim is removed from the map, conditionally on the identity of its `Slot`, the new slot takes its place, and the hand moves on.
+* An entry for the same page with a different node is unlinked first, and its bytes are released.
+* A node larger than the shard's whole budget isn't cached.
+* While the new node doesn't fit in the budget, the hand advances, clearing `referenced` bits, until it reaches an unreferenced slot. That victim is unlinked and removed from the map, conditionally on the identity of its `Slot`.
+* The new slot is linked just behind the hand, so it gets a full sweep before it can be evicted.
 
 This approximates LRU. Readers never contend with each other, and they contend with writers only on the map's internal bins.
 

engine/src/main/java/io/hstore/engine/EngineOptions.java

@@ -12,7 +12,7 @@ public record EngineOptions(
         int pageSize,
         int pagesPerSegment,
         long walSegmentBytes,
-        int cachedNodes,
+        long cacheBytes,
         Durability durability,
         WalMode walMode,
         int historyLimit,
@@ -30,6 +30,9 @@ public record EngineOptions(
         if (compactionLiveRatio <= 0 || compactionLiveRatio >= 1) {
             throw new IllegalArgumentException("compaction live ratio must be between 0 and 1: " + compactionLiveRatio);
         }
+        if (cacheBytes < 1) {
+            throw new IllegalArgumentException("node cache must be at least one byte: " + cacheBytes);
+        }
         if (feedRetention < 1) {
             throw new IllegalArgumentException("feed retention must keep at least one generation: " + feedRetention);
         }
@@ -38,67 +41,67 @@ public record EngineOptions(
     }
 
     public static EngineOptions defaults() {
-        return new EngineOptions(16 * 1024, 16 * 1024, 64L << 20, 1 << 16, Durability.SYNC, WalMode.PAGE_REFERENCES, 64,
+        return new EngineOptions(16 * 1024, 16 * 1024, 64L << 20, 256L << 20, Durability.SYNC, WalMode.PAGE_REFERENCES, 64,
                 256L << 20, 0.5, 100_000, List.of(), List.of(), CrashPoint.Injector.NONE);
     }
 
     public EngineOptions withPageSize(int bytes) {
-        return new EngineOptions(bytes, pagesPerSegment, walSegmentBytes, cachedNodes, durability, walMode, historyLimit,
+        return new EngineOptions(bytes, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withWalSegmentBytes(long bytes) {
-        return new EngineOptions(pageSize, pagesPerSegment, bytes, cachedNodes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, bytes, cacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withPagesPerSegment(int pages) {
-        return new EngineOptions(pageSize, pages, walSegmentBytes, cachedNodes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pages, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withDurability(Durability mode) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cachedNodes, mode, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, mode, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withWalMode(WalMode mode) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cachedNodes, durability, mode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, mode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withHistoryLimit(int generations) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cachedNodes, durability, walMode, generations,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, generations,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withCheckpointWalBytes(long bytes) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cachedNodes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
                 bytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
-    public EngineOptions withCachedNodes(int nodes) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodes, durability, walMode, historyLimit,
+    public EngineOptions withCacheBytes(long bytes) {
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, bytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withCompactionLiveRatio(double ratio) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cachedNodes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, ratio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withFeedRetention(long generations) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cachedNodes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, generations, slots, derivations, faults);
     }
 
     public EngineOptions withExtensions(List<Slot<?>> extensionSlots, List<Derivation> extensionDerivations) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cachedNodes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, extensionSlots, extensionDerivations, faults);
     }
 
     public EngineOptions withFaults(CrashPoint.Injector injector) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cachedNodes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, injector);
     }
 }