Rename cache_mb to node_cache_mb and count it in heap bytes

benchmarks/src/main/java/io/hstore/bench/HStoreStore.java

@@ -32,7 +32,7 @@ final class HStoreStore implements Store {
     private long writtenBeforeReopen;
     private long readBeforeReopen;
 
-    static final long DEFAULT_CACHE_BYTES = (1L << 20) * 16 * 1024;
+    static final long DEFAULT_CACHE_BYTES = Runtime.getRuntime().maxMemory() / 2;
 
     private final int history;
 
@@ -44,7 +44,7 @@ final class HStoreStore implements Store {
         this.cacheBytes = cacheBytes > 0 ? cacheBytes : DEFAULT_CACHE_BYTES;
         EngineOptions engine = EngineOptions.defaults();
         this.options = DatabaseOptions.defaults().withEngine(engine
-                .withDurability(sync ? Durability.SYNC : Durability.ASYNC).withCacheBytes(this.cacheBytes).withHistoryLimit(history)
+                .withDurability(sync ? Durability.SYNC : Durability.ASYNC).withNodeCacheBytes(this.cacheBytes).withHistoryLimit(history)
                 .withCompactionLiveRatio(liveRatio).withPagesPerSegment(pagesPerSegment > 0 ? pagesPerSegment : engine.pagesPerSegment()));
         this.database = HypergraphDatabase.open(directory, options);
         if (create) {

docs/architecture/overview.md

@@ -154,7 +154,7 @@ sequenceDiagram
 
 Notes:
 
-* Nodes are decoded once and then shared by every reader through `NodeCache`. It has 16 shards, each a `ConcurrentHashMap` plus a CLOCK (second-chance) ring, with a total byte budget of `cache_mb`, each node counted at the size of its page. Lookups are lock-free; only admission takes a per-shard lock. Decoded nodes are immutable (`owner == null`), so sharing them needs no copying. See [pages.md](../storage/pages.md#caching-and-io-accounting).
+* Nodes are decoded once and then shared by every reader through `NodeCache`. It has 16 shards, each a `ConcurrentHashMap` plus a CLOCK (second-chance) ring, with a total byte budget of `node_cache_mb`, each node counted at an estimate of its heap size (`Node.heapBytes`). Lookups are lock-free; only admission takes a per-shard lock. Decoded nodes are immutable (`owner == null`), so sharing them needs no copying. See [pages.md](../storage/pages.md#caching-and-io-accounting).
 * `IoTrace` is bound with `ScopedValue.where(...)` around a query. `recordRead` throws `HStoreException.limit` once page reads plus cache hits exceed the query's page budget (`query_page_budget`).
 * Historical reads use `TransactionManager.snapshotAt(generation, branch)`. It looks the generation up in the retained `history` map; see [catalog-and-generations.md](../storage/catalog-and-generations.md#history-retention-and-time-travel).
 

docs/benchmarks.md

@@ -21,7 +21,7 @@ flowchart LR
 |---|---|---|
 | Version | 0.1.0 (this repository) | 1.4 at commit `99485a1`, storage `bdb-je` on Berkeley DB JE 5.0.73 |
 | Data model used | node type `Entity` (indexed by key, property `name`); `SET_EDGE` type `Relation` | `String` atoms; `HGPlainLink` links |
-| Cache | node cache of 16 GiB (enough for the whole dataset), or `CACHE_MB` | JE cache at 30% of the heap (HyperGraphDB default), or `CACHE_MB`, plus the HyperGraphDB atom cache |
+| Cache | node cache of half the heap, or `CACHE_MB` | JE cache at 30% of the heap (HyperGraphDB default), or `CACHE_MB`, plus the HyperGraphDB atom cache |
 | `async` durability | `durability = async` | HyperGraphDB default: JE `WRITE_NO_SYNC` |
 | `sync` durability | `durability = sync`, `wal_mode = references`, group commit | JE `COMMIT_SYNC` |
 | History | `history_limit = 64` generations retained for time travel (default) | none |
@@ -251,7 +251,7 @@ not what drives HStore's footprint:
   defers the work to its cleaner. In `sync` mode group commit amortises the fsync and HStore leads (1.35×).
 * **Parallel incidence reads at scale 4 (0.46×).** Every probe batch opens a snapshot, which pins a
   generation in a shared skip-list map. At 10 threads this coordination, plus node-cache admission under
-  churn (the hot set exceeds the node cache), costs more than JE's latch-free cache hits. At scale 1
+  churn (the hot set exceeded the 1,048,576-node cache these runs used), costs more than JE's latch-free cache hits. At scale 1
   the results are mixed: 0.66× async and 1.34× sync.
 * **Disk at scale 4 (0.54×).** Live data is comparable: about 1 GB of live node images in HStore against
   1.27 GB in JE. The difference is garbage that has not been reclaimed yet. Both engines are log-structured
@@ -279,16 +279,16 @@ run in brackets, and "Results agree" requires every run of both stores to return
 and `HEAP` override the thread count and heap size. Close other
 workloads first: the numbers are only meaningful on an otherwise idle machine.
 
-**Equal cache budgets.** By default the two caches are not the same size. HStore's node cache gets 16 GiB, which is
-enough to hold the whole dataset. HyperGraphDB gets JE's default of 30% of the
+**Equal cache budgets.** By default the two caches are not the same size. HStore's node cache gets half the heap
+(2 GiB with the default `HEAP=4g`). HyperGraphDB gets JE's default of 30% of the
 heap. `CACHE_MB` gives both the same budget:
 
 | HStore | HyperGraphDB |
 |---|---|
-| node cache of `CACHE_MB`, each node counted at the size of its page | JE cache of exactly `CACHE_MB` |
+| node cache of `CACHE_MB`, each node counted at an estimate of its heap size | JE cache of exactly `CACHE_MB` |
 
-Decoded nodes take more heap than their page, and HyperGraphDB's own atom and incidence caches can't
-be capped in bytes (they shrink under memory pressure). So check the retained-heap line in the report to see what
+HStore's estimate runs 10 to 40% below the heap a node really takes, and HyperGraphDB's own atom and incidence
+caches can't be capped in bytes (they shrink under memory pressure). So check the retained-heap line in the report to see what
 each store actually kept. Results go to a directory ending in `-cache-<mb>`.
 
 ```

docs/operations/configuration.md

@@ -91,7 +91,7 @@ max_connections          = 200
 | `page_size` | `HSTORE_PAGE_SIZE` | `16384` | Page size in bytes. Must be a power of two from 1 KiB to 1 MiB. It is recorded in `<data>/FORMAT` at creation, and opening with a different value fails with `database uses N byte pages, options request M`. |
 | `durability` | `HSTORE_DURABILITY` | `sync` | `sync`: a commit is acknowledged after the group-commit barrier has `fsync`ed it. `async`: acknowledged once appended, so a crash can lose a suffix of acknowledged commits but never corrupts the database. |
 | `wal_mode` | `HSTORE_WAL_MODE` | `references` | `references`: the WAL logs `PageRef(pageId, length, crc32c)` records and data pages are synced before the WAL. `images`: the WAL logs full page images. See [WAL and recovery](../transactions/wal-and-recovery.md). |
-| `cache_mb` | `HSTORE_CACHE_MB` | `256` | Megabytes of decoded tree nodes kept in memory, each counted at the size of its page. |
+| `node_cache_mb` | `HSTORE_NODE_CACHE_MB` | `256` | Megabytes of Java heap for decoded tree nodes, each counted at an estimate of the memory it takes. |
 | `checkpoint_wal_mb` | `HSTORE_CHECKPOINT_WAL_MB` | `256` | WAL volume since the last checkpoint that triggers a background checkpoint. |
 | `history_limit` | `HSTORE_HISTORY_LIMIT` | `64` | Committed generations kept addressable for `AT GENERATION`, `AS OF`, `HISTORY` and the Studio time slider. |
 | `compaction_live_ratio` | `HSTORE_COMPACTION_LIVE_RATIO` | `0.5` | Sealed segments whose live bytes fall below this fraction of their allocated bytes are relocated by compaction: one segment (the emptiest) per background pass after every `checkpoint_wal_mb` of written node images, or all of them with `COMPACT`. Time-travel history is preserved; retired segments are deleted once `history_limit` has rolled past them. Must satisfy 0 < r < 1; other values fail at startup. |
@@ -139,13 +139,22 @@ higher write amplification per commit, because each commit rewrites one root-to-
 16 KiB suits mixed workloads. Use 4–8 KiB for update-heavy workloads with small hyperedges, and 32–64 KiB
 for analytics over giant hyperedges. The value cannot be changed after `init`.
 
-**`cache_mb`.** The node cache holds *decoded* nodes, not raw pages. Each node counts as the size of its page on
-disk (at most `page_size`), so the setting is a byte budget. The heap it really uses is somewhat higher, because a
-decoded node takes more memory than its page. Interior nodes are touched by every lookup and are what you want
-resident. A rule of thumb: 2% of the data size for interior coverage, plus the working set of hot leaves. The
-Dashboard *Cache hit rate* tile and `STATS;` show whether the cache suffices. The old `cache_nodes` setting, a
-node count, is no longer accepted: the server refuses to start with it, whether it comes from `hstore.conf`, the
-environment or a flag, and says to use `cache_mb`.
+**`node_cache_mb`.** This is the only cache HStore sizes itself, and it holds *decoded* nodes on the Java heap.
+Raw pages are cached by the operating system like any other file data, in memory the OS can take back when it
+needs to. Each node is counted at an estimate of its size on the heap, worked out from its entry count and the
+encoded size of its values, so the setting is close to the heap the cache really takes. Measured against the
+heap it was 10 to 40% low, never above. A decoded node is three to six times the size of its page, so a cache
+counted in page bytes would have used several times its setting. The default of 256 is about the heap the earlier
+default of 64 really used when nodes were counted at their page size.
+
+The server binary uses the serial collector, and every full collection walks everything this cache holds, so a
+bigger cache means longer pauses. Size it for what every lookup touches: the interior nodes, about 2% of the data,
+plus the leaves you read all the time. The Dashboard *Cache hit rate* tile and `STATS;` show whether it is big
+enough.
+
+The value must be between 1 and 16,777,216 (16 TiB). Two older names are refused at startup, whether they come from `hstore.conf`, the environment or a flag:
+`cache_mb`, the name this setting had before it was released, and `cache_nodes`, a node count. Both messages name
+`node_cache_mb`.
 
 **`checkpoint_wal_mb`.** A checkpoint writes the current generation's catalog image and truncates WAL segments
 before the checkpoint LSN. Lower values shorten recovery, since recovery replays WAL after the last checkpoint,

docs/operations/docker.md

@@ -76,7 +76,7 @@ Without a password and without `HSTORE_AUTHENTICATION=off`, the container refuse
 ### Server settings
 
 Every [setting](configuration.md) is available as `HSTORE_<SETTING>`, for example `HSTORE_DURABILITY=async`,
-`HSTORE_LOG_STATEMENT=ddl`, `HSTORE_LOG_MIN_DURATION_MS=250`, `HSTORE_CACHE_MB=1024`,
+`HSTORE_LOG_STATEMENT=ddl`, `HSTORE_LOG_MIN_DURATION_MS=250`, `HSTORE_NODE_CACHE_MB=512`,
 `HSTORE_STUDIO=off`, `HSTORE_EMBEDDING_PROVIDER=ollama`. Environment variables override
 `/var/lib/hstore/data/hstore.conf`. Flags appended to the `docker run` command line override both:
 

docs/operations/studio.md

@@ -141,7 +141,7 @@ Errors use one JSON shape:
 | `GET /api/graph` | yes | Hypergraph neighbourhood for visualisation; see below. |
 | `GET /api/schema` | yes | Types visible to the session: `id`, `name`, `kind`, `version`, `count`, `roles`, `properties[{name,type,indexed,required}]`, `jsonIndexes[{path,type}]`, plus `branches`. |
 | `GET /api/history` | yes | `{"current": G, "generations": [{"id","wallTime","txnId"}]}` for the retained generations. |
-| `GET /api/stats` | ADMIN | Engine counters, segments (`id`, `state`, `pages` = node images, `bytes`, `live`), recovery outcome, configuration (`pageSize`, `segmentBytes`, `durability`, `walMode`, `cacheBytes`, `historyLimit`, `checkpointWalBytes`), semantic index generation, Studio session count, branches. |
+| `GET /api/stats` | ADMIN | Engine counters, segments (`id`, `state`, `pages` = node images, `bytes`, `live`), recovery outcome, configuration (`pageSize`, `segmentBytes`, `durability`, `walMode`, `nodeCacheBytes`, `historyLimit`, `checkpointWalBytes`), semantic index generation, Studio session count, branches. |
 
 Example `GET /api/info`:
 

docs/storage/pages.md

@@ -215,7 +215,7 @@ Most nodes fit in 4 KiB, so most reads are a single 4 KiB `pread`. A short read
 The cache stores **decoded, frozen** `Node` objects, not bytes. A hit therefore costs no decoding and no allocation. The cache has 16 shards; a node goes to shard `mix(pageId) >>> 60`. Each shard has:
 
 * a `ConcurrentHashMap<Long, Slot>` from page id to `Slot(pageId, node, volatile referenced)`;
-* a byte budget of `max(64 KiB, cache_mb / 16)` and a circular, doubly linked ring of slots with a CLOCK hand. Each slot weighs its page: header plus payload when it was read, the image size when the commit wrote it;
+* a byte budget of `max(64 KiB, node_cache_mb / 16)` and a circular, doubly linked ring of slots with a CLOCK hand. Each slot weighs `Node.heapBytes()`, an estimate of the decoded node's heap size: 144 bytes for the node and its summary, its key and value or child arrays, then 96 bytes plus the encoded value per leaf entry, or 128 bytes per child reference in a branch. Those constants come from measuring retained heap against cache contents at two dataset sizes;
 * a `ReentrantLock` taken only for admission.
 
 **Reads are lock-free.** `get` is a `ConcurrentHashMap.get`, plus setting `referenced = true` if it was clear. The check avoids a redundant volatile write on hot entries.

engine/src/main/java/io/hstore/engine/EngineOptions.java

@@ -12,7 +12,7 @@ public record EngineOptions(
         int pageSize,
         int pagesPerSegment,
         long walSegmentBytes,
-        long cacheBytes,
+        long nodeCacheBytes,
         Durability durability,
         WalMode walMode,
         int historyLimit,
@@ -30,8 +30,8 @@ public record EngineOptions(
         if (compactionLiveRatio <= 0 || compactionLiveRatio >= 1) {
             throw new IllegalArgumentException("compaction live ratio must be between 0 and 1: " + compactionLiveRatio);
         }
-        if (cacheBytes < 1) {
-            throw new IllegalArgumentException("node cache must be at least one byte: " + cacheBytes);
+        if (nodeCacheBytes < 1) {
+            throw new IllegalArgumentException("node cache must be at least one byte: " + nodeCacheBytes);
         }
         if (feedRetention < 1) {
             throw new IllegalArgumentException("feed retention must keep at least one generation: " + feedRetention);
@@ -46,62 +46,62 @@ public static EngineOptions defaults() {
     }
 
     public EngineOptions withPageSize(int bytes) {
-        return new EngineOptions(bytes, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
+        return new EngineOptions(bytes, pagesPerSegment, walSegmentBytes, nodeCacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withWalSegmentBytes(long bytes) {
-        return new EngineOptions(pageSize, pagesPerSegment, bytes, cacheBytes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, bytes, nodeCacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withPagesPerSegment(int pages) {
-        return new EngineOptions(pageSize, pages, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pages, walSegmentBytes, nodeCacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withDurability(Durability mode) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, mode, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodeCacheBytes, mode, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withWalMode(WalMode mode) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, mode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodeCacheBytes, durability, mode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withHistoryLimit(int generations) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, generations,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodeCacheBytes, durability, walMode, generations,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withCheckpointWalBytes(long bytes) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodeCacheBytes, durability, walMode, historyLimit,
                 bytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
-    public EngineOptions withCacheBytes(long bytes) {
+    public EngineOptions withNodeCacheBytes(long bytes) {
         return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, bytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withCompactionLiveRatio(double ratio) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodeCacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, ratio, feedRetention, slots, derivations, faults);
     }
 
     public EngineOptions withFeedRetention(long generations) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodeCacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, generations, slots, derivations, faults);
     }
 
     public EngineOptions withExtensions(List<Slot<?>> extensionSlots, List<Derivation> extensionDerivations) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodeCacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, extensionSlots, extensionDerivations, faults);
     }
 
     public EngineOptions withFaults(CrashPoint.Injector injector) {
-        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, cacheBytes, durability, walMode, historyLimit,
+        return new EngineOptions(pageSize, pagesPerSegment, walSegmentBytes, nodeCacheBytes, durability, walMode, historyLimit,
                 checkpointWalBytes, compactionLiveRatio, feedRetention, slots, derivations, injector);
     }
 }

engine/src/main/java/io/hstore/engine/StorageEngine.java

@@ -76,7 +76,7 @@ private StorageEngine(Path directory, EngineOptions options, FileChannel lockCha
         FeedCodec codec = new FeedCodec(slots);
         this.feed = ChangeFeed.open(directory.resolve("feed"), codec, options.walSegmentBytes());
         this.recovery = Recovery.recover(checkpoint, wal, pages, feed, codec);
-        this.cache = new NodeCache(options.cacheBytes());
+        this.cache = new NodeCache(options.nodeCacheBytes(), options.pageSize());
         PagedNodeSource source = new PagedNodeSource(pages, cache);
         TransactionManager.Storage storage = new TransactionManager.Storage(pages, source, wal, feed, codec, slots,
                 options.derivations(), options.durability(), options.walMode(), options.faults(), options.historyLimit());