Skip to content

Size the node cache in megabytes - #69

Merged
venkat1701 merged 5 commits into
mainfrom
feat/cache-mb
Oct 11, 2026
Merged

venkat1701 merged 5 commits into
mainfrom
feat/cache-mb

Conversation

@venkat1701

Copy link
Copy Markdown
Collaborator

Closes #59

Stacked on #66 (which is stacked on #65), because the benchmark adapter changes in all three. Merge those first; then this PR's base needs switching to main.

The cache limit was a count of decoded nodes, so it said little about memory: a node can be a few bytes or a full 16 KiB page. As agreed:

  • New setting cache_mb (HSTORE_CACHE_MB), default 256. The old default of 65,536 nodes could reach 1 GiB, so this is smaller in the worst case. Easy to change if you want a different default.
  • Each node counts as the size of its page: header plus payload when read from disk, the image size when a commit writes it. Real heap use runs somewhat above the limit, since a decoded node is bigger than its page; the docs say so.
  • cache_nodes is removed. If it's set in hstore.conf, as HSTORE_CACHE_NODES or as --cache-nodes, the server refuses to start (exit code 2) with: cache_nodes is no longer supported: the node cache is now sized in megabytes. Remove cache_nodes and set cache_mb instead. Before this, the variable and the flag would have been silently ignored.

How the cache changed: each of the 16 shards now has a byte budget (at least 64 KiB). Eviction is still CLOCK (second chance), but on a circular linked ring instead of a fixed array, so entries can differ in size. New entries go just behind the hand, so they get a full sweep before they can be evicted. A node bigger than a shard's whole budget isn't cached.

Also: Studio's /api/stats reports cacheBytes instead of cachedNodes, and the dashboard shows the cache in bytes. The benchmark keeps its effectively unlimited default (16 GiB, the same ceiling as the old 1M nodes), and CACHE_MB is now passed straight through instead of being divided by the page size.

Tested:

  • NodeCacheTest:
    • 20,000 puts of 100 B to 16 KiB never exceed the budget, and the cache stays mostly full
    • pages read between every insert survive 49,000 cold inserts
    • an oversized node isn't cached
    • putting a page again replaces it and its size
  • ServerTest: cache_mb is parsed, the default is 256 MiB, and cache_nodes is refused from the file, the environment and a flag
  • full ./mvnw install passes (97 tests); each commit compiles on its own, including the benchmarks module
  • real CLI: hstore config shows cache_mb; all three ways of setting cache_nodes give the message above, with exit code 2
  • live server with --cache_mb 64: /api/stats reports 67,108,864
  • native Docker image: healthy with HSTORE_CACHE_MB=128 (stats report 134,217,728), clean shutdown, and HSTORE_CACHE_NODES stops it with the message

Unrelated thing I noticed while testing: in JVM mode (java -m io.hstore.server), SIGTERM ends the process with code 143 before "shut down cleanly" is logged. main does the same, and the native image shuts down cleanly. I'll open a separate issue for it.

The cache limit was a number of decoded nodes, which says little about
memory: a node can be a few bytes or a full page. Each shard now has a
byte budget and every node counts as the size of its page, header plus
payload when read and the image size when written. CLOCK eviction is
kept, on a linked ring so entries can differ in size. The cache_nodes
setting becomes cache_mb, 256 by default.
…che_mb

Whether it comes from hstore.conf, an HSTORE_CACHE_NODES variable or a
flag, startup now stops with a message saying to use cache_mb, instead of
failing on an unknown key or quietly ignoring it.
@venkat1701
venkat1701 changed the base branch from feat/bench-amplification to main October 11, 2026 09:13
@venkat1701
venkat1701 merged commit a806952 into main Oct 11, 2026
@venkat1701
venkat1701 deleted the feat/cache-mb branch October 11, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Node cache is sized in nodes, not bytes

1 participant