Repository navigation
Rename cache_mb to node_cache_mb and count it in heap bytes - #78
Merged
Merged
Conversation
Once pages are read through memory maps, the OS caches the raw bytes and this budget only covers decoded nodes on the heap. The engine option, the builder and the Studio stats key now say so. Nothing here has been released yet, so no alias is kept.
cache_mb read as if it limited all caching, but it only sizes the decoded nodes kept on the heap. Setting cache_mb in hstore.conf, as HSTORE_CACHE_MB or as --cache-mb now fails at startup with a message that names node_cache_mb, the same way cache_nodes does. The cache_nodes message points to the new name as well.
The server is a native image with the serial collector, so everything the node cache holds is copied at every full collection. With page reads going through memory maps, a miss costs a copy out of the OS page cache and a decode rather than a system call, so a smaller decoded cache costs much less than it used to. The default drops from 256 to 64 MiB.
The configuration page explains that only decoded nodes live on the heap, that the serial collector walks all of them on a full collection, and that both cache_mb and cache_nodes are refused at startup.
node_cache_mb counted each node as the size of its page, but a decoded node takes three to six times that on the heap. At scale 4 a setting of 256 kept 1.1 to 1.3 GiB, so the setting said little about memory. Node.heapBytes estimates the decoded size from the node itself: a fixed 144 bytes for the node and its summary, its arrays, 96 bytes plus the encoded value for each leaf entry, and 128 bytes per child reference. The constants come from comparing the retained heap with the cache's contents after the benchmark's read workloads. The estimate came out 10 to 40% under the measured heap at scales 1 and 4, against 3.6 to 5.8 times under before.
With nodes counted at their heap size, 64 would hold about a quarter of what the old default of 64 held, because that one was really using about 250 MB of heap. 256 keeps the same memory and the same read speed, and the number now says what it uses. The setting's help text says so too.
venkat1701
changed the title
Each of the 16 shards got a sixteenth of the budget, and a node heavier than its shard was never cached. With a small node_cache_mb and large pages that silently skipped every interior node. The cache now picks as many shards as leave room for 64 pages each, from 1 up to 16.
A huge value overflowed when it was shifted into bytes, and 0 failed later with an engine message that didn't name the setting.
Now that nodes count at their heap size, a 16 GiB cap could never bind before a 4 GiB heap ran out. The docs also describe the cap and the estimate as they are now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second half of moving the page cache off the heap, stacked on #69. #77 reads pages through memory maps, so raw bytes sit in the OS page cache. The one cache HStore still sizes itself holds decoded nodes on the Java heap. This renames it to say that, and makes the number mean what it says.
What changed
cache_mbbecomesnode_cache_mb,HSTORE_NODE_CACHE_MBand--node-cache-mb.cache_mbnever shipped in a release, so instead of an alias the server refuses it at startup from the config file, the environment or a flag, with a message that namesnode_cache_mb. Thecache_nodesmessage now points at the new name too.EngineOptions.cacheBytes/withCacheBytesbecomenodeCacheBytes/withNodeCacheBytes, and the Studio stats keycacheBytesbecomesnodeCacheBytes.Node.heapBytes) instead of their page size.Why the count changed
On an Azure D8ds_v5 at scale 4, a setting of 256 kept 1.1 to 1.3 GiB on the heap. A decoded node takes three to six times the size of its page, so counting pages said little about memory.
Node.heapBytesworks it out from the node: 144 bytes for the node and its summary, its arrays, then 96 bytes plus the encoded value for each leaf entry, or 128 bytes per child reference in a branch. I got the constants by filling the cache through the benchmark's read workloads and comparing retained heap with what was cached. With the estimate, the cache's own count came out 10 to 40% under the measured heap at scales 1 and 4. Before, it was 3.6 to 5.8 times under.Why 256
An earlier version of this PR set the default to 64, counted in page bytes, which was really about 250 MB of heap. 256 counted in heap keeps that same memory and the same read speed. On the same VM at scale 4, a cache about twice that size made member reads about 40% faster and two-hop reads about 58% faster, so anyone with a large working set should raise it. The server binary uses the serial collector, and every full collection walks this cache, so it isn't free to raise.
Testing
ServerTest: the new setting from the environment, the 256 default, andcache_mbandcache_nodesrefused from file, environment and flag, each message namingnode_cache_mb. Takingcache_mbout of the refused list fails the test.Merge order
Ideally after #77, since misses are cheaper with it. Without it, misses still work; they're just slower.