Repository navigation
Reduce SKStore dependency storage overhead - #1585
Merged
Merged
Conversation
pikatchu
force-pushed
the
skstore-compact-dependencies
branch
from
November 15, 2026 12:31
c1b71fa to
3eaa41e
Compare
Store singleton dependency readers directly and pack the inverse directory-to-path index into immutable sorted blocks. Reuse the existing reader block implementation for paths, preserving ordering, equal-key representatives and snapshot semantics. Both improvements apply unconditionally; the existing multi-reader policies still select arrays, trees or blocks for larger reader sets. Add regressions against an independent relation model for insertion, deletion, block boundaries, directory removal, reinsertion and retained snapshots. Validation: 127 prelude tests pass in all 16 combinations of the existing storage options. All 21 incremental compiler regressions pass with both default and packed settings. The final compiler self-build produces 266 LLVM modules byte-identical to the tested prototype, and its generated compiler passes the smoke test. On the controlled prototype self-build, peak RSS fell from 20.21 to 18.67 GiB and reachable store memory from 13.32 to 12.40 GiB (23 million fewer objects). Total user+system CPU fell from 323.73 to 315.47 seconds. These are single cold-build trials; warm-update performance has not been benchmarked. The unconditional implementation's confirmation build retains the same memory result (18.67 GiB peak, 12.40 GiB reachable), with 329.16 seconds of CPU. It uses a rebuilt compiler; these timings do not establish a reliable speedup. Co-authored-by: mbouaziz <mbouaziz@users.noreply.github.com>
mbouaziz
force-pushed
the
skstore-compact-dependencies
branch
from
November 19, 2026 21:48
3eaa41e to
e421659
Compare
mbouaziz
enabled auto-merge
November 19, 2026 21:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Store singleton dependency readers directly and pack the inverse directory-to-path index into immutable sorted blocks. Reuse the existing reader block implementation for paths, preserving ordering, equal-key representatives and snapshot semantics. Both improvements apply unconditionally; the existing multi-reader policies still select arrays, trees or blocks for larger reader sets.
Add regressions against an independent relation model for insertion, deletion, block boundaries, directory removal, reinsertion and retained snapshots.
Validation: 127 prelude tests pass in all 16 combinations of the existing storage options. All 21 incremental compiler regressions pass with both default and packed settings. The final compiler self-build produces 266 LLVM modules byte-identical to the tested prototype, and its generated compiler passes the smoke test.
On the controlled prototype self-build, peak RSS fell from 20.21 to 18.67 GiB and reachable store memory from 13.32 to 12.40 GiB (23 million fewer objects). Total user+system CPU fell from 323.73 to 315.47 seconds. These are single cold-build trials; warm-update performance has not been benchmarked.
The unconditional implementation's confirmation build retains the same memory result (18.67 GiB peak, 12.40 GiB reachable), with 329.16 seconds of CPU. It uses a rebuilt compiler; these timings do not establish a reliable speedup.