Skip to content

feat(jit): filesystem shared-library cache with isolated state binding - #56

Merged
esrrhs merged 6 commits into
masterfrom
feat/jit-filesystem-cache
Nov 16, 2026
Merged

esrrhs merged 6 commits into
masterfrom
feat/jit-filesystem-cache

Conversation

@esrrhs

@esrrhs esrrhs commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Summary

Implement a best-effort filesystem cache for GCC/Clang JIT shared libraries while keeping runtime data isolated per loaded image.

  • Replace embedded State/heap-slot/interned-string addresses with generated C statics bound once by __fakelua_bind. Literal bytes and numeric constants remain in generated C; canonical strings and runtime objects remain State-owned.
  • Use SHA256 identity plus a checked manifest: generated C, host source/build ABI (including active multi-config configuration/custom flags), compiler binary/path/version, relevant environment/platform, header contents and compiler backend/linker/CRT dependencies. Unsupported custom include/library/search-path configurations, wrappers and coverage builds keep the uncached path.
  • Serialize each key with a thread mutex plus OS file lock; stage complete entries and rename atomically. Validate checksums/dependencies, recover corrupt/partial entries, release locks on process exit, and fall back cleanly when caching is unavailable.
  • Copy each load to a unique private workspace instead of loading/hardlinking the persistent inode: mutable static variables, init flags, closures and diagnostics must not be shared between States.
  • Retain State-canonical string identity and lifetime across module replacement/unload. Wide string-key accessors use cached-hash dispatch plus identity collision guards instead of linear comparison regressions.
  • Add StateGCCConfig.enable_cache (default true) / cache_directory, documentation in both READMEs, and a reproducible public-API benchmark harness. TCC stays in-memory; INTERP is not cached. No new dependency is introduced (OpenSSL is already linked).

Additional review regressions and discovered fix

Added six further cases beyond the original cache tests, in their existing jitter/exception modules:

  1. Retain a raw string-ID CVar across actual producer-image unload (verified by expired weak handle), then use it as an interned key in another module and verify pointer identity after cache reload/reset.
  2. File-level nonnumeric static initialization runs exactly once per private load, takes the correct State-specific native callback result, and reruns after same-State cache-hit recompilation without changing another State.
  3. Failed module initialization followed by same-State cache-hit retry neither caches partial values/init flags nor retains dangling diagnostic pointers.
  4. Recursive initialization can compile the same cache key while outer init is active; the temporary outer image unregisters its own location, not the inner image's.
  5. Malformed/oversized manifests and deleted persistent images recover while an already loaded private image continues executing.
  6. Same-State failed-init retry for GCC, TCC and INTERP, in debug and optimized configurations; this case also remains enabled in coverage builds.

Case 3 reproduced a real crash (exit 139) before the fix. The binder now returns the exact image's Lua-location pointer and the GCC/TCC handle owns it before init runs, including throwing/reentrant init paths. Do not infer ownership from GetLuaLocs().back() after successful init.

Existing new cache cases also cover 8-thread/4-process races, abrupt process exit with held lock/partial staging, independent States, native callbacks, all 256 byte values/NUL/binary keys, hash collisions, diagnostics filenames, compiler/header/source/debug invalidation, cache disable/fallback and compile failure not publishing.

Local validation

  • Release build and 1208 scoped module tests passed: core 439 + language 514 + library/native 255, with module groups run serially. The final language rerun includes the last added cross-backend exception regression.
  • 16/16 cache-focused tests passed without sandbox or symbolizer stub.
  • Large module runs mounted a no-op addr2line only to avoid prohibitively slow C++ stacktrace symbolization; this was not used for the separate 16-test run. Network/process/database integration suites are not claimed as locally run.
  • Earlier Debug/USE_COV targeted validation passed 4/4 before the final init-location hardening; that full variant was not rerun locally after the follow-up. Current CI must validate it; the new cross-backend recovery test is coverage-enabled.
  • Ninja Multi-Config fixture compiled Debug/Release identities separately; custom configuration-flag changes also changed the ABI fingerprint.
  • git diff --check passes. Original unrelated coverage/log artifacts and all build outputs are excluded from this PR.
  • macOS, Windows and ASAN have not been executed locally. Their workflows are unchanged and must report their actual CI results.
  • Two earlier concurrent-module runs had one unexplained GCC abnormal exit with an empty compiler log each. Subsequent complete serial runs passed; no compiler retry was added to mask them, and no definitive cause is claimed.

Reproduce the scoped groups from the Release build's bin directory:

./unit_tests --gtest_filter='jitter.*:state.*:runtime.*:vm_cvar_call.*:common.*:syntax_tree.*:util.*:var.*'
./unit_tests --gtest_filter='exception.*:closure.*:test_basic.*:infer.*:package.*:test_log.*'
./unit_tests --gtest_filter='test_string.*:test_math.*:test_table.*:test_native.*:test_native_version.*:native.*:test_container.*:test_utf8.*:algo.*'
./unit_tests --gtest_filter='jitter.gcc_cache_*:native.gcc_cache_*:exception.gcc_cache_*'

Performance (initial cache implementation, before the FIFO/formatting follow-up)

Linux x86_64, GCC 14.2.0, Release -O3, CPU 7 pinned, 9 samples per invocation, same public-API harness and Lua workload; no concurrent builds/tests from this task. Baseline library/header are the unmodified cc0e0c84873b4d4de2d75acc181e3bd0104e1e12, built before edits. Same-process warm median excludes the first cold sample.

Compilation mode Median/first hit
Unmodified baseline 1751.20 ms
New cold cache 1821.46 ms (+4.0%)
Same-process warm 4.816 ms (~364x faster)
Independent process, first existing-cache hit 64.963 ms (~27x faster)
That process, subsequent warm median 4.669 ms

The persistent image checksum and mtime remained unchanged across independent execs, with exactly one image in a confirmed fresh benchmark cache.

Steady-state workload (fixed loop counts), ms Baseline New warm
arithmetic 28.7213 28.3984
dynamic string field lookup 55.8536 53.4750
string comparison 24.9490 22.8392
varying interned 16-field lookup 82.8542 72.0993

These are representative microbenchmarks, not a universal zero-regression guarantee. Embedded address immediates become static-variable reads; filesystem lookup/intern callbacks do not enter the JIT operation hot path. Lua frontend work and a cheap compiler-version metadata query remain on a cache hit. The once-per-load binder also returns the diagnostic location; it is not a runtime dispatch table.

The standalone benchmark README was removed at the user's request. The harness remains in
tools/benchmark_jit_cache.cpp. Build against the matching current header/library, e.g.:

c++ -O3 -std=c++23 -Iinclude tools/benchmark_jit_cache.cpp -Lbuild/lib -lfakelua \
  -Wl,-rpath,"$PWD/build/lib" -o /tmp/benchmark-jit-cache

Then run:

taskset -c 7 ./benchmark-jit-cache cold /tmp/fakelua-cache-benchmark-cold
taskset -c 7 ./benchmark-jit-cache warm /tmp/fakelua-cache-benchmark-warm
taskset -c 7 ./benchmark-jit-cache warm /tmp/fakelua-cache-benchmark-warm # new process

For the unmodified base, build this same harness with -DFAKELUA_CACHE_BASELINE, its original header and baseline library. This excludes only the new cache configuration API, not any measured loop/code.

Limits / operational notes

Use a trusted private local cache directory. The default is now the original temp tree:
<system-temp>/fakelua/cache/uid-<uid> on POSIX or <system-temp>/fakelua/cache on Windows,
selected by the same helper as transient JIT workspaces. cache_directory still overrides it.
cache_max_entries defaults to 256 complete entries; 0 means unlimited, not disabled.
Eviction is FIFO by publication sequence, not access time, and runs on lookup/publication.
A directory-wide thread/process lock coordinates restore/publication/eviction; C compilation
remains outside it. 256 fixed key-lock stripes plus one index lock bound lock metadata;
eviction never unlinks lock files. Stripe collisions can serialize unrelated compiles.
Deletion/permission failures are best effort and may leave temporary overshoot; clients
sharing a directory should use the same count limit. This is not a byte-size limit.

System temp cleanup is policy-dependent, not guaranteed. Deletion of cache entries/the
cache subtree is expendable acceleration: misses recompile and loaded private modules
survive. POSIX external unlink can detach active lock inodes, so file locking alone is
not claimed to solve that case: detached locks are checked, staging paths are writer-private,
publication is atomic and executable bytes are verified before private-copy loading.
Concurrent external cleanup can cause redundant compilation/cap overshoot, not partial-image
execution. Interrupted writers may leave uncounted .tmp.* directories; clean while idle.
For deterministic manual cleanup, stop cache users first, especially before deleting locks. A State's existing single-threaded-use contract is unchanged: concurrency tests use independent States. Custom/hidden-input toolchains must opt out, and toolchains should not be mutated during compilation.

Requested follow-up: temporary defaults, FIFO and generator organization

  • Reuse GetTmpDirectory() for the original system-temp/fakelua tree; POSIX private UID leaf
    stays 0700, symlinks/insecure or foreign-owned parents are rejected. No home/XDG/LOCALAPPDATA default.
  • Add configurable bounded FIFO, monotonic publication order (including corrupt-age overflow
    recovery), count complete image/manifest pairs only, and preserve unrelated/partial files.
  • Coordinate concurrent different-key publishers/readers/evictors without taking key locks
    from inside the directory lock. Loaded private copies continue running after eviction.
  • Extract CGen::GenerateBindings() and use Includes/StringDecls/Bindings Sections; remove
    special prefix/local-stream concatenation. The binder ABI and once-per-load hot-path design are unchanged.
  • Add 12 cache regressions plus 1 utility regression in the existing modules: default/custom paths,
    POSIX permissions/symlinks, FIFO/0/lowered limits/non-LRU hits/corrupt ages, threaded/process publishers,
    bounded lock metadata, preservation of unrelated files/loaded images, root/entry deletion,
    unlink of a held process lock, and cleanup racing restore/compilation/live execution.
  • Before repository-wide formatting, Release build and 49/49 cache/binding + utility tests passed
    without sandbox or symbolizer replacement.
  • At the user's request, formatted all 244 tracked repository-owned C/C++ files with
    root .clang-format, using clang-format 21.1.8; 71 files changed, in a separate
    formatting-only commit. Excluded checked-in generated Flex/Bison outputs and vendored
    src/platform/FlexLexer.h, plus all untracked/build/dependency trees and unsupported languages.
  • Formatter fixed-point/idempotence checks pass (--dry-run --Werror). Lua/other raw string
    literals were checked unchanged. The formatted C runtime coverage view was synchronized
    byte-for-byte into kCRuntimeHeader; preprocessed runtime tokens match the pre-format version.
  • Final formatted Release build passed; 455/455 core-module + native/exception cache tests passed
    without sandbox or a symbolizer replacement (145.470 seconds). This is the final follow-up's
    actual scope, not a claim to rerun all 1208 older scoped cases or external integration suites.
  • Build command (existing local CPM build uses its already configured bootstrap path):
    env -u CPM_SOURCE_CACHE cmake --build build-cpm-fix --target unit_tests -j 2
  • Final test command, from build-cpm-fix/bin:
    ./unit_tests --gtest_filter='jitter.*:state.*:runtime.*:vm_cvar_call.*:common.*:syntax_tree.*:util.*:var.*:native.gcc_cache_*:exception.gcc_cache_*'

Final follow-up release measurements

Same public-API harness, Linux x86_64, /usr/bin/g++ 14.2.0, Release library,
CPU 7 pinned, nine samples per invocation, no concurrent build/test from this task:

Mode Final follow-up milliseconds
Cold compile median (fresh disposable cache before each sample) 1668.05
Same-process cache-hit median (exclude first cold sample) 4.775835
Independent exec, first existing-cache hit 59.138
That exec, subsequent-hit median 4.803665

Cold-to-warm compilation is about 349x for this workload. Final same-process warm
steady-state medians (arithmetic / dynamic field / string comparison / varying
interned lookup) are 25.808 / 48.63425 / 20.7931 / 66.049 ms. The unmodified-base
results above were measured earlier, not a newly interleaved baseline run; do not
interpret noise or these examples as a universal zero-overhead guarantee.

Reproduce this follow-up with the existing local Release layout:

/usr/bin/g++ -O3 -std=c++23 -Iinclude tools/benchmark_jit_cache.cpp \
  -Lbuild-cpm-fix/lib -lfakelua -Wl,-rpath,"$PWD/build-cpm-fix/lib" \
  -o build-cpm-fix/benchmark-jit-cache-followup
taskset -c 7 build-cpm-fix/benchmark-jit-cache-followup cold "$PWD/build-cpm-fix/fakelua-cache-benchmark-followup-cold"
taskset -c 7 build-cpm-fix/benchmark-jit-cache-followup warm "$PWD/build-cpm-fix/fakelua-cache-benchmark-followup-warm"
taskset -c 7 build-cpm-fix/benchmark-jit-cache-followup warm "$PWD/build-cpm-fix/fakelua-cache-benchmark-followup-warm"

@codecov-commenter

codecov-commenter commented Nov 16, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 68.16239% with 298 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.79%. Comparing base (cc0e0c8) to head (a3ec65b).

Files with missing lines Patch % Lines
src/jit/gcc_cache.cpp 3.11% 249 Missing ⚠️
src/interp/interpreter.cpp 94.73% 14 Missing ⚠️
src/jit/gcc_jit.cpp 85.18% 8 Missing ⚠️
src/native/string/native_string.cpp 52.94% 8 Missing ⚠️
src/native/process/native_process.cpp 0.00% 4 Missing ⚠️
src/native/mysql/native_mysql_pool.cpp 0.00% 3 Missing ⚠️
src/compile/c_gen.cpp 97.93% 2 Missing ⚠️
src/native/http/native_http.cpp 92.30% 2 Missing ⚠️
src/native/mysql/native_mysql.cpp 50.00% 2 Missing ⚠️
src/native/net/net_event.cpp 87.50% 2 Missing ⚠️
... and 3 more
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #56      +/-   ##
==========================================
- Coverage   89.96%   88.79%   -1.18%     
==========================================
  Files         125      127       +2     
  Lines       28838    29649     +811     
==========================================
+ Hits        25945    26326     +381     
- Misses       2893     3323     +430     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

libc++ on macOS uses __int128 for file_clock ticks, which has no operator<< and broke the BuildWithMacOS compile.
MinGW last_write_time often comes from stat() and is only 1-second
resolution, so a same-size header rewrite in the same process could
reuse a stale FileDigest and miss invalidation.
Open cache images, manifests, and lock files with FILE_SHARE_DELETE, and
unlink with POSIX disposition so an external cleaner can remove a tree
without waiting on Restore or a held lock handle.

MinGW libstdc++ remove_all can livelock in __erase when a child is open
or delete-pending and fs::remove fails without an error_code; the
Windows Test job hit that in gcc_cache_external_cleanup_races_restore_without_partial_copy
and timed out after 60 minutes. Cache eviction now uses a one-pass
Windows walker, StillLinked compares file identity after an unlink, and
the cleanup-race tests use the same non-retrying delete.
@esrrhs
esrrhs merged commit c80a1e0 into master Nov 16, 2026
9 checks passed
@esrrhs
esrrhs deleted the feat/jit-filesystem-cache branch November 16, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants