Repository navigation
feat(⚡️): keep Dawn's compiled pipelines across launches - #594
Conversation
…create a cleared cache directory
… its size is checked
…ff when the JVM can't copy its path
…directory, and let a failed JNI copy throw
…without a directory
… as a session stores
…p and while storing
|
This is very compelling Thank You. |
wcandillon
left a comment
wcandillon
left a comment
There was a problem hiding this comment.
This is great and I am excited about this improvement.
Since you made the PR, we now set vulkan_monolithic_pipeline_cache for all Android devices, does this affect this feature on Android?
The documentation reads nicely. Is the 64MB claim still true?
If there is a crash happening, could we corrupt the cache file?
Do we need a single writer thread with a queue for safe access to the cache?
… as no growth An entry larger than half the limit is not stored, so one blob cannot take more than half the cache from the others. A replacement counts only the bytes it adds towards the next tidy, and the counter is reset with a compare-exchange so growth that lands during a reset is kept. Tidy checks the inode as well as the time and size before removing a file, so it does not remove one a store has just replaced. The statics a flush on another thread may still use at exit are never destroyed, and chain() says whether it chained the cache.
With vulkan_monolithic_pipeline_cache Dawn writes its one VkPipelineCache to the blob cache only inside PerformIdleTasks(), which nothing called, so on Android the Vulkan pipeline data never reached the disk. An IdleFlusher now calls it about a second after the last pipeline was created (about ten seconds into a long run of creations) and once on device.destroy(), for devices that have the blob cache, run on Vulkan, and have the toggle applied. A device without implicit synchronization is flushed only on destroy(). Cancelled on device loss, and never waited for when the device is released, since that can happen inside a Dawn callback. Messages Dawn logs during a flush stay native.
|
Thanks for the review. Short answers first, details below. I checked these on a Solana Seeker (Mali-G615, Android 16) and pushed the changes in a987838, 3361634 and 8357d57. 1. Does Yes, and it needed a fix. With that toggle Dawn keeps one What I changed:
One design note, since this changes the shape of #595's toggle: with the monolithic cache, persisting depends on this flusher (one blob per device, rewritten whole, and lost if the app is killed before a flush). Without the toggle, Dawn writes a small entry per pipeline as soon as it compiles and the load/store callbacks alone would persist it. I kept the toggle because #595 is there to avoid the per-pipeline Measured on the Seeker (debug build, one launch = four example scenes = 4 render + 1 compute pipelines, total time in
The full-cache launches were substantially faster, consistent with the Vulkan entry doing the work; the shader entries alone made little visible difference here. Each round ran empty, then shader-only, then full, in that fixed order, so system-level warmup and order effects are not controlled; it is one device, one debug build, simple pipelines, and only the synchronous creation path. I would not extrapolate the numbers, and I dropped the "tens to hundreds of milliseconds" claim from the docs for that reason. Raw numbers (ms, total / slowest single pipeline)
Each launch: force-stop, cold start, open Hello Triangle, Hello Triangle MSAA, Image Blur and Cube in turn (7 s each), idle 4 s so the flush runs. "Shader only" is the full cache with the 2. Is the 64MB claim still true? The limit is, but the docs were stale: they said the cache is cleared, and since 55997fe it is trimmed. The target is 64 MiB, checked when the app starts and again after about 8 MiB of accumulated growth from stores. A replacement counts only the bytes it adds, and the count is approximate. The oldest-written entries go first (reads do not refresh an entry). Entries above 32 MiB are not stored, so one blob cannot take more than half the cache from everything else, and any previous entry under that key is left as it was; the docs say that a Vulkan blob that outgrew that would stop being updated. 3. Could a crash corrupt the cache file? Entries are written to a temporary file, closed, then renamed into place, so an ordinary process crash does not publish a partial replacement; a later 4. Do we need a single writer thread with a queue? Not for integrity. Temporary names are unique per process and write, rename is atomic, and Not verified
All 42 host tests pass; the flusher and selected cache tests also passed repeated ThreadSanitizer runs. Separate from this PR: |
|
@copilot resolve the merge conflicts in this pull request |
# Conflicts: # packages/webgpu/apple/WebGPUModule.mm # packages/webgpu/cpp/rnwgpu/api/GPUAdapter.cpp # packages/webgpu/cpp/rnwgpu/api/GPUDevice.cpp
…clear a cache past its limit Dawn stores from the thread that compiles, a JS thread included, and holds the device's lock while it stores its pipeline cache. A store now copies the bytes and returns; one writer thread does the disk work, in order. Tidying runs on that thread too, and is simpler: a cache found past 64 MB at setup is cleared, since which of its entries the app still uses cannot be told, and a session that would take the cache past the limit stops adding to it (a replacement no larger than the entry it replaces is never dropped). Only names shaped exactly like the cache's own are counted or removed.
… app pauses Dawn writes its monolithic Vulkan pipeline cache only when the embedder says the device is idle. Rather than guess at idleness with a thread of our own (IdleFlusher), use the moments the platform gives: device.destroy(), and the app going to the background, which Android reports through onHostPause. Each device flushes on the thread of the runtime it was created on, so Dawn's callbacks keep running where they always did. There is nothing to flush on Metal, where Dawn stores its blobs as it compiles.
…runtime, and never for an imported device An app killed in the foreground lost every pipeline built since launch: nothing flushed between onHostPause and destroy. GPUQueue::submit now asks its device to flush when ten seconds have passed since the last one. That is cheap: Dawn only serializes when a pipeline compiled since, and skips the store when the cache's size did not change. The pause flush posted to each device's runtime, but a worklet runtime of the Mailbox kind delivers a post only with its next operation, so a device created there never flushed. Devices with ImplicitDeviceSynchronization, the default, now flush on the calling thread; PerformIdleTasks takes the device's lock itself. Only devices without it are posted to their runtime. A device from importDevice() belongs to someone else: it keeps its origin and is never flushed, on destroy included.
…asure the cache again before dropping one A job that threw took the writer thread down with it, and when no thread could be started the jobs ran inline on Dawn's thread, under the device's lock. Jobs now run under try/catch, and are dropped without a thread. The byte count the writer keeps went stale when the OS purged the files but kept the directory, so stores were dropped from an empty cache. A store that would be dropped measures the directory first. Replacements no larger than the entry they replace stay exempt, as the header promises. Temporary names match exactly the one shape the cache writes; the test pins the older `<hash>.<pid>.<seq>.tmp` shape as a file the cache leaves alone. The test helpers that create and age files return whether they did, and every call asserts it.
…letest download cached The tests ran inside the Android build job, where a failure read as an Android failure and a cold googletest fetch added to the slowest job. They now run on ubuntu in under a minute, and actions/cache keeps the FetchContent checkout under node_modules/.cache/cpp-tests/_deps, keyed on the CMake file.
…evice wrapper on the UI thread, and measure the cache again only when it changed The periodic flush of a device that synchronizes itself goes to the blob cache's thread (DawnBlobCache::post), holding the device by its Dawn handle and a shared flag rather than its wrapper, which must only go on its runtime's thread. flushPipelineCaches() copies what it needs out of each wrapper under the registry's mutex (the registry now holds plain pointers and asks weak_from_this()), so the pause-time flush never takes a strong reference on the UI thread. A store the cache refuses as too large is counted on the calling thread, and a flush that sees the count move marks the device's pipeline cache as outgrown and stops having Dawn serialize it. In the cache, the directory is published before its tidying is posted, and a store finding its directory untidied (the tidying dropped for want of a thread, or the store bound for another directory) tidies it first. Before a store is dropped the directory is measured again only when its modification time changed since the last measure, or that measure is ten seconds old. Also: GPUQueue's members in initializer order, the docs on when the Vulkan cache data is written (a submit ten seconds after the last write), and the googletest cache in CI limited to the download and its sub-build.
|
🎉 This PR is included in version 0.21.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
Dawn can reuse what it produces while building shaders and pipelines across launches, but only when the embedder gives it storage. That storage is either load/store callbacks on the device descriptor (
DawnCacheDeviceDescriptor), or a caching interface on the platform the instance was created with, which is how Chrome does it. react-native-webgpu provides neither, so every launch builds every pipeline from scratch.On Vulkan (Android) the cached data includes the driver's pipeline data, which is where most of the time goes. On Metal (iOS) it is Dawn's MSL translation; Metal's own system cache keeps the compiled pipelines.
This adds a small file-backed cache (
cpp/rnwgpu/DawnBlobCache) and chains it onto every devicerequestDevice()creates. It uses device callbacks rather than a platform interface because the instance can be React Native Skia's.getCacheDir()/webgpu-dawn.Caches/<bundle id>/webgpu-dawn. The bundle id is included because macOS shares~/Library/Cachesbetween apps.dawnToggles: { enabledToggles: ["disable_blob_cache"] }is Dawn's own toggle. It clears that device's cache callbacks. The install-time housekeeping still runs.Measurements
These were taken with Release builds of the example app from this branch. The later commits change only directory handling. A temporary screen, not part of the PR, requests a device and builds 48 distinct render pipelines, each with a fragment shader using lighting, noise and a short loop. Times run from
requestAdapter()to the last pipeline resolving, so they leave out the housekeeping above. Cold is the first launch after install; warm is a later launch.disable_blob_cacheIn a production app that builds 88 pipelines at startup, the same logic applied as a patch to 0.13.0 took the Seeker from 2.6 s, plus 0.6 s of JS-thread stall creating shader modules, to 0.27 s on warm launches.
Tests
yarn test:cppbuilds 20 host unit tests ofDawnBlobCachewith CMake (3.24 or later) and GoogleTest against Dawn's headers, with no GPU needed. CI runs them in the Android job. They cover:They pass under AddressSanitizer and UndefinedBehaviorSanitizer with
-Werror. Each behavioural fix made during review has a test that fails without it.What the tests don't cover:
GPUAdapter::requestDevice, the JNI and Apple wiring, and the opt-out toggle, which were checked on the devices above;The test setup mirrors #550, which adds the repo's first host C++ tests and fetches GoogleTest the same way. Whichever lands second needs the two
CMakeLists.txtsource lists and the CONTRIBUTING lines combined.Not covered
importDevice(), such as React Native Skia's Graphite device, are created by their own library and don't get the cache. Skia would need the same descriptor on its side.