[jwx0925]

What this PR does / why we need it:

After producing and publishing an immutable blob, a local producer otherwise has to download the same bytes again when an image first reads them. Add ImageService::import_local_blob(path, url, digest, size) and POST /v1/cache/import on the existing ublkd Unix socket to seed the daemon's managed file cache from a staging file.

Imports validate the complete SHA-256 before writing, then verify each reread 256 KiB chunk against the first pass before publishing it. Verified partial fills remain safe after mutation, truncation, or write failure; normal source fallback and existing cache capacity/eviction remain available. Admission is synchronous, limited to one active import per ImageService and blobs up to 64 GiB. Success does not pin data or guarantee crash durability.

Support P2P with an explicit local-cache-first policy:

{
  "cacheConfig": {"cacheType": "file"},
  "p2pConfig": {
    "enable": true,
    "address": "http://127.0.0.1:9731/accelerator",
    "cacheFirst": true
  }
}

With cacheFirst: true, reads always enter the managed file cache. Missing or evicted ranges use RegistryFS with the selected P2P accelerator and refill the cache. Rechecking accelerator availability during image creation preserves that read path; an unsuccessful availability check selects the direct registry source beneath the cache. This does not add per-request failover after accelerator selection.

The option defaults to false, preserving existing P2P bypass behavior, and requires the effective cache type to be file. P2P-enabled imports without this option return HTTP 409, including before the first image selects a path, so an accepted warmup cannot silently become irrelevant due to subsequent bypass selection.

Use the same effective cache-type resolution for initialization and import admission: an omitted/empty nested cacheConfig.cacheType falls back to the legacy top-level cacheType, defaulting to file. This fixes imports being rejected despite a successfully initialized file cache.

Registry publication, upload-side orchestration, retries, and staging cleanup remain producer responsibilities. Device-pool and mount lifecycles are unchanged. Cache-first P2P can retain a second local copy if the accelerator also caches data; standalone usage documents this tradeoff.

Validation:

• Reproduced both regressions against the prior code: legacy/default configuration returned 409, and selecting P2P bypassed the imported cache.
• Built overlaybd-ublkd and the affected test targets in Linux/amd64 Release with the repository-pinned Photon v0.9.6.
• Passed ublkd_cache_import_test, p2p_cache_test, and ublk_dispatch_test, plus all three existing ImageTest.* tests.
• The new P2P test uses real ImageService, Photon file cache, RegistryFS v1/v2, and loopback HTTP origin/accelerator servers. Ten routing scenarios cover early imports, zero-HTTP imported hits after staging deletion, partial/full eviction, accelerator refill and subsequent local hits, unavailable/disabled P2P, and default bypass. Additional cases reject unsupported cache types.
• Both cache test targets also build and pass with BUILD_UBLK_FRONTEND=OFF; no KVM or ublk device is needed.
• No cluster deployment or sandbox performance benchmark was rerun for this update.

Which issue(s) this PR fixes:

None linked.

Please check the following list:

• Does the affected code have corresponding tests, e.g. unit test, E2E test?
• Does this change require a documentation update? Updated standalone usage and the configuration reference.
• Does this introduce breaking changes that would require an announcement or bumping the major version? No; the new P2P policy is opt-in.
• Do all new files have an appropriate license header?

[jwx0925]

@pakkah Could you please review this PR? It adds local blob ingestion into the managed file cache through ImageService and a ublkd API. Thanks!

[jwx0925]

@codex review

[pakkah]

LGTM

[Copilot]

Note

Copilot was unable to run its full agentic suite in this review.

Copilot review overview

6 open issues

• cached is opened with O_RDONLY, but the code then calls ftruncate() (and later pwrite()) on… · New
• The switch has no default: handling. If CacheImportStatus is extended in the future (or an… · New
• This test writes ~1.2 GiB of data (plus hashing/reads) under /tmp, which risks slow/flaky CI runs… · New
• effectiveCacheType() does not mutate GlobalConfig and should be marked const. This makes the… · New
• There are now two different translation units named cache_import.cpp in the tree… · New
• The 400 error message is a bit ambiguous and doesn’t clearly indicate the expected JSON schema… · New

What changed in this PR

Adds a local “cache import” capability to seed the managed file cache from a producer’s local blob, plus a P2P “cache-first” mode so reads continue to traverse the cache when acceleration is enabled.

Changes:

• Add /v1/cache/import HTTP endpoint and JSON request parsing/mapping to ImageService::import_local_blob.
• Implement ImageService::import_local_blob with digest/size verification, concurrency guard, and cache population behavior.
• Add integration tests for cache import and P2P cache routing; document the new endpoint and config.

File	Description
src/​ublk/​ublkd_main.cpp	Routes new POST endpoint /v1/cache/import to the handler.
src/​ublk/​cache_import.h	Declares HTTP parsing/status mapping entrypoint for cache import.
src/​ublk/​cache_import.cpp	Implements JSON parsing and maps CacheImportStatus to HTTP codes/messages.
src/​ublk/​CMakeLists.txt	Builds the new ublk-side cache import handler into ublk_frontend_lib.
src/​cache_import.cpp	Implements ImageService::import_local_blob (verification + cache fill).
src/​image_service.h	Adds cache import API/types and a single-import concurrency flag.
src/​image_service.cpp	Uses effective cache type; enforces cacheFirst requires file cache; routes remote_fs based on cacheFirst.
src/​config.h	Adds p2pConfig.cacheFirst; adds GlobalConfig::effectiveCacheType().
src/​ublk/​test/​cache_import_test.cpp	New integration test for local cache import scenarios/faults/eviction.
src/​test/​p2p_cache_test.cpp	New integration test for P2P routing behavior with/without cache-first.
src/​test/​CMakeLists.txt	Adds the two new test executables and CTest entries/timeouts.
src/​CMakeLists.txt	Includes cache_import.cpp in overlaybd_image_lib.
docs/​standalone-usage.md	Documents cache import endpoint and cache-first P2P behavior.
README.md	Documents p2pConfig.cacheFirst config key.

_{Review effort: Lite}

[Copilot]

🟡 Changes recommended

The raw URL check accepts hostless URLs and blob suffixes found only in fragments, allowing imports to return HTTP 200 for URLs that cannot supply the intended blob on a cache miss.

1 open issue

• Validate blob URLs against their parsed authority and path · New

6 resolved since last review

• cached is opened with O_RDONLY, but the code then calls ftruncate() (and later pwrite()) on…
• This test writes ~1.2 GiB of data (plus hashing/reads) under /tmp, which risks slow/flaky CI runs…
• The switch has no default: handling. If CacheImportStatus is extended in the future (or an…
• The 400 error message is a bit ambiguous and doesn’t clearly indicate the expected JSON schema…
• There are now two different translation units named cache_import.cpp in the tree…
• effectiveCacheType() does not mutate GlobalConfig and should be marked const. This makes the…

_{Review effort: Balanced}

[jwx0925]

@BigVan Could you please help review this PR? It adds local blob import into the managed file cache and an opt-in cache-first P2P read path. Thanks!