You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
One cached object larger than the low-water target makes the next eviction sweep empty the whole store, including that object #98
LeastRecentlyUsedEvictionPolicy.Evict() deletes objects coldest-first until TotalBytes <= MaxSize * LowWaterMark (LeastRecentlyUsedEvictionPolicy.cs#L27-L65).`` Nothing compares a single object's size against that target. The miss path (ObjectRouteHandler.cs ~L335) and the upload tee (~L229) stage and publish an object of any size.
If one object is larger than the low-water target (90% of MaxSize by default), a sweep:
deletes every other object, and the store is still over target;
then deletes the large object as well, even though it is the hottest object because it was just fetched.
The store ends up empty. If that object keeps being requested, for example a large .umap or a build artifact that every CI pod pulls, the cycle repeats every MaintenanceInterval:
the next miss re-fetches and re-stores it;
the next sweep wipes everything again.
The hit ratio stays near zero, and the whole volume is rewritten every interval. This happens for an object between LowWaterMark × MaxSize and MaxSize, which fits the budget, as well as for one larger than MaxSize.
Repro
This test uses the Create/StoreAsync helpers in GitLfsCache.Tests/Storage/EvictionTests.cs:
(ObjectStorestore,LeastRecentlyUsedEvictionPolicypolicy,FakeTimeProvidertime)=Create("1000B");// target 900Bfor(inti=0;i<5;i++){awaitStoreAsync(store,100,(char)('a'+i));time.Advance(TimeSpan.FromHours(1));}awaitStoreAsync(store,950,'z');// fits the budget, exceeds the low-water markEvictionResultresult=policy.Evict();// evicted=6 bytes=1450 remaining=0 total=0
Suggested fix
Don't cache an object that can't survive a sweep. The href token already carries Size. When token.Size is over a cap, skip TryOpenStaging and relay the object uncached, as the staging-failure path already does. The cap could be a new Store:MaxObjectSize setting, or default to a fraction of MaxSize × LowWaterMark. Apply the same rule in UploadAsync, and count these in a metric such as gitlfscache.objects_too_large.
As a backstop, the sweep should not delete an object larger than the remaining excess while smaller objects can still be freed, and it should never empty the store for one object.
Acceptance criteria
With MaxSize=1000B and LowWaterMark=0.9, a 950-byte download is served correctly. Either it isn't stored, or it's stored without the store being emptied.
An object larger than MaxSize is relayed uncached, and the metric records it.
Priority: High. One large, frequently requested object (a .umap, a CI artifact) empties the whole store on every MaintenanceInterval. The hit ratio drops to about zero and the volume is rewritten each cycle. Nothing reports it, and it happens with default settings for any object between 90% and 100% of MaxSize, as well as for any object larger than MaxSize.
Area / suggested assignment: Storage and eviction (LeastRecentlyUsedEvictionPolicy.Evict), plus the admission paths in ObjectRouteHandler (miss staging around L335, upload tee around L229). It needs one owner across both, because the primary fix is at admission and the backstop is in the sweep.
Next step: add an admission cap (Store:MaxObjectSize, defaulting below MaxSize × LowWaterMark) that relays oversized objects uncached and counts them in a metric. Then make the sweep skip an object larger than the remaining excess while smaller objects can still be freed. The repro in the issue is ready to use as the regression test.
Activity
matt-edmondson commentedon Oct 7, 2026
Triage
.umap, a CI artifact) empties the whole store on everyMaintenanceInterval. The hit ratio drops to about zero and the volume is rewritten each cycle. Nothing reports it, and it happens with default settings for any object between 90% and 100% ofMaxSize, as well as for any object larger thanMaxSize.LeastRecentlyUsedEvictionPolicy.Evict), plus the admission paths inObjectRouteHandler(miss staging around L335, upload tee around L229). It needs one owner across both, because the primary fix is at admission and the backstop is in the sweep.Store:MaxObjectSize, defaulting belowMaxSize × LowWaterMark) that relays oversized objects uncached and counts them in a metric. Then make the sweep skip an object larger than the remaining excess while smaller objects can still be freed. The repro in the issue is ready to use as the regression test.Generated by Claude Code