Repository navigation
Scope ContextFull admission backoff to the refused request - #55
Draft
beastllama wants to merge 1 commit into
Draft
beastllama wants to merge 1 commit into
beastllama wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause: a host-wide
ContextFullretry hold prevents later queued work from reaching admission even when it can fit. Evidence: the CPU regression one100ee8leaves a fitting warm request unadmitted (expected 4, found nullcached tokens).Move the occupancy threshold and retry deadline onto the refused job. The existing admission picker skips jobs whose backoff is still active, preserving their queue position so an eligible head regains priority when its deadline arrives or occupancy falls. Existing pool-fit fairness, single-fill, cache ownership, and near-completion gates remain in place.
The new synchronous CPU fixtures cover fitting warm admission, retention and eventual admission of the refused job, bounded retries, deadline priority, retry after a partial occupancy decrease, and cancellation of only the waiting job. All 61 lane-host tests pass with Zig 0.17.0 in ReleaseSafe; the engine API semantic compile-only check, formatting, and diff checks also pass.
Validation limits: the broad engine API suite panics in the Imprint allocation-failure test; the same focused test also panics on the unmodified pinned base. The native build is blocked in this WSL environment by
checking cache failed: manifest_stat Unexpected, before a successful server build. GPU and end-to-end inference validation are pending. Please treat this as a draft until those platform checks are available.Related: #51 addresses fill/drain scheduling. This change addresses the separate host-wide
ContextFullretry hold.Reproduce the focused CPU checks: