Skip to content

Scope ContextFull admission backoff to the refused request - #55

Draft
beastllama wants to merge 1 commit into
bertholomus:deepseek-v41-zig-tp2from
beastllama:fix/contextfull-job-backoff
Draft

beastllama wants to merge 1 commit into
bertholomus:deepseek-v41-zig-tp2from
beastllama:fix/contextfull-job-backoff

Conversation

@beastllama

Copy link
Copy Markdown

Root cause: a host-wide ContextFull retry hold prevents later queued work from reaching admission even when it can fit. Evidence: the CPU regression on e100ee8 leaves a fitting warm request unadmitted (expected 4, found null cached tokens).

Move the occupancy threshold and retry deadline onto the refused job. The existing admission picker skips jobs whose backoff is still active, preserving their queue position so an eligible head regains priority when its deadline arrives or occupancy falls. Existing pool-fit fairness, single-fill, cache ownership, and near-completion gates remain in place.

The new synchronous CPU fixtures cover fitting warm admission, retention and eventual admission of the refused job, bounded retries, deadline priority, retry after a partial occupancy decrease, and cancellation of only the waiting job. All 61 lane-host tests pass with Zig 0.17.0 in ReleaseSafe; the engine API semantic compile-only check, formatting, and diff checks also pass.

Validation limits: the broad engine API suite panics in the Imprint allocation-failure test; the same focused test also panics on the unmodified pinned base. The native build is blocked in this WSL environment by checking cache failed: manifest_stat Unexpected, before a successful server build. GPU and end-to-end inference validation are pending. Please treat this as a draft until those platform checks are available.

Related: #51 addresses fill/drain scheduling. This change addresses the separate host-wide ContextFull retry hold.

Reproduce the focused CPU checks:

zig test -lc -O ReleaseSafe --dep lanes \
  -Mroot=zig/src/core/engine_api.zig \
  -Mlanes=zig/src/core/lanes/lanes.zig --test-filter lane_host
zig test -lc -O ReleaseSafe -fno-emit-bin --dep lanes \
  -Mroot=zig/src/core/engine_api.zig \
  -Mlanes=zig/src/core/lanes/lanes.zig

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant