Repository navigation
Conversation
… metrics (#2135) Each queued request holds one wait reason, decided where the host last tried to admit it (pick, choose, a family planner's wait or skip, takeJob's ledger promise) and cleared on admission; publishing the ledger only counts the stored reasons and never asks the family's pool. The planner's Verdict carries the reason with its wait or skip, and DeepSeek-V4.1 names kv_pool, memory_floor or prefill_concurrent from its own counters, so the floor is never reported as memory_pressure. /metrics gains requests_waiting_by_reason{reason} (all ten labels, summing to requests_waiting), request_failures_total{reason}, restore_fallbacks_total and request_refusals_total{reason}; /v1/load gains waiting_by_reason and refusals. Refusals are counted where the chat, embeddings, rerank and speech routes answer with a client code, and every reply's bytes are unchanged. Shape B as designed by devopsfortunato on #2196. Co-authored-by: devopsfortunato <57877156+devopsfortunato@users.noreply.github.com>
…est, DeepSeek's mapping tested on the host (#2135) request_refusals_total gains the admission flags' 429s under the code each reply carries (max_inflight_exceeded, max_inflight_per_key_exceeded, admission_capacity_exceeded, requests_per_second_exceeded), counted once at the request gate, and an images engine's busy 503 as engine_busy. request_failures_total is now always shown, the server's count beside the host's: the chat routes' 500 for a stale device is engine_stale, and a family planner's refusal, a failed reply with its own words and no code, is planner_refused. A begin that reports its saved entry damaged and the pass that then reports the failed restore count one fallback: each request carries whether its fallback is counted. DeepSeek-V4.1's verdict mapping moves from sched_host.zig into sched_verdict.zig, which a new host step (test-dsv41-wait-reasons-host, in both nightly DeepSeek lists) drives through a real lane host round into the ledger. The docs say the stored reasons are as of the last admission attempt. Every reply's bytes are unchanged.
ashhart
force-pushed
the
lead/wait-reasons
branch
from
November 13, 2026 15:27
448734d to
f77700d
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For #2135. This builds the shape @devopsfortunato proposed on #2196 (shape B, agreed there on 9 Nov), with them as co-author.
What changes (reply bytes are unchanged throughout):
Rounds.Verdict.waitcarries the reason). The lane host stores it on the queued job when it is queued or re-examined and clears it on admission.waitReason()becomes a read of the stored reason, so the load ledger's publish path no longer callsadmit.fitsfor every queued job inside the host lock.kv_pool,memory_floorandprefill_concurrentcome from its planner (families/deepseek_v41/sched_verdict.zig) instead of all reading asengine_busy.requests_waiting_by_reason{reason=...}with every label present at 0;request_refusals_total{reason=...}for what a client sees as a refusal (the 503 codes, the named 400s, and the--admission429s under the code each reply carries);request_failures_total{reason=...}for failures with their own bodies (including chat's stale state asengine_staleand a planner's own refusal asplanner_refused);restore_fallbacks_totalonce per request./v1/load's scheduler section gainswaiting_by_reasonandrefusals.requests_waiting, asserted where the ledger is read.docs/api.mddocuments the new keys and says which names are client codes and which are internal reasons; the reasons are as of the last admission attempt.Tests (each fails without its change):
test-wait-reasons-hosttest-wait-reasons-server-hosttest-dsv41-wait-reasons-host(new: a stand-in planner through a real lane-host round)test-server-cputest-goldenRemoving each new count (and reverting the fallback flag and the DeepSeek mapping) fails all 7 new tests, each on its own number.
zig fmtclean; lean 0 new.Not counted yet, and listed so they aren't forgotten: images' Closed/Unsupported 503s and its InvalidRequest 400, System One's 503s, slide's and imprint's 503s, the gate's drain and failed-wake 503s, and connection-level admission 503s. A follow-up names each.