Repository navigation
docs: the serve flags the docs name, checked against the CLI (seven fixes) - #2367
Merged
Merged
Conversation
ashhart
force-pushed
the
lead/docs-flags-q30
branch
from
November 6, 2026 13:41
406bbc8 to
b75540b
Compare
Owner
Author
|
/atc exchange 99b168004a0b9ff77c3a2a521d4d6bf082d06e1d8688a4a788e62e9d36f927dc |
Owner
Author
|
Rebased onto main ( |
ashhart
force-pushed
the
lead/docs-flags-q30
branch
from
November 7, 2026 03:15
5149c17 to
68b23d7
Compare
README.md:294 said --metrics-open allows /metrics and /v1/load without a key. The gate opens three paths: /metrics, /v1/metrics and /v1/load (zig/src/server/auth.zig:260); docs/api.md:21 already listed all three. The row now names /v1/metrics too.
docs/memory-profiling.md:115-116 said the Flash Next preview serves one reply at a time even when --parallel is supplied. The rule is conditional: an explicit --parallel above one shares sessions (zig/src/families/flashnext/batch_meta.zig:62-64, wired at zig/src/native/metal.zig:394-397); the default auto and --parallel 1 keep the single-reply host. The paragraph now states the condition.
…nd Flash Next docs/api.md:798-799 said only Nemotron keeps conversation states on disk on CUDA and that other engines refuse the flag at startup. The refusal fires only for a family that is not Nemotron and declares no kept disk (zig/src/native/cuda_refusal.zig:62): Flash Next (zig/src/families/flashnext/cuda_lanes.zig:41) and the Qwen3.5 hybrids (zig/src/families/qwen3_5/cuda_native.zig:47, cuda_native_moe.zig:29) declare kept_disk, so they keep them too. The page already counted three CUDA families at docs/api.md:869; the sentence now names them.
docs/recall.md:54 said a Flash Next server refuses --recall before it loads, unconditionally. The refusal lives in the Metal host (zig/src/native/metal.zig:312-316, fired at :327 for the replay engine). On CUDA Flash Next runs the lane engine: no refusal exists in zig/src/native/cuda_refusal.zig, and the lane host attaches the record (zig/src/core/lane_host.zig:168, wired through zig/src/server/main.zig:150). The line now scopes the refusal to a Mac and names the CUDA lane engine.
docs/memory-diagnostics.md:34-36 grouped Qwen3.5-2B with the engines that omit prompt_cache_plan and said the 2B keeps no prompt states. With --prompt-cache-gib the Metal 2B plans the budget, reports prompt_cache_plan and opens a prompt cache (zig/src/native/qwen35.zig:122-141: the plan is set at :132, the store at :134-140); docs/recipes/qwen3.5-2b-native.md:47 documents the opt-in. The sentence now names GLM-5.3-Flash and the CUDA server as the omitters and states the 2B keeps states only with the flag.
ashhart
force-pushed
the
lead/docs-flags-q30
branch
from
November 7, 2026 04:35
68b23d7 to
e9304a0
Compare
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every serve flag named in the README,
docs/anddocs/recipes/, checked againstzig/src/server/cli.zig's flag table and the families' refusal layer. Seven wrong or incomplete claims are fixed across seven pages, and each commit cites the code that settles it. The--aliasline incuda.mdis fixed in #2361 instead.README.md--metrics-openopens/metricsand/v1/load/v1/metrics(server/auth.zig:260), asdocs/api.mdalready saysdocs/memory-profiling.md--parallel--parallelabove one shares sessions (families/flashnext/batch_meta.zig'sshared)docs/api.mdnative/cuda_refusal.zig:62)docs/recall.md--recallnative/metal.zig:312). On CUDA, Flash Next runs the lane engine, whose host attaches the record (core/lane_host.zig:168)docs/control.md--spill-giband--vision--visionis native on every build, and--spill-gibon Metal and CUDA (server/cli.zig:57,:117)docs/memory-diagnostics.md--prompt-cache-gib, the Metal 2B plans the budget and opens a store (native/qwen35.zig:122-141)docs/recipes/qwen3.8-27b.mdTF_PROMPT_ROWSstates only the refusal of a smaller valueoverride())The sampling, thinking, GPU, tensor-parallel, drafting, vision and speech flags came back with no wrong claims. Docs only.
An earlier head also changed
docs/recipes/xpu-arc.md's prompt cache default to 2 GiB. That was wrong: the page describes the Nemotron XPU engine, whose default is 4 GiB (native/xpu_nem.zig:161-162). The 2 GiB figure is the Qwen3.8-27B lane engine's (native/xpu.zig:112-116). It's dropped.