Repository navigation
Conversation
README.md:294 said --metrics-open allows /metrics and /v1/load without a key. The gate opens three paths: /metrics, /v1/metrics and /v1/load (zig/src/server/auth.zig:260); docs/api.md:21 already listed all three. The row now names /v1/metrics too.
docs/memory-profiling.md:115-116 said the Flash Next preview serves one reply at a time even when --parallel is supplied. The rule is conditional: an explicit --parallel above one shares sessions (zig/src/families/flashnext/batch_meta.zig:62-64, wired at zig/src/native/metal.zig:394-397); the default auto and --parallel 1 keep the single-reply host. The paragraph now states the condition.
…nd Flash Next docs/api.md:798-799 said only Nemotron keeps conversation states on disk on CUDA and that other engines refuse the flag at startup. The refusal fires only for a family that is not Nemotron and declares no kept disk (zig/src/native/cuda_refusal.zig:62): Flash Next (zig/src/families/flashnext/cuda_lanes.zig:41) and the Qwen3.5 hybrids (zig/src/families/qwen3_5/cuda_native.zig:47, cuda_native_moe.zig:29) declare kept_disk, so they keep them too. The page already counted three CUDA families at docs/api.md:869; the sentence now names them.
docs/recall.md:54 said a Flash Next server refuses --recall before it loads, unconditionally. The refusal lives in the Metal host (zig/src/native/metal.zig:312-316, fired at :327 for the replay engine). On CUDA Flash Next runs the lane engine: no refusal exists in zig/src/native/cuda_refusal.zig, and the lane host attaches the record (zig/src/core/lane_host.zig:168, wired through zig/src/server/main.zig:150). The line now scopes the refusal to a Mac and names the CUDA lane engine.
docs/memory-diagnostics.md:34-36 grouped Qwen3.5-2B with the engines that omit prompt_cache_plan and said the 2B keeps no prompt states. With --prompt-cache-gib the Metal 2B plans the budget, reports prompt_cache_plan and opens a prompt cache (zig/src/native/qwen35.zig:122-141: the plan is set at :132, the store at :134-140); docs/recipes/qwen3.5-2b-native.md:47 documents the opt-in. The sentence now names GLM-5.3-Flash and the CUDA server as the omitters and states the 2B keeps states only with the flag.
…t, naming it TENSORFOLD_PREFILL_SHARE outside 0.01-1 or not a number read as the default share 1, so a share of 0 or 5 meant whole prompt steps without a word; TENSORFOLD_PREFILL_POLICY other than fcfs read as the default order; TOKENS and BUDGET text that is not a whole number read as their defaults. serve now refuses those values at start with the variable and its value, before the model loads. Each check uses its reader's own parse (lane_host_env.zig), so a value it accepts reads as before; empty values still mean unset, BUDGET 0 is still none, TOKENS still clamps to 64-16384, and TENSORFOLD_PREFILL_ORDER's other values still turn SRPT off.
Observe unfinished NCCL init and peer warm-up once per minute without a phase-held lock or driver call. Report only captured transport classes, and suggest NCCL_P2P_DISABLE=1 only for an observed P2P path with two ranks. Followers say ready to follow rank 0 rather than claiming calibration has finished. Keep phase wiring and host construction in a helper so the native adapter has room for further changes. End the observer on ready, failed host startup and normal cleanup. Host checks cover blocked observation, cadence, transport-specific advice and production readiness callbacks, with runtime controls for omitted observation, unconditional advice and missing failure cleanup. No transfer, kernel or lane behavior changes. Refs #1994.
Print peer facts before warm-up while retaining capture through warm-up and leaving the optional transfer probe at its existing point. Name the capture file in startup and minute warnings, report liveness connection successes, and correlate NCCL id handoffs with the first eight SHA-256 hex digits. Wrap the same blocking communicator init with best-effort version, visible device and bus details plus elapsed completion. Keep wire operations, existing waits and warm-up postings unchanged. Host fakes cover ordered lines, capture lifetime, delayed connections, optional-probe order and failed diagnostic queries. Refs #1994.
… zig/build/cuda.zig stays at 600 lines
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2367 #2369 #2354 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.