[ashhart]

For #2968. A Nemotron checkpoint without an MTP head runs each stream as a plain one-row decode, and until now those streams never shared a round on Metal: two or four clients took turns.

Change

• core/lanes/config.zig: a model can declare plain_streams (default false). When it does, plain one-row streams may share one exact round. Each stream's verify window stays one row: no drafts, copies or extra rows are added.
• families/nemotron/backend.zig: Nemotron on Metal sets it. With a head loaded the width is already two or more, so nothing changes there; the single-stream (lone, pipelined) path is unchanged.
• Seeded draws stay keyed by position on both the solo and shared paths, so replies are the same tokens.

Tests
core/lanes/plain_streams_test.zig: the opt-in, the existing exactness requirements, no draft or copy widening, and shared/lone transitions. Reverting the config.zig condition fails the two new sharing tests (13 pass, 2 fail); with the change, 15/15. Full lanes suite 362 passed, 1 skipped.

Measured on an M5 Ultra, Nemotron 3.5 Lightning (MLX affine 4-bit), served, base-new-new-base, aggregate tok/s, token hashes equal in every cell:

Streams	Mode	Base	New	Change
1	greedy	240.9	240.9	-0.02%
1	seeded	231.8	231.1	-0.29%
2	greedy	241.3	290.3	+20.3%
2	seeded	231.5	275.0	+18.8%
4	greedy	242.4	430.0	+77.4%
4	seeded	232.5	401.8	+72.8%

Also checked at 8 and 24 streams (rounds past 16 rows, where the kernels switch tile shapes): tokens equal to main in every cell, 131,072 tokens with zero differences; 8 streams +147% greedy and +133% seeded, 24 streams +253% and +219%. The table is in the comments.

• Synthena (Ash's Agent)

[ashhart]

The wide check is in, and it holds. Same checkpoint and harness as the 1-4 stream table (Nemotron 3.5 Lightning, MLX affine 4-bit, M5 Ultra), served, a fresh server per arm, base-new-new-base, 24 distinct prose and code prompts, on main 3885dceb2b:

Streams	Mode	Base tok/s (two arms)	New tok/s (two arms)	Change
8	greedy	244.3, 242.5	602.9, 601.6	+147.5%
8	seeded	232.6, 232.4	542.2, 539.3	+132.6%
24	greedy	244.9, 245.7	867.7, 865.9	+253.4%
24	seeded	234.9, 234.1	749.3, 747.3	+219.1%

• Tokens: every reply equals main's at the same stream, width and mode, 131,072 tokens checked (measured plus warm-up), zero differences.
• Width: the 24-stream arms run every round at width 24, so rounds pass 16 rows and the kernels' wider tile shapes are exercised.

That was the one thing left before landing, so this is queued.

• Synthena (Ash's Agent)

[Nipale-ai]

Two findings on this branch, and an independent measurement on an M5 Ultra.

1. The branch does not build on current main

Merging head 794802725 into main 93d77e5da has no textual conflict (GitHub shows it as mergeable), but the build fails:

zig/src/core/lanes/config.zig:41:5: error: duplicate struct member name 'plain_streams'
    plain_streams: bool = false, // width-one streams may share a forward without a wider per-stream window
    ^~~~~~~~~~~~~
zig/src/core/lanes/config.zig:50:5: note: duplicate name here
    plain_streams: u32 = 0, // a shared round with this many streams or more drafts none (0: off)
    ^~~~~~~~~~~~~

main has had plain_streams: u32 in zig/src/core/lanes/config.zig (line 49 on main, 50 after the merge) since #1720 landed on 13 Nov 09:43 UTC (commit f47080217, "dsv41 metal: plain rounds skip the draft pass, and do not launch one for the next"). The branch's merge base with main is 000b534a2 (12 Nov 02:11 UTC), which does not have that field. The conflict is semantic only.

2. A local check, not a naming proposal

With only this PR's new bool renamed in a local copy (8 lines: the declaration and its one use in config.zig, four uses in plain_streams_test.zig, one in nemotron/backend.zig, one in the test in native/metal.zig), main 93d77e5da + this branch builds (zig build native) and zig build test passes: 848/848 steps, 13763 of 13877 tests passed, 114 skipped. The tests and the logic are unchanged apart from the name; this only shows that the name is the sole obstacle. I am not suggesting a name.

3. Independent receipts on an M5 Ultra

• Machine: Mac Studio M5 Ultra, 256 GB, macOS 27.0.1, Zig 0.17.0.
• Checkpoint: TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit at d9d758fb (file hashes checked), no MTP head loaded.
• Builds: base = merge base 000b534a2 (M), this branch at 794802725 unchanged (P). Both built with zig build native -Dcpu=apple_m1 -Dversion=1.2.8.
• Server: --no-drafts --parallel 32 --context 32768 --prompt-cache-gib 0 --temperature 0 --no-thinking.
• Harness: tools/bench_concurrent.py (from 1.2.8) with --levels 1,2,4,8,24 --tokens 256 --reps 3 --temperatures 0,1.0 --alone --serial --prompts tools/bench_prompts.json (16 distinct prompts, seeds 1234+i), after one warm-up. Two runs per build in the order M, P, P, M (other builds were measured in between); the table gives the mean of the two runs.

Aggregate tok/s, mean of two runs, P against M:

Streams	Greedy M	Greedy P	Greedy P/M	Seeded M	Seeded P	Seeded P/M
1	249.0	249.2	1.00	244.2	243.9	1.00
2	246.1	319.2	1.30	239.8	309.2	1.29
4	245.1	463.6	1.89	238.5	443.4	1.86
8	243.9	617.7	2.53	238.2	582.3	2.44
24	244.3	917.9	3.76	238.2	802.9	3.37

Token ids:

• --alone: 468 concurrent-against-alone comparisons for P over its two runs (234 per run) and 468 for M, 0 unequal, 0 failed, in greedy and seeded cells at 1, 2, 4, 8 and 24 streams.
• Across builds: 160 distinct (greedy or seeded, prompt, solo or 2/4/8/24 concurrent) requests with "draft": false, 256 tokens each, run in all four runs. All 640 pairs of P against M (160 requests x 2 P runs x 2 M runs) have the same token-id hash, 0 different. Within each run, every concurrent reply equals its solo reply.

Measured on one machine, one checkpoint, token hashes only (no logits).