[ashhart]

Two changes to DeepSeek-V4.1's DSpark drafter on Metal. Neither touches the target, so every reply is unchanged: drafted still equals "draft": false.

• FP8 draft head.
  • With drafts on, the drafter's weights include an FP8 copy of the vocabulary head, made from the bf16 head at load: E4M3 codes with E8M0 scales in 32x32 blocks, the release's own weight format. That's 0.67 GB more memory, quantized in about 0.2 s (dspark_head.zig).
  • The drafter's head runs it on the FP8 row kernels; the target's head stays bf16.
  • The drafter keeps its top 64 candidates, and accepted drafts were unchanged in the runs below.
• Concurrent propose. The propose launch encodes in a concurrent encoder, with a barrier wherever a step reads the one before. A new test holds a launch byte-equal to the same launch in a serial encoder; removing one barrier makes it fail (241,076 bytes differ).
• Residency.
  • None of the drafter's buffers were in a residency set, and macOS let them go between passes. About 20 drafter command buffers a run waited 40-570 ms between commit and their first GPU instruction while it made them resident again. The target's buffers never waited in the same runs.
  • Pass.init now holds everything a pass binds in one residency set on the engine's queue: weights, the FP8 head, rings, act_quant rows, plan and working buffers, about 8 GiB. deinit releases it.
  • Holding the weights alone still left 14 waits.

Measured on an M5 Ultra (512 GB), release weights, five prose prompts of tools/bench_prompts.json at 512 tokens, one stream, with #3287's FP8 rows underneath. TF_DSV41_ROUND_PROF (#2951) was used for the timings only.

	Before	FP8 head + concurrent	+ residency set
propose GPU, ms a round	3.73	3.06	3.03
ingest wall, ms	0.5	1.86-2.61	0.50
greedy tok/s	56.4-57.5	57.1-58.5	59.7-59.9
sampled tok/s	47.4-48.1	48.6-49.8	51.6-51.7

• Accepted drafts are unchanged (greedy about 1412/2510, sampled 1296/2675), and all 10 reply hashes are identical in every arm.

• The drafter head alone at 1 / 5 / 10 / 15 rows: bf16 1339 / 1410 / 2985 / 3176 µs, FP8 647 / 958 / 2153 / 2993 µs. Those use dsv41 metal: FP8 decode rows load first and meet tile roots in one simdgroup, same bits #3287's kind d; on main alone the head takes the existing kind b.

• zig build test-dsv41-metal -Dcpu=apple_m1 and test-dsv41-metal-serve pass on this branch on main, on an M5 Max. The serve suite covers drafted vs "draft": false, concurrent vs solo and seeded repeats. check_dspark.zig checks the drafter's logits against the FP8 head. Lean: 0 new.

• Synthena (Ash's Agent)