[ischencheng]

Serves llama.cpp deepseek41 GGUFs on the family's own model classes, text only. Part of #671 (Model Support: DeepSeek-V4.1). llama.cpp has the converter since b11870 (ggml-org/llama.cpp#31086), so "llama.cpp has no converter for it" on that line is out of date.

What it does

• Config. parse_gguf_config builds DeepseekV41Args from the KVs and goes through model_config_from_args, pulled out of parse_config, so both flows build one ModelConfig. Some files leave out engram.vocab_size and engram.compressed_vocab_size and llama.cpp reads neither, so they come from ple.head_vocab_sizes and the token map.
• Weights. iter_gguf_weights yields the names the HF loader yields. Dense tensors keep their ggml type and the routed experts load as GGUF banks. The converter stores the Engram gate as the product q_weight * k_weight, so it loads into q_weight with k_weight all ones.
• Engram. All tables sit in one per_layer_token_embd tensor. gguf_engram_tables slices it per layer and EngramDiskTable streams the rows as on the HF path, then decodes them on the GPU with ggml.dequantize or a plain cast for BF16/F16/F32. The hash takes the file's engram.token_map. Its offsets, primes and multipliers must match the KVs llama.cpp hashes with, and the table must be wide and long enough; parse_gguf_config checks both, so a mismatched file is refused before any dense weight loads.
• Small moves. embed takes the quant config because token_embd is packed, and the HF spec lists embed as unquantized so that path is unchanged. _float moved from deepseek_v4/gguf.py to gguf/tensors.py as dequantized_tensor, with the same body. The SwiGLU clamp and expert_count readers moved from deepseek_v4/gguf.py to gguf/config.py (uniform_swiglu_clamp, routed_expert_count) and both readers use them, so a scalar clamp or a per-layer expert_count array is handled the same way in both.
• Refusals. expert_gating_func other than sqrtsoftplus, hash-routed layers, rope scaling other than yarn, more than one latent KV head, non-uniform swiglu clamps, per-layer expert prunes, Engram tensors without ple.layers, a missing per_layer_token_embd, and a per_layer_token_embd.scale (check_scales lets an NVFP4 tensor's .scale through, and the table reader would not apply it).

Verification

L40S 46 GB (sm_89), driver 610.57.04, torch 2.14.1+cu130, 16 vCPUs (ggml-cpu picks zen4), weights on a network volume. The 535f894 row, the first gguf_check table and the first A10 test run are on 535f894 (main 0ad521d), before the review changes; those change where the checks run and the two helpers' place, not how weights are read. The other rows are from base 33cbb3c, before main took the llama.cpp pin (#873); the reader's weight path is unchanged since. 214 GiB RAM, 322 GiB for the two hybrid rows below. At 214 GiB the container hit its memory limit in the hybrid arm, and I haven't looked into why. Unit tests on A10 (sm_86).

ft serve --model <first shard> --max-seq-len-override 4096 --moe-strategy offload --moe-cache-rate 0.2 --moe-collect-stats --max-running-requests 16

Each row: 4 prompts streamed at bs=1, 96 tokens each, then 16 arithmetic prompts together, then 48 staggered at concurrency 16.

Checkpoint	Arm	bs=1 decode tok/s	arithmetic	staggered	GSM8K (first 32)
unsloth/DeepSeek-V4.1-Flash-1017-GGUF UD-IQ1_M	offload, --swa-decoder-replay exact	17-25	16/16	48/48
same	offload, default replay	16-25	16/16	48/48	31/32
same, 535f894	offload, default replay	24-32	16/16	48/48	30/32
same	hybrid, --moe-hybrid-max-fetch 0, exact replay, every route on the CPU	13-14	16/16	48/48
same	hybrid, --moe-cache-auto	29-37	16/16	48/48
ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF IQ1_M, Engram file linked as shard 6	offload, default replay	28-34	16/16	48/48	32/32
llama-server 29c720eaf on the UD-IQ1_M file, -ngl 999 --cpu-moe -t 16		1.1-2.3	16/16		31/32

On 535f894 GSM8K drops one. The lemon-tree (index 12) and vacuum (index 13) answers run out of their 384 tokens before the #### line in all three runs, so they score on the last number written: on 12 this run stops on the 12-year break-even it was weighing, where the earlier run and llama-server stop on 13; on 13 all three stop in a check line after reaching 18.

The config parsed from the UD-IQ1_M file matches deepseek-ai/DeepSeek-V4.1-Flash-1017's config field by field, apart from the DSpark fields the file doesn't carry and engram_vocab_size (16000057 from the file against 16000000, which derive the same primes).

Against llama-server on the same file:

• Prompt token counts match on all 4 prompts (14, 22, 33, 1360). tok_compare.py against the deepseek-ai/DeepSeek-V4.1-Flash-1017 tokenizer: 0 differing texts (13 edge, 126,527 vocab, 20,000 combos).
• The 16 arithmetic answers are the same text. All 32 GSM8K final answers are the same (offload arm, base 33cbb3c); on 535f894 30 are, and the other two are the cut-off 12 and 13 above.
• Greedy over 96 tokens, exact replay: tokens in common before the first split are 10, 96, 96, 9.
• On a 1-bit file the kernels alone move logits that much, so I compared next-token logprobs on fixed prefixes (2 prompts, llama's own tokens). On the same 24 prefixes, median RMS difference over the top 8 tokens of llama-server with -ngl 999 --cpu-moe: FreeToken 0.56 nats, same top token on 22; llama-server with -ngl 0 0.36 nats, same top token on 21. At the prefix where the first prompt splits, the -ngl 0 run picks the same token FreeToken does.
• FreeToken with GPU experts against FreeToken with every route on ggml-cpu also splits, at 85, 96, 96, 5.
• Control: with every Engram lookup zeroed, FreeToken against its own run splits at 1, 1, 28, 5, while all 16 arithmetic answers stay the same. So the prompts do see the table.

scripts/gguf_check over 9 repos, headers only (148 stubs, kernels probed on the A10). It checks config, a dry load and kernel coverage, not real weights:

Set	Result
unsloth 1017	6/9, the other 3 are under Not covered
ISTA-DASLab GSQ-RCO, Abiray	4/4 with the shard 6 link their files need, as in llama.cpp
kyuz0, audreyt, antirez, Adolphsson, Lucebox, peasantsmith	refused, see Not covered

Tests on the A10, 535f894: tests/models tests/moe tests/kernels/test_ggml.py tests/server tests/checkpoint 1365 passed, 250 skipped, 5 failed. main at 0ad521d on the same GPU: 1355 passed, 250 skipped, and the same 5 fail (qwen4_exp/test_qsa_backend.py::test_chunked_prefill_matches_one_shot x3, dflash/test_draft.py::test_block_forward_matches_reference fp32 x2). The new tests extend the existing deepseek_v41 files: a tiny checkpoint through the GGUF and HF readers gives equal prefill logits and decode logits within 2^-7 of the largest logit (the bf16 fused and ggml decode kernels round differently), the table slices decode to the source rows for BF16 and Q8_0, eager and from a captured graph, ids the file's token map merges hash to the same rows, and a wrong hash KV refuses.

After the review changes, this head merged into main 66190a9, on the A10: tests/models tests/server tests/checkpoint 1114 passed, 222 skipped, 5 failed; main 66190a9 1101 passed, 222 skipped, the same 5 fail. The new tests (a scalar clamp parses and a per-layer expert_count refuses, a table with a .scale refuses, a wrong hash KV refuses at parse_gguf_config, ft checkpoint on a GGUF leaves no output directory) fail with 535f894's sources and pass on this head. gguf_check over 41 header stubs from unsloth/DeepSeek-V4-Flash-0731-GGUF, puwaer/DeepSeek-V4-Flash-0731-reap-150b-gguf, huihui-ai/Huihui-DeepSeek-V4-Flash-0731-abliterated-GGUF, unsloth/DeepSeek-V4.1-Flash-1017-GGUF and antirez/deepseek-v4.1-flash-gguf gives the same result per file on 535f894 and on this head: deepseek4 21/21, deepseek41 6/11 (the 5 under Not covered).

Not covered

• The deepseek41v mmproj and the DSpark draft files are not read.
• unsloth's UD-Q3_K_XL, UD-Q4_K_XL and UD-Q8_K_XL refuse at the GGUF reader with 43 is not a valid GGMLQuantizationType. Their MXFP8 tables use type 43 at 33 bytes per 32 values, which only unsloth's llama.cpp PR reads. Upstream 43 is one-byte F8_E4M3.
• Files converted for earlier llama.cpp branches, or with other table types, refuse with a missing KV or the same reader error.
• --moe-strategy cpu refuses these mixed-type files, like other mixed-type GGUFs. A single-type file wasn't tested.
• ft checkpoint refuses these GGUFs before it creates the output directory (a new ftw_check_source hook that convert_checkpoint calls before FTWWriter): serve the .gguf.
• Real weights ran only on the two IQ1_M files in the table.
• Not run: AIME, TP, consumer cards, a local NVMe.

[jason-fxz]

Thanks for the PR. The reader matches llama.cpp's conversion/deepseek41.py, and I found no wrong-output problem on the published file. A few small nits:

• convert.py now runs the side-file hook before the dense loop for every checkpoint, so the HF DeepSeek-V4.1 and Qwen3.8-Flash-Next conversions copy their side files before any dense weight, and a refused GGUF leaves an empty output directory. Refusing the GGUF before FTWWriter(...) and keeping the hook after the dense loop, as on main, avoids both.
• check_scales accepts a per_layer_token_embd.scale next to an NVFP4 table, but gguf_engram_tables never reads it, so such a table would decode without its scale. Refusing it would be safer; the UD-IQ1_M file has no .scale tensor.
• A per-layer expert_count array or a scalar swiglu_clamp_exp raises TypeError here, while deepseek_v4/gguf.py handles both. Sharing those helpers under models/gguf/ would keep the two readers in step.
• The Engram KV checks in gguf_engram_tables could run in parse_gguf_config, so a mismatched file is refused before the dense weights load.

Checked on an H100 (torch 2.14.1+cu130): tests/models, tests/server and tests/checkpoint pass on this head and with it merged into main. No real weights were run here.

[ischencheng]

Thx, all four are in 86f716e. convert_checkpoint now refuses the GGUF through a new ftw_check_source hook before FTWWriter, and the side-file hook is back after the dense loop. parse_gguf_config runs the Engram KV and table checks and refuses per_layer_token_embd.scale, and the clamp and expert_count readers moved to gguf/config.py for both readers; each change has a test that fails on 535f894.