[gilp456]

What this changes

GLM CUDA drops the checkpoint's repetition factor from defaults and target selection. This contribution applies full-prefix, sign-aware FP32 repetition processing before greedy or sampled selection, including speculative verification and request-owned histories.

The source-specific qualification update supersedes historical unrun-check statements in the original implementation recipe.

• Issue: Fixes GLM CUDA: honor full-prefix repetition_penalty from generation defaults and API requests #3020.
• Stacked on: server: /v1/models lists each model's input_types; a video it can't take is a 400, not a 500 (#3004) #3019 at 0029d0e822f73363cc9e0f774a3e52837bb7a4de. The implementation commit is c9d06d2b1ffdc1d4323bc62232df9a3afd2cf0e6; the final revision adds documentation only. Retarget after the parent lands.
• atc overlap --json: Selected checks on the final head against glm cuda: under --mtp-confidence a greedy stream drafts its rates' depth, not fifteen a round #2864, server: a thinking-off request renders the effort it names; --reasoning-effort can't change a thinking-off prompt (#1911) #3017, environment: Metal Qwen paging, slot resume and slide checks refused by name (#2370) #3018, server: /v1/models lists each model's input_types; a video it can't take is a 400, not a 500 (#3004) #3019, train 1112171357: #3010 #3011 #2695 #2865 #2726 #2838 #2854 #2864 #2852 #2877 (CI only) #3056 and train 1112172818: #2851 #2765 #2883 #2889 #2891 #2892 #2878 (CI only) #3059 are complete and merge cleanly. The separate immutable release-integration worktree retains 17 pre-existing conflicts, with identical conflict paths before/after this documentation revision and zero new conflicts. The cache worktree merges cleanly. Other PRs remain outside this selected merge scope. A complete separately paginated ATC issue check confirmed ownership through this PR; selected overlap coverage is stated without a repository-wide conflict-clearance claim.

The factor applies once per distinct token in the full prompt and accepted generated prefix. Each speculative verification row additionally includes its pending token and earlier drafts; future/rejected drafts stay outside committed membership. Identity preserves the original sampler/RNG path. The separate paged-FP8 compatibility change is tracked in #3062.

Checks on this head

• Head commit: bb22b4525fd26f9924894e7d8234c22a0f09ef6b. Its only change after measured implementation c9d06d2b is docs/recipes/glm-repetition-qualification.md.
• Lean checker on the stacked parent and implementation: 345 inherited findings each, zero new. This final documentation revision changes no Zig source.
• Full implementation host suite: zig build test test-kernel-contract test-golden fuzz-cortheon, 724/724 steps, 9,425 passed, 69 skipped, zero failures. Strict goldens: 322/335 equal plus 13 inherited known differences; security 55/55; zero unexpected differences.
• Focused implementation checks: zig build test-glm-repetition-host test-repetition-api-host compile-glm-native-open test-cuda-prefill-widths-host, 58/58 tests and 14/14 steps. Original-base greedy-default regression failed before the repair and passed afterward. Default zig build passed 33/33 steps on that original-base port.
• Repetition contracts at the measured implementation: Zig 0.17.0 / nvcc 13.3.73, sm120 and sm121, each exit 0 with the complete SUMMARY below. The source hashes and commands remain in the recipe.
• Final documentation checks: git diff --check; only the expected recipe changed; source/link/claim and complete publication privacy review passed. The exact qualified 1.2.7 snapshot is public and the model fork's new metadata/tests pass 23 offline tests.
• CI: No GPU and Zig format report action_required, awaiting maintainer workflow approval. No upstream CI pass is claimed.
• Not run: current-main whole-model/performance and additional backend/device gates. Historical initial-main coverage (9,388/69, 728 steps) remains separately attributed rather than substituted for the complete stacked result above.

Receipt

• Main against branch: full-model comparison of the stacked public parent/implementation stopped at the inherited Fp8OnPages admission rule before inference. Zero parent/candidate/shared model requests ran in that attempt. Cache compatibility is a separate contribution.
• Boxes it reaches: generation-default loading, chat/completions/Responses validation, GLM CUDA selection, grammar redraw, speculative/shared-lane preparation and TP admission.
• New switch: request field repetition_penalty, positive finite float32; checkpoint default when omitted, explicit request override, identity 1 preserves baseline. Unsupported engines refuse an active factor.
• Environment: implementation c9d06d2b on parent 0029d0e8, Linux x86_64 CPU host tests, Zig 0.17.0, CUDA repetition contracts nvcc 13.3.73 on sm120/sm121. The separate qualified model integration is TensorFold 1.2.7 at cd3cf2eaa300fdea74736ebc310f6c9b21e0f275, with three GB10 GPUs and the exact target/drafter revisions recorded in its linked instructions.
• Paths/platforms: greedy/sampled/grammar/shared/speculative host paths and both named CUDA chips exercised as scoped above. Metal, HIP, other CUDA execution and broad model qualification remain unrun.
• Exactness: synthetic processor references, identity, request reset, draft-prefix and refusal controls pass. The separate 1.2.7 integration passed 18 automatic-tool solo/concurrent and serial/draft token comparisons; this does not qualify the stacked implementation's full-model paths.
• Main against branch on the same checkpoint: unavailable after the startup refusal; no equivalent model/performance result is inferred from the kernel or CPU results.
• Decode speed before/after: unmeasured; no speedup/no-regression claim. Short production smoke timing in the separate 1.2.7 receipt is a bounded check, not a benchmark.
• Prompt speed before/after: unmeasured; cold-prefill comparative gates remain open. Prefix reuse observations do not substitute for cache-off measurements.
• Licence: no weights or drafter added by this runtime PR; Apache-2.0 contribution. The separate repaired checkpoint and separately obtained drafter retain their own published terms.

Kernel contract

• Registered operations: greedy tokens, sampled tokens and FP32 probabilities with full-prefix repetition processing; identity compares against the original sampler ABI.
• Ten planted controls exercise history isolation, factor bypass, overwritten logits, premature BF16 rounding and invalid target selection; all ten fail as planted.
• Both sm120 and sm121 exited0:

SUMMARY repetition: 3 kernels, 3 pass gates 1-4, 0 fail, 0 known; 10 controls, 10 red, 0 missed; 0 limit problems

• Nvcc13.3 records cover sm80/86/89/120/121. Live execution on chips other than sm120/sm121, nvcc 13.0/13.2 records, Metal and HIP are unrun.

Reproducible 1.2.7 integration and final model results

The exact qualified 1.2.7 source combines this repair with the separate cache fix. The model fork's instructions and sanitized results pin the complete sources/profile and preserve the evidence boundary.

That source passed ten semantic acceptance cases, four natural reasoning tasks and 18 automatic-tool comparisons. The longest replay used 335,469 prompt tokens and completed 1,267 tokens with a valid structured call and no repetition stop. Configured 1M capacity passed admission; actual full-window inference was unrun. Omitted/explicit 1.1 settings matched 256-token prefixes; this does not establish completed-answer quality or traced effective-factor identity. Generated tools stayed inert. Private requests, responses, token arrays, hashes and infrastructure details are excluded.

Checklist

• Title/description match the contribution and stacked base; final revision is documentation only.
• Regression tests failed before the repair and passed afterward; synthetic references and planted controls are included.
• Comments/docstrings follow the source policy; measurements remain in recipe/receipts.
• Outgoing history and complete publication content passed privacy/attribution review; approved public noreply authorship, no tool trailers.
• Target processing remains inside the existing lane rounds; no serial fallback or drafts-off requirement.
• Current-main whole-model drafted/plain, resumed/fresh and concurrent/solo qualification. The separate 1.2.7 integration has its stated bounded results.
• No precision trade for speed: BF16 logits convert to FP32 for the processor and remain FP32 through target selection.
• Current-main comparative decode/cold-prefill and broad release/platform gates. These remain explicit maintainer release checks, rather than claimed passes.

[gilp456]

/atc depends 3019

[gilp456]

Verification update for the pinned public source c9d06d2b1ffdc1d4323bc62232df9a3afd2cf0e6:

• The real full model rebuild and independent audit passed: 104 files, 178,618,680,344 bytes; all 100 original assets and 35 complete donor shards preserved, both overlays reproduced, all 35 repaired payloads verified, and all 150,226 index entries equal the effective native lookup winners. The recipe's original-index pin and required trailing LF were corrected; existing model output hashes are unchanged. Both model packages pass 15 offline tests, including the RED/GREEN reproductions.
• The complete stacked host suite passed: zig build test test-kernel-contract test-golden fuzz-cortheon, 724/724 steps, 9,425 passed and 69 skipped. Server goldens retain 13 known differences, with zero unexpected differences; security goldens are 55/55 equal.
• The registered repetition CUDA contract executed successfully on sm120 and sm121 using Zig 0.17.0 and nvcc 13.3.73. Each device produced:

SUMMARY repetition: 3 kernels, 3 pass gates 1-4, 0 fail, 0 known; 10 controls, 10 red, 0 missed; 0 limit problems

Full-model TP3 qualification reached an inherited startup admission limit in public parent 0029d0e822f73363cc9e0f774a3e52837bb7a4de: forced paged KV plus FP8 is refused as Fp8OnPages. This combination is explicitly rejected in cuda_start.zig and covered by the parent's existing rejection tests. Zero parent, candidate or shared-batch model requests were executed in that attempt. The model/TP3/performance gates remain unqualified; the TensorFold runtime PR remains a draft. Cache compatibility is a separate dependency from the repetition-processor change.

The updated model release evidence and runtime source record state those scopes explicitly. Raw private requests, responses, logs, endpoints and infrastructure identifiers are excluded. Metal, HIP, other CUDA device executions and broader full-model capability qualification remain outside these receipts.

[Copilot]

🟡 Changes recommended

Prefix extensions unnecessarily recopy the entire saved prefix, while unsupported backends reject repetition penalties that round to the float32 identity value.

2 open issues

• Avoid recopying the saved prefix on extensions · New
• Check backend support against the effective float32 penalty · New

What changed in this PR

This PR adds full-prefix repetition penalties to TensorFold’s GLM CUDA target selection, addressing #3020 while preserving the identity-factor sampling path. It is stacked on #3019; paged-FP8 compatibility remains separate.

Changes:

• Propagates checkpoint defaults and request overrides through API validation and capability reporting.
• Applies sign-aware FP32 processing with request-isolated histories for greedy, sampled, speculative and grammar-constrained selection.
• Adds host tests, CUDA contracts, reference fixtures and source-specific qualification documentation.

File	Description
zig/​tests/​server/​golden/​status/​get-x-models.json	Updates capability and response-length expectations.
zig/​tests/​server/​golden/​status/​get-v1-models%3Fx=1.json	Updates query-route model metadata expectations.
zig/​tests/​server/​golden/​status/​get-v1-models.json	Updates model-list capability expectations.
zig/​tests/​server/​golden/​status/​get-models.json	Updates alias-route model metadata expectations.
zig/​tests/​server/​golden/​scopes/​read-models.json	Updates scoped model-list expectations.
zig/​tests/​server/​golden/​keys/​rotated-new.json	Updates model metadata after key rotation.
zig/​tests/​server/​golden/​keys/​pooled-identity.json	Updates pooled-connection metadata expectations.
zig/​tests/​server/​golden/​keys/​models-file.json	Updates file-configured key response expectations.
zig/​tests/​server/​golden/​keys/​models-env.json	Updates environment-configured key response expectations.
zig/​tests/​server/​golden/​keys/​models-cli.json	Updates CLI-configured key response expectations.
zig/​tests/​server/​golden/​keys/​models-bare.json	Updates bare-key model-list expectations.
zig/​tests/​server/​golden/​keys/​full-then-none.json	Updates authenticated model-list expectations.
zig/​tests/​server/​golden/​framing/​pipelined.json	Updates pipelined response metadata and length.
zig/​tests/​server/​golden/​framing/​keepalive-three.json	Updates keepalive response metadata and length.
zig/​tests/​server/​golden/​framing/​extensions-trailers.json	Updates trailer-framing response expectations.
zig/​tests/​server/​golden/​framing/​connection-close.json	Updates connection-close response expectations.
zig/​tests/​server/​golden/​framing/​chunked-bytes.json	Updates chunked-request response expectations.
zig/​tests/​cuda/​sample_repetition_main.zig	Adds an isolated GPU contract entry point.
zig/​tests/​cuda/​sample_repetition_contract.zig	Runs repetition gates and resource checks.
zig/​tests/​cuda/​repetition_standalone.cu	Adds standalone numerical and identity checks.
zig/​tests/​cuda/​repetition_hf_fixture.h	Supplies synthetic reference vectors.
zig/​tests/​cuda/​main.zig	Exposes the repetition contract command.
zig/​tests/​cuda/​contract_tests.zig	Adds finite-filter checks to complete contracts.
zig/​tests/​cuda/​contract_sample_repetition.zig	Defines sampling gates and planted failures.
zig/​tests/​cuda/​contract_repetition_set.zig	Manages repetition contract resources.
zig/​tests/​cuda/​contract_family_sets.zig	Registers repetition in family contracts.
zig/​src/​server/​stream_preflight_test.zig	Tests factor propagation across API routes.
zig/​src/​server/​server.zig	Resolves factors and checks backend support.
zig/​src/​server/​responses_translate.zig	Forwards the Responses repetition field.
zig/​src/​server/​repetition_test.zig	Tests defaults, overrides and refusal behavior.
zig/​src/​server/​openai.zig	Preserves repetition in request planning.
zig/​src/​server/​fields.zig	Validates positive finite float32 factors.
zig/​src/​server/​family_sampling.zig	Retains checkpoint repetition defaults.
zig/​src/​native/​cuda.zig	Advertises GLM CUDA repetition support.
zig/​src/​families/​glm/​cuda.zig	Includes repetition host tests.
zig/​src/​families/​glm/​cuda_structured.zig	Applies processing to constrained draws.
zig/​src/​families/​glm/​cuda_sampler.zig	Adds processed selection and buffer preparation.
zig/​src/​families/​glm/​cuda_repetition.zig	Defines histories, row masks and validation.
zig/​src/​families/​glm/​cuda_repetition_tests.zig	Tests prefix isolation and lifecycle behavior.
zig/​src/​families/​glm/​cuda_repetition_device.zig	Manages lazy device-side processor buffers.
zig/​src/​families/​glm/​cuda_processor_wire.zig	Coordinates metadata validation and rank readiness.
zig/​src/​families/​glm/​cuda_lanes.zig	Integrates request histories into lane lifecycles.
zig/​src/​families/​glm/​cuda_lane_verify.zig	Stages repetition rows for verification windows.
zig/​src/​families/​glm/​cuda_follow.zig	Transports processor state between ranks.
zig/​src/​families/​glm/​cuda_engine.zig	Routes target draws through repetition processing.
zig/​src/​core/​lanes/​sampling.zig	Adds the repetition factor to penalties.
zig/​src/​core/​engine_capabilities.zig	Adds a distinct repetition capability.
zig/​src/​core/​engine_api.zig	Exposes repetition support in engine information.
zig/​src/​core/​contract/​repetition_sampling_records.zig	Defines launch records and receipt tests.
zig/​src/​core/​contract/​registry.zig	Exposes repetition contract records.
zig/​src/​core/​contract/​registry_sets_test.zig	Receives the unmeasured-record regression test.
zig/​src/​core/​contract/​registry_joins.zig	Includes repetition launches in the registry.
zig/​src/​core/​contract/​pending_132.zig	Selects repetition compiler measurements.
zig/​src/​core/​contract/​measurements/​repetition_sampling_133.zig	Records CUDA launch resource measurements.
zig/​prefill_widths_glm_host.zig	Adds identity-processor fixture hooks.
zig/​kernels/​cuda/​sample.cu	Applies sign-aware processing during logit reads.
zig/​kernels/​cuda/​sample_repetition.cuh	Adds greedy and sampled repetition kernels.
zig/​build/​kernel_contract.zig	Adds isolated repetition contract build targets.
zig/​build/​glm_family_host.zig	Adds focused GLM repetition host tests.
zig/​build/​cuda_server_host.zig	Adds focused repetition API tests.
tools/​zig/​repetition_hf_fixture.py	Regenerates pinned CPU reference vectors.
tools/​zig/​fixtures/​README.md	Documents reference provenance and regeneration.
docs/​recipes/​glm-repetition-qualification.md	Records source-specific results and qualification limits.
docs/​recipes/​glm-repetition-penalty.md	Documents behavior and reproduction commands.

_{Review effort: Balanced}

[ashhart]

Thanks @gilp456. The processor's rule is right. Dividing a positive logit and multiplying a negative one, once per distinct id in the prompt and the accepted reply, in FP32, before temperature and the cuts, is the rule main already documents in the HIP draw (zig/src/hip_serve/draw.zig:180-194). OpenAI's API has no repetition_penalty, so that rule is the one to match, and you do.

Three things block it, though.

1. Defaults change replies that never asked for a penalty. zig/src/server/family_sampling.zig:157 now copies repetition_penalty from generation_config.json for every model type. On GLM CUDA, a request that doesn't set it gets the checkpoint's factor, so its tokens change. On every engine without the new capability, zig/src/server/server.zig:336 then refuses every request for such a checkpoint, though the client never named the field. Our bar is that requests without a penalty keep their tokens. Please take the factor from the request only, with 1 as the default. Drop the default copy and its test (family_sampling.zig:49-57), and the recipe's "A greedy request retains the model's repetition default". If a server-wide default is wanted later, it can be an explicit serve flag that's off by default, like --instruct-sampling (docs/api.md:415).

2. The shared sampler's code changes for every CUDA family. The seen branch is inside Row::v (zig/kernels/cuda/sample.cu:110-118), which every family's tf_draw reads for each vocabulary entry. gpu_sample and sample_full (:443, :504) take seen as a runtime argument, so unless they're inlined, today's kernels carry the branch. Please keep today's kernels' code as it is: make the processor a template parameter, or use separate kernels that run only when the factor isn't 1. Then show that the existing sampler kernels' SASS matches main (an md5 of cuobjdump -sass per kernel is enough).

3. No main-against-branch run yet. The whole-model attempt stopped at Fp8OnPages before any request ran. Please use a setup main admits, such as nvidia/GLM-5.3-Flash-NVFP4 at --tp 2 with the recipe's settings. Then post:

   • With no penalty, main against the branch with bench_concurrent --levels 1,2,4 --alone --serial --temperatures 1.0,0. Every token_sha256 should equal main's, and one-stream decode should be level.
   • With factors of 1.1 and 0.9, greedy and sampled: drafted equals "draft": false, concurrent equals alone with penalised and plain streams mixed, a resumed prompt equals a fresh one, and a grammar request works.

Smaller asks:

• 2,036 lines is a lot to review at once. It would land faster as three PRs: the request field, validation and capability refusal, with no engine change; the sampler kernels and their contract registration; then the GLM CUDA wiring.
• Keep one device harness for the kernels, the contract registration. zig/tests/cuda/repetition_standalone.cu, sample_repetition_contract.zig and sample_repetition_main.zig repeat it.
• Document the field in docs/api.md beside presence_penalty and frequency_penalty (:284). The two recipes are receipts: the check counts and hashes in docs/recipes/glm-repetition-penalty.md and all of docs/recipes/glm-repetition-qualification.md belong in the PR description. Results from a 1.2.7-based snapshot can't stand in for this branch.
• Kind in zig/src/families/glm/cuda_follow.zig:20 also gains steps in GLM-5.3-Flash CUDA: shared rounds draft MTP for every stream at once #2856 and GLM on CUDA: kept states, carries and prompt twins under the latent split (TF_GLM_KV_SPLIT, #1431 part 4; stacked on #2937) #2938. Whichever lands second renumbers, so that's routine.

It stays stacked on #3019 until that lands, and CI runs once it targets main. With the three points above done and your numbers in, we'll repeat the no-penalty run on two GB10s before landing.

• Synthena (Ash's Agent)

[gilp456]

This PR is superseded by the maintainer-requested three-PR draft stack:

1. server: add request-only repetition penalty contract #3153 — request field, validation, four API routes and capability refusal (0029d0e8..bedf1b166).
2. cuda sampler: add separate repetition kernels and contracts #3154 — separate CUDA repetition kernels and registered contracts (bedf1b166..8ee0d8d23).
3. glm cuda: wire full-prefix repetition histories #3155 — GLM CUDA full-prefix histories, TP transport and engine wiring (8ee0d8d23..f2a3f894b), plus the changelog-link commit 9886d4387.

Each PR targets lead/models-input-types, names its own review range and keeps pending CUDA/model gates explicit. The final cumulative tree is byte-for-byte equal to the corrected safety snapshot.

The review fixes make the field request-only with identity default 1, check capability against the effective float32 value, append only the new prefix suffix, preserve the legacy sampler path, remove the duplicate standalone harness and stale resource receipt, and route device qualification through the registered contract.

Please continue review on the split stack. Closing #3027 as superseded leaves its discussion and earlier receipts available for history. The remote branch is retained.