Skip to content

glm cuda: wire full-prefix repetition histories - #3155

Draft
gilp456 wants to merge 5 commits into
ashhart:lead/models-input-typesfrom
gilp456:codex/glm-repetition-glm
Draft

gilp456 wants to merge 5 commits into
ashhart:lead/models-input-typesfrom
gilp456:codex/glm-repetition-glm

Conversation

@gilp456

@gilp456 gilp456 commented Nov 13, 2026 •

Copy link
Copy Markdown

What this changes

Wires the request contract and repetition-only kernels into GLM CUDA with request-owned full-prefix histories. Each row includes the committed prompt/reply and accepted verification ancestry; prefix extensions append only their new suffix, while shortening or replacement rebuilds membership from authoritative IDs.

Stack order: #3153 (request/API), #3154 (kernels/contracts), then this GLM engine layer. All three drafts target lead/models-input-types and identify their own review ranges.

Checks on this head

  • Head commit: 9886d4387; implementation commit f2a3f894b.
  • Lean checker: 345 findings on the target base and head, zero new findings.
  • Tests run:
    • zig build test-glm-repetition-host compile-glm-native-open --summary all: 7/7 steps, 23/23 tests passed.
    • zig build test test-kernel-contract test-golden fuzz-cortheon --summary all: 679/679 steps; 8,699 passed, 222 skipped, zero failed.
    • zig fmt --check build.zig zig, git diff --check, and tools/pr_check.py: pass; 65 changed files cumulatively and zero new lean findings.
  • Parent-layer checks: API receipt is in server: add request-only repetition penalty contract #3153; kernel receipt is in cuda sampler: add separate repetition kernels and contracts #3154.
  • CI: pending on this draft head.
  • Not run, and why: current-head CUDA/SASS/resource qualification and TP2 whole-model exactness/performance require an authorized CUDA/model window.
  • Plain zig build: the exact target base and earlier cumulative heads each reach 26/29 steps, then tf-qwen35-cuda hits the same seven Windows std.c/fabric-network errors in unchanged code.
  • Cluster activity: zero SSH, deployment, service, Docker or model-process commands.

Receipt

  • Main against branch: pending current-head TP2 model comparison.
  • Boxes it reaches: GLM CUDA greedy, sampled, grammar redraw, speculative verification, request history, follower transport, refusal/retry and lane lifecycle.
  • New switch? Default, accepted values, profile row: server: add request-only repetition penalty contract #3153 adds a request-only field; omitted and null use identity 1; no checkpoint or server-wide default.
  • Environment: Windows host, Zig 0.17.0; host GLM and native-open checks above; CUDA compiler/device/checkpoint unavailable.
  • Paths and platforms the change reaches, which I ran and which I could not: GLM host state/transport and native-open compilation ran; CUDA SM80/86/89/120/121 and model execution remain pending.
  • Exactness: host tests cover full-prefix membership, suffix-only extension, replacement/shortening, accepted ancestry, rejection/reset and follower recovery. Current-head drafted/plain, resumed/fresh, concurrent/solo and grammar model equality remain pending.
  • Main against this branch, same machine and checkpoint: pending with an admitted TP2 GLM setup.
  • Decode speed before and after: pending at one stream and the widest admitted counts, greedy and sampled.
  • Prompt speed before and after: pending cold cache-off measurements at admitted lengths.
  • Licence of any new weights or drafter, or "n/a": n/a; no weights or drafter added.

Checklist

  • The title and description identify the cumulative stack and this PR's own range.
  • History regressions cover suffix-only extension and authoritative rebuilds.
  • Comments and docstrings are one line. Measurements remain outside source.
  • Publication content contains no personal data, machine names, internal hosts, local paths or AI attribution lines.
  • Every token remains inside the existing lane rounds; active repetition uses request-owned state.
  • FP32 repetition arithmetic preserves the agreed sign rule.
  • Current-head CUDA contract, resource and identity-SASS gates.
  • Drafted/plain, resumed/fresh and concurrent/solo TP2 equality.
  • Main-against-branch decode and cold-prefill comparison.

ashhart and others added 5 commits November 12, 2026 14:04
…ake is a 400

A client could not tell from /v1/models whether a model takes image or
video parts, and a video_url part sent to a server whose model reads video
but which has no libtfvideo (GLM-5.3-Flash with the default image) ended in
HTTP 500 at the first video (ashhart#3004).

Each model entry now ends with "input_types": ["text"], plus "image" with
--vision, plus "video" when the checkpoint offers video and libtfvideo
loads. The keys before it keep their order and bytes. There is no new
option: the server decides it once at startup (input_types.zig), loading
libtfvideo with the loader every video uses, which keeps its answer, and
only when the model offers video. Server.media() takes its image and video
flags from the same Inputs, so every request route (chat, completions,
responses, messages, tokenize) refuses a video_url part on a server that
does not list "video" with the 400 an unsupported part gets
(invalid_request_error). The startup log names the missing library once.
If libtfvideo still fails at a video, that is now a 400 naming the library
instead of a 500.

The 17 server goldens holding a /v1/models reply gain the key on each
entry (the fake engine is text only) and 25 bytes of Content-Length per
entry, and nothing else. New tests in models_input_types_test.zig cover
the listing (text, image, image and video, and video without the library),
the 400 for both kinds of model without video, and the library refusal.
docs/api.md describes the key; docs/vision.md says when libtfvideo loads.

Co-authored-by: WebReflection <85749+WebReflection@users.noreply.github.com>
@gilp456

gilp456 commented Nov 13, 2026

Copy link
Copy Markdown
Author

/atc depends 3154

@ashhart
ashhart force-pushed the lead/models-input-types branch from 0029d0e to eab1e7e Compare November 14, 2026 03:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants