Skip to content

server: add request-only repetition penalty contract - #3153

Draft
gilp456 wants to merge 2 commits into
ashhart:lead/models-input-typesfrom
gilp456:codex/glm-repetition-api
Draft

gilp456 wants to merge 2 commits into
ashhart:lead/models-input-typesfrom
gilp456:codex/glm-repetition-api

Conversation

@gilp456

@gilp456 gilp456 commented Nov 13, 2026 •

Copy link
Copy Markdown

What this changes

Adds the request-layer repetition_penalty contract while engine capability remains opt-in. Chat, completions, Messages and Responses accept a positive finite float32 factor; omission and null use identity factor 1; a backend without the capability refuses an effective nonidentity factor before generation. If generation_config.json declares an active factor, startup tells the operator to pass it explicitly.

Stack order: this request/API layer, #3154 (kernels/contracts), then #3155 (GLM engine wiring). All three drafts target lead/models-input-types and identify their own commit ranges.

Checks on this head

  • Head commit: bedf1b166.
  • Lean checker: 345 findings on the base and head, zero new findings.
  • Tests run:
    • zig build test-repetition-api-host test-golden --summary all: 8/8 steps; 29/29 focused unit tests; 55/55 fixture/security goldens; 319/331 server goldens with 12 known differences and zero unexpected differences.
    • zig build test test-kernel-contract test-golden fuzz-cortheon --summary all: 677/677 steps; 8,679 passed, 222 skipped, zero failed.
    • zig fmt --check build.zig zig, git diff --check, and tools/pr_check.py: pass; 31 changed files and zero new lean findings.
  • CI: pending on this draft head.
  • Not run, and why: CUDA, device, model and performance gates belong to the later kernel and GLM layers.
  • Plain zig build: 26/29 steps succeeded; tf-qwen35-cuda hits seven Windows std.c/fabric-network errors. The exact parent 0029d0e8 reproduces the same seven errors and 26/29 result; this PR does not change that code.
  • Cluster activity: zero SSH, deployment, service, Docker or model-process commands.

Receipt

  • Main against branch: n/a for active repetition processing; this layer supplies request admission and refuses active factors until an engine advertises support.
  • Boxes it reaches: chat, completions, Messages and Responses parsing, validation, capability reporting and engine request planning.
  • New switch? Default, accepted values, profile row: request field only; positive finite float32, with omitted or null equal to 1; no profile, checkpoint or server-wide default.
  • Environment: Windows host, Zig 0.17.0; host and golden checks above; GPU and checkpoint n/a for this layer.
  • Paths and platforms the change reaches, which I ran and which I could not: native server request paths ran on the host; CUDA device and model paths remain in the later stack.
  • Exactness: identity/default propagation and effective-float32 capability refusal are covered by host tests; model token equality remains a final-stack gate.
  • Main against this branch, same machine and checkpoint: n/a for this API-only layer.
  • Decode speed before and after: n/a; this layer has no active engine implementation.
  • Prompt speed before and after: n/a; this layer has no prompt-processing implementation.
  • Licence of any new weights or drafter, or "n/a": n/a; no weights or drafter added.

Checklist

  • The title and description match the head commit, receipt and stack.
  • New startup-note and effective-float32 tests failed before their fixes and pass on this head.
  • Comments and docstrings are one line. Measurements remain outside source.
  • Publication content contains no personal data, machine names, internal hosts, local paths or AI attribution lines.
  • Existing lane rounds, draft settings and precision remain unchanged.
  • Final-stack drafted/plain, resumed/fresh and concurrent/solo model gates.
  • Final-stack decode and cold-prefill comparison.

ashhart and others added 2 commits November 12, 2026 14:04
…ake is a 400

A client could not tell from /v1/models whether a model takes image or
video parts, and a video_url part sent to a server whose model reads video
but which has no libtfvideo (GLM-5.3-Flash with the default image) ended in
HTTP 500 at the first video (ashhart#3004).

Each model entry now ends with "input_types": ["text"], plus "image" with
--vision, plus "video" when the checkpoint offers video and libtfvideo
loads. The keys before it keep their order and bytes. There is no new
option: the server decides it once at startup (input_types.zig), loading
libtfvideo with the loader every video uses, which keeps its answer, and
only when the model offers video. Server.media() takes its image and video
flags from the same Inputs, so every request route (chat, completions,
responses, messages, tokenize) refuses a video_url part on a server that
does not list "video" with the 400 an unsupported part gets
(invalid_request_error). The startup log names the missing library once.
If libtfvideo still fails at a video, that is now a 400 naming the library
instead of a 500.

The 17 server goldens holding a /v1/models reply gain the key on each
entry (the fake engine is text only) and 25 bytes of Content-Length per
entry, and nothing else. New tests in models_input_types_test.zig cover
the listing (text, image, image and video, and video without the library),
the 400 for both kinds of model without video, and the library refusal.
docs/api.md describes the key; docs/vision.md says when libtfvideo loads.

Co-authored-by: WebReflection <85749+WebReflection@users.noreply.github.com>
@gilp456

gilp456 commented Nov 13, 2026

Copy link
Copy Markdown
Author

/atc depends 3019

@ashhart
ashhart force-pushed the lead/models-input-types branch from 0029d0e to eab1e7e Compare November 14, 2026 03:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants