Skip to content

train 1113190654: #2814 #2697 #2990 (CI only) - #3262

Closed
ashhart wants to merge 25 commits into
mainfrom
lead/train-1113190654
Closed

ashhart wants to merge 25 commits into
mainfrom
lead/train-1113190654

Conversation

@ashhart

@ashhart ashhart commented Nov 13, 2026

Copy link
Copy Markdown
Owner

CI for a landing train: #2814 #2697 #2990 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.

jschmied and others added 25 commits November 13, 2026 18:42
T0's group backend could not abort GLM's communicator before the
RoCE group was installed, because it only borrowed it. GLM now hands
the backend the group that owns the communicator from Engine.init,
so an abort always reaches it once, and the start-up unwind uses the
group's guarded destroy, which skips the destroy after an abort.
A failed spawn returned while the ranks already started still wrote to
the test's frame, the same pattern #2700 fixed in host.runWith. Both tests
now join the ranks they started on every return; in the gather test that
also covers rank 0's own steps between the spawns and the join. Every wait
in Bootstrap has a deadline, so the join is bounded.
Each app run uploads its ~230 MB AppImage, and with GitHub's default 90-day retention 2,069 copies (475 GB) had piled up
in Actions storage by 12 Nov, enough to hit the account's Actions limit. The build is smoke-tested in the job itself;
the upload is only for a quick download, which 3 days covers.
A reply that keeps a tool-call envelope as text now carries
tensorfold.kept_tool_calls with unoffered and malformed counts and the
unoffered names the model wrote, capped at eight names of 64 bytes
each. /metrics counts both families. docs/api.md no longer claims an
unoffered call comes back as a tool call.
The four bad_tool cases (whole, stream, one call allowed, and its
stream) keep an envelope that names the unoffered tool "other", so
their tensorfold block now ends with kept_tool_calls: one unoffered,
none malformed, names ["other"]. No other expected reply changes; the
/metrics cases stay on the known-differences list, which they reach
before the new counters.
With the profiler on, prompt_pair.pairedAt and prompt_solo.at return false
(the e.prompt_profs[3] != null term in cuda_prompt_pair.zig and
cuda_prompt_solo.zig), so every prompt step is a single chunk. The
environment table did not say so, and a profile could be read as the
served shape.
@ashhart

ashhart commented Nov 13, 2026

Copy link
Copy Markdown
Owner Author

Stopped: main is frozen for the 1.2.8 release. These PRs land first after it.

@ashhart ashhart closed this Nov 13, 2026
@ashhart
ashhart deleted the lead/train-1113190654 branch November 13, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants