Repository navigation
Conversation
…x kernels nothing launches q27_prompt_product, _silu, _decay, _qk, q27_prompt_conv and q27_prompt_recur were reached only through prompt_gpu.Kernels and prompt_state_gpu.Kernels, which nothing builds with pipelines. The host records in prompt_glue.zig, prompt_gpu.zig, prompt_state.zig and prompt_state_gpu.zig, the build and runtime source lists, the runtime analysis line and the prompt.metal write in tests/qwen27/run.zig go with them.
…see calls No family calls rotate(); the shared attention's runtime callers (Gemma 4 and Qwen27 DFlash) rope their rows elsewhere. Shape loses rotary_dim and rope, which only the kernel read, and lanes_see feeds raw K and Q straight to append and attention.
…passed to shared_attention; changelog line
…says dsv41_cuda.module pinned the family, kernel and runtime modules to Debug, and dsv41-assets built tf-dsv41-assets on it, so the AOT planner always ran unoptimized. Defilan measured one aot-needs enumeration at --context 524288 on a GB10: about 45 minutes as built, under 7 with the family modules at ReleaseSafe, planning the same 138 variants (#2270). module and runtime keep their Debug signatures for every other caller; moduleAt and runtimeAt take the mode, and dsv41-assets builds the tool on moduleAt(.ReleaseSafe), so its runtime and kernel modules go to ReleaseSafe too. ReleaseSafe keeps the safety checks, and no code under the tool reads the optimize mode, so the plan is unchanged. test-dsv41-assets-host stays on the Debug module. Refs #2270. Co-authored-by: Defilan <7034124+Defilan@users.noreply.github.com>
…nt blocks (relu^2 up, fp32 or bf16 down)
…and bf16 down, gates 1-5 with two controls
…, as pack does, with a test
…Apache-2.0, as its model card now says
Give tf_lane the reg path's 16-row grid axis and runtime sums stride. Each block keeps the same group products, FMAs and K reduction; cooperative and older-chip encodes retain 16 rows. Qwen27 selects a whole-prompt key for non-reg widths behind the existing Ultra gate. Decode and mixed stacks stay short. Compare each width with main's frozen short kernel, including ragged tails, producer sums, strided outputs and three controls. Pin real quant dispatch counts with main-red host regressions. Refs #2818
The full lane contract compiles all four tf_lane_sums sources for M1 and M2 at a 1024-thread pipeline limit. Record those measured values and keep the actual 256-thread launch. Tensor-op sources remain explicitly unmeasured on these chips. Pin the distinction in a host regression; no shader changes. Refs #2818
Only whole-prompt layouts compile the independent row-block header. Short layouts retain a scalar group coordinate and their original row extent; guarded sums remain bounds-safe. Keep older-chip sources on that short variant as well. Compare both short and long programs against frozen main at every maintained byte-check shape and row count. Refs #2818
Leave other affine lane callers on their scalar 16-row source and capacity. Only the Qwen prompt key and byte fixture request independent row blocks. Pin the shared shader bytes, describe the row layout accurately and cover the 16/17-row boundary. Refs #2818
…, read at GLM's open GLM-5.3-Flash's attention keeps no rotary positions, so only its trained length bounds the window. TF_GLM_CONTEXT_CEILING, a .glm row, sets the window --context may ask for, up to 2^31 - 32, where the slots keep i32 positions. GLM reads it at its open once its .glm rows are checked; unset, the ceiling and the refusal are main's. A window past the trained length says so at startup.
…a_env.zig cuda_native.zig had passed the 600-line limit. ceiling_switch and raisedCeiling move to cuda_env.zig, which owns the .glm check, and the i32 test to cuda_env_test.zig; cuda_native.zig keeps the two names the CUDA open looks up.
…est file and cuda_window.zig Main grew cuda.zig to 587 lines, so the branch's additions passed the 600-line limit again. The open-time test of raised context ceilings moves to cuda_ceiling_test.zig, imported beside cuda_health_test.zig, and the lookup of a family's raised ceiling and its switch to cuda_window.zig's familyCeiling; cuda.zig keeps one call.
…t.zig Main's eb29a68 added a line to cuda_native.zig, so merged with main the file comes to 601 lines, past the 600-line limit. The test that capacity keeps a verify window past the context, a multiple of 64, moves whole to cuda_pages_test.zig, which tests GLM's pages at that capacity: 594 lines in cuda_native.zig here, 595 merged with main.
A fraction after --compact-at auto left auto mode on, because the flag set compact_auto and a later fraction never cleared it, so the server kept the first value where every other flag keeps the last. Each occurrence now replaces both fields, which a config file's value under a command-line override depends on, and a filtered host step runs the test that fails without the change.
Pin eviction of redundant own history before another conversation's newest state under pressure. Retain prompt-born provenance and prove a same-prompt regenerate restores history when all three states fit. Refs #2881
Give test-native a timed shard with the Metal toolchain. Keep it nightly until the CI runner establishes its wall time. Refs #2880
…xperts, Drama 3 and Kolibri-1 read through it
…t says an f16 NaN stays a NaN
… of few rows On gfx1151 a routed prompt holds about 80 experts of 8 rows or fewer at any length. On the 64-row matrix tile each of their blocks reads 128 columns of codes a stage, a page of weights every four columns, and the cost of those reads follows the pages a block touches. wmma_item_tile now runs gemm_kp_wave for an item of at most the launch's cut: a lane a column's group, a column's whole row of codes read at once, no barrier. gemm_kp_wave is a wave's passes of gemm_kp_tile, factored out so both tiles run them; the K-parallel tiles keep their statement order and their register and scratch use. A pass of the item tile takes 2 rows and 2 columns a lane, which stays in registers, and keeps its partials in the block's code stages, so the tile's LDS is unchanged. The cut is a card's: `few_rows` in the tuning row (8 on gfx1151's item64 row, none elsewhere), carried to the kernel in Routing.few by prefillLaunch. A kernel test runs one plan through both paths at every width and compares bytes; the contract runs the tile with the cut and a control whose passes stop after a wave's first. The tile's record holds ROCm 10.1's hipcc. The build now tracks tiles/matrix_item.hpp. Refs #2795
…ks bounds (#2878 review) The review's asks on the tile's cut for items of few rows: - registry: `rowOf` returns null for an entry before the registry's own instead of subtracting first, which trapped. A toy-registry test covers a named entry, an unnamed one and entries before and after it. - tuning: only `mlx.routed.prefill.item64` rows set `few_rows`, at no more than the tile's K-parallel passes take, and gfx1151's is set. - launches: `prefillLaunch` over a registry of toy entries carries its row's cut in `route.few`, whatever the caller's route held, and none for a row without one. It fails when the copy is lost. Refs #2795
8 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2851 #2765 #2883 #2889 #2891 #2892 #2878 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.