Skip to content

minimax_m3 metal: prefill runs a chunk a call, so a new prompt no longer stalls other streams' rounds (#3021) - #3089

Open
ashhart wants to merge 1 commit into
mainfrom
lead/minimax-m3-metal-prefill-step
Open

ashhart wants to merge 1 commit into
mainfrom
lead/minimax-m3-metal-prefill-step

Conversation

@ashhart

@ashhart ashhart commented Nov 13, 2026

Copy link
Copy Markdown
Owner

What this changes

MiniMax-M3 on Metal ran a new stream's whole prompt in one call on the lane thread, so with shared rounds every other stream's reply waited for the whole prefill. The backend now declares prefill_step and cancel_prefill beside reports_fill, so the lane host runs one chunk a call and decode rounds run between chunks, as Gemma 4 (#3025), Nemotron (#3054) and LFM2-MoE (#3058) do.

Checks on this head

  • Lean checker: 0 new; every file under 600 lines. zig fmt --check: clean.
  • test-minimax-m3-metal-steps-host: 5/5 (4 named, including the drafter's absorb order); check-metal-prefill-rest-host passes (it compiles the real prefill, prefill_step and cancel_prefill bodies).
  • Not run here: the Metal device steps. We'll run them on an M5 Mac before landing.

Receipt

  • Main against branch: owed, on an M5 Mac: replies byte-identical to main at T=0 and seeded, including a long prompt arriving while another stream decodes, a prompt-cache resume from a mark and a cancel mid-pass; lone decode and lone TTFT within run-to-run noise.

Checklist

  • The title and description match the head commit.
  • Comments and docstrings are one line.
  • No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

…ger stalls other streams' rounds

MiniMax-M3's Metal backend gains prefill_step and cancel_prefill, and now
reports its fill (#3021). The prompt pass moves out of backend.zig into
prompt_pass.zig; warm() keeps its stream-less feed.

Each call runs one chunk at exactly the whole pass's cuts (the prompt
chunk cap, then a planned start or kept mark inside the chunk); MiniMax
declares no chunk invariance, so max_rows is ignored and the cuts never
move. The whole pass and the steps share one chunk body, so replies are
unchanged. Each step drains other streams' steps in flight before its
forward, as the whole pass drained before it began.

For a drafting stream that steps, each chunk's tapped rows are served by
features() while no forward has rewritten that scratch, and a mark reached
by the chunk is kept at the next call, after the drafter has absorbed it.
A cancel keeps the state at the last chunk's end; a stream that leaves
mid-pass is forgotten; image rows are made once, at the first call.

Host tests on recording fakes: stepped chunks equal the whole loop's, a
resumed pass cuts as main does, a cancel keeps the last chunk's end,
another stream decodes between two chunks, and a drafter absorbs every
chunk with the mark kept after its absorb. The new step joins the macOS
host CI list.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant