Repository navigation

perf(attention): run up to 8 query rows per request in the triton decode kernel - #900

Merged

jason-fxz merged 2 commits into

mainfrom

feat/decode-multi-row

Nov 13, 2026

Merged

perf(attention): run up to 8 query rows per request in the triton decode kernel#900
jason-fxz merged 2 commits into
mainfrom
feat/decode-multi-row

Commits on Nov 13, 2026
- perf(attention): run up to 8 query rows per request in the triton decode kernel
  jason-fxz
  committed
  e3c4e4a
- fix(attention): keep multi-row triton decode free of register spills on sm_80 to sm_120
  jason-fxz
  committed
  7e353d1