Skip to content

perf(moe): fuse GLM prefill SwiGLU and UE8M0 quantization - #1709

Merged
hiworldwzj merged 3 commits into
mainfrom
bsh/glm-prefill-swiglu-fp8
Nov 16, 2026
Merged

hiworldwzj merged 3 commits into
mainfrom
bsh/glm-prefill-swiglu-fp8

Conversation

@shihaobai

Copy link
Copy Markdown
Collaborator

Fuse SwiGLU and FP8 quantization in the Hopper GLM expanded-MoE prefill path, preserving main's UE8M0 scales and intermediate rounding. Unsupported shapes, UE8M0-off execution and autotuner warmup retain the existing path.

Validation against main dc53088 on H100: 238 GPU tests pass; seven real-expert consumer cases are byte-exact, including a 32K chunk plus a fallback tail. With main's default kernel configs, the expert consumer is 1.22–1.26x faster at 896–32768 expanded rows (W13, activation/quantization, W2 and gather; excludes DeepEP communication).

hiworldwzj and others added 2 commits November 16, 2026 06:36
…tion

Drop gemm_out_a as soon as each activation path finishes reading it, and spell out the FP8 autotune key instead of unpacking a conditional dict.

Co-authored-by: Cursor <cursoragent@cursor.com>
Note that the fused SwiGLU path is only for the Hopper GLM expert shape, and only for chunks of 896 to 32768 rows.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hiworldwzj
hiworldwzj merged commit 2f56e50 into main Nov 16, 2026
1 check passed
@hiworldwzj
hiworldwzj deleted the bsh/glm-prefill-swiglu-fp8 branch November 16, 2026 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants