Repository navigation
Conversation
sufubao
changed the base branch from
refactor/attention-04-varlen
to
refactor/attention-05-runtime
November 13, 2026 08:05
sufubao
marked this pull request as draft
November 13, 2026 08:11
sufubao
force-pushed
the
refactor/attention-session-rewrite
branch
2 times, most recently
from
November 13, 2026 08:44
50c5c0c to
d7c97eb
Compare
sufubao
force-pushed
the
refactor/attention-session-rewrite
branch
from
November 13, 2026 09:42
d7c97eb to
72a2951
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
重写 #1721 的 attention 执行层,目标分支为
main。每个 backend 只实现一个 phase;ModelAttention负责选型、分组和绑定,AttentionSession统一执行校验和 workspace lease,State 保留存储、checkpoint、恢复及传输职责。当前 head 为
68ac92e28,已合入main的1a2a306d3并解决 import 冲突。此次同步包含上游 logprobs 和 PD 参数检查变更,不计为 attention 重构收益。执行必须处于 session scope 内,退出后拒绝 layer forward;数值探针使用同一入口,删除独立的 allocate/plan/build 路径。decode Graph 使用各模型声明的输入清单,校验存在性、shape、dtype、device 和同地址 view 的 stride,所有 lane 通过后才复制。图内生成的 DeepSeek-V4 C4 索引不属于重放输入。
out校验拒绝 strided view 行间的 padding 区域。GDN/KDA/MLA 和稀疏 attention 的模型配置由 adapter 转为不可变
LayerSpec.config,backend 不再读取模型字典。State 返回存储视图,由 adapter 组装计算资源;ReplaySSM backend 只收到数据视图,cursor 由模型在所有层完成后推进一次。GLM indexer 改用模型持有的SparseQueryMetadata。DeepSeek-V4 初始化从 spec 读取逻辑 head 数,并在模型侧处理 FlashMLA padding,修复启动时访问已删除属性的问题。设计说明:English / 中文。DeepSeek3.2 仍有 ragged metadata 的 provider 依赖;辅助 capture hooks 的测试只验证协议顺序,尚无真实 provider 的故障恢复验证。workspace lease 负责完成顺序,不是并发锁。
本次修订
674f11ebb的验证:最终合并提交
68ac92e28的 logprobs、采样参数和结构化输出集成测试为 304 passed,0 skipped,包括 MTP/overlap 用例。首次运行因未显式加载 AnyIO 插件跳过了 71 项异步测试;启用插件后的上述结果取代该次结果。针对当前main的整份 PR 固定版本格式检查通过。已逐文件核对:attention、State、模型实现和设计文档与674f11ebb相同;相关公共模型目录中变化的logprobs_capture.py与上游main一致。前述主要 GPU/真实模型结果来自674f11ebb,并非全部在合并提交上重跑。GPU 测试使用 torch 2.11/cu130、sglang-kernel 0.4.5、FlashInfer 0.6.18,限定 GPU 7、PyTorch 显存上限 8 GiB。3 项双 GPU 测试跳过,2 项未修改的大地址测试因单个张量超过显存上限未完成验证。GLM 的 410 项本地 PD/状态迁移组合分批在独立进程运行。真实模型启用 prefill/decode CUDA Graph,使用 64-token 前缀、512-token 缓存池、8 步交替 MTP 接受长度;验证本地 GPU/CPU/DP/PD buffer 恢复,不代表跨机 PD、HTTP 服务、全模型覆盖或吞吐验证。
实验命令、失败记录和 JUnit 保留在
/data6/exp_logs/runs/261114-163004-*、261114-163339-*、261114-163439-*;修订提交674f11ebb的两组真实模型报告在261114-163835-*、261114-164039-*。合并验证在261114-165127-*;最终源码与证据核对记录为/data6/exp_logs/runs/261114-165250-python3-tmp-attention-review-merge-receipt-py/verification.json。此前的测试数字属于旧版本,此处仅列本次修订的结果。