Repository navigation
test(graphcredit): add matched-budget gate1 harness - #395
Draft
diaochengxi wants to merge 4 commits into
Draft
diaochengxi wants to merge 4 commits into
diaochengxi wants to merge 4 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
Related to #391. Stacked on #394.
本 PR 实现 GraphCredit Gate 1:在相同模型、rollout、组大小、更新步数和评测集上比较
episode-level GRPO 与
StateGraphAdvantage的训练方向。任务设计
受控二维网格任务包含两步:
因此 GRPO 会把最终结果广播到 episode 内全部动作,而 GraphCredit 可以对相同
pre_state_key下的中间动作单独归一化。实现
GRPOAdvantage和StateGraphAdvantage;turn_credits、action span、policy version 和 terminal success;本地验证
164 passedpre-commit:全部通过A2 状态
已使用新授权连接 A2,确认:
/workspace/shared_assets/models/Qwen/Qwen3-0.6B已缓存。Gate1 脚本已成功下载到 A2 工作区,但跳板授权是一次性的。随后传输文档时远端 curl
挂起,中断命令导致中继关闭;当前尚无 GRPO/GraphCredit 两臂训练结果。本 PR 不会在
结果完成前宣称训练收益。
待完成