Repository navigation
refactor(fsdp): DiffusionLossContext stops carrying loop state - #368
Merged
Merged
Conversation
6 tasks done
`train` returned its train output directly, so the driver's ray.get on the train refs pulled every rank's output into the controller before the next group read it again from the object store. Return `Box(ray.put(output))` like the rollout shards: the driver gets a small box, the payload stays in the store until the paired rank of a later group unboxes it. Also note the train-output hook as a `--teacher-load` consumer in docs/fsdp_model_roles.md, which cli-reference.md already did. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
train returns Box(ray.put(output)); the driver discarded its ray.get and passed the task refs on, so the consumer's box.inner hit an ObjectRef. Keep the boxes the way the rollout shards are kept. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
…re micro-batching
…tprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
…e the student's samples
… from the critic group's scores
#338 renamed the hook to --custom-train-input-postprocess-path; the DMD2 defaults and the critic override still set custom_train_input_function_path, so --loss-type dmd2 failed in parse_args. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
…metrics to judge it
… reward reads the media
miles/ray/rollout.py imports imageio_ffmpeg to mux the eval audio; the GPU image had it only as a transitive dependency and the hosted CPU suite not at all, so tests/fast/rollout/test_eval_media.py failed to import there. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
Merged
5 of 6 tasks
Whether a hook forwards the teacher is the hook's business; the argument layer cannot tell from --loss-type and the hook paths who will read it, so the check only ever guessed. --teacher-load is now accepted under every loss type (dmd2 still requires it).
Stop filling the --custom-*-path arguments on behalf of nft / dmd2 and stop overriding them per trainer role: a group's prepare, formula, train-input and train-output hooks come from loss_hub.LOSS_HOOKS keyed by (loss_type, role), the way miles core's get_loss_function does. Only custom_loss reads the path flags. The critic overrides shrink to the six values that really differ (checkpoint, lr, load/save dirs, steps, batch).
A path given under any other loss type would be silently ignored now that the table selects the hooks, so fail at parse time instead.
DMD2 has no log-prob ratio to recompute, and once the recompute loop passes its real step index the critic's last window would run its scoring prepare and formula a second time. sft_loss and nft already reject the flag.
microbatch_id and optim_step_idx were written onto DiffusionLossContext before every forward and read back by the hooks; the planned micro-batch prefetch would race on them. They are now required keyword arguments of every prepare and formula hook, passed by the two trunk loops; the numbering is unchanged (training from 0, the old-log-prob recompute of windows 1.. reuses their training forwards' numbers).
The DMD2 critic formula appended its scoring inputs to a list on the context and the train-output hook read them back: a hook-to-hook channel invisible in the call graph. The trunk now owns one list per train call and passes it to the formulas and to the train-output hook.
Nothing writes the context after construction any more: it is the rollout-level, read-only view the hooks share. eq=False keeps identity comparison; a synthesized hash would fail on the dict fields.
Rockdu
force-pushed
the
dmd2/refactor-a-hook-registry
branch
from
November 11, 2026 13:03
fa6087f to
972e01e
Compare
Rockdu
force-pushed
the
dmd2/refactor-c-ctx-loop-state
branch
from
November 11, 2026 13:03
b6aee00 to
60acc02
Compare
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #367 (the hook-registry PR), which sits on #343: the diff here is the 4 commits above it.
What
DiffusionLossContextstops carrying loop state.microbatch_idandoptim_step_idxbecome required keyword arguments of every prepare hook (andoptim_step_idxof every formula), passed by the two trunk loops; the list the DMD2 critic formula fills for the train-output hook is owned by_train_coreand passed to the formulas and totrain_output(ctx, train_pairs, train_output_inputs).@dataclass(frozen=True, eq=False)): a rollout-level, read-only view; nothing writes it after construction.--loss-type dmd2rejects--diffusion-recompute-old-log-prob(no log-prob ratio to recompute; with a real step index the recompute loop would run the critic's scoring window twice), likesft_lossandnftalready do.Why
The planned micro-batch prefetch (prepare of micro-batch n+1 while the loss of n runs) races on any per-micro-batch field shared through the context, and a hook-to-hook channel through the context is invisible in the call graph. With the counters and the list as arguments, every per-call fact is in the signature and the context can be shared freely. Numbering is unchanged: training counts from 0; the old-log-prob recompute of windows 1.. reuses their training forwards' numbers.
Not in this PR: moving the DMD2 teacher forward into the trunk and deleting the train-output hook (left to the DMD2 stack's author).
Validation
CPU, on this head:
tests/fast(minus the Mac-only distributed / ray-socket cases)score_last_step_samplestest_train_input_hook.py(microbatch_id, optim_step_idx, write_old_log_prob)per forward; the train-output hook receives the same list object every forward appended totest_reference_models.py_forward_train_pair_batchhands prepare{microbatch_id, optim_step_idx, pad_to_len}and the formulaoptim_step_idxand the same listpre-commit run --all-filesGPU (4x H200, sglang-miles head), before the GPU runs were paused, on the same
miles/code except two argparse help strings:test_sd3_ocr_grpo_2xGPU8/8,test_sd3_nft_pickscore_3xGPU10/10,test_h3_t2va_sft_2xGPU2/2 metric series bitwise.Behaviour ledger (intentional changes): a custom prepare with the old signature
prepare(ctx, batch, *, pad_to_len=None)fails at the first micro-batch withTypeError: unexpected keyword 'microbatch_id'; a custom train-output hook with the old two-argument signature fails the same way; a custom formula written as documented (with**kwargs) keeps working; readingctx.microbatch_id/ctx.optim_step_idx/ctx.train_output_inputsis anAttributeError; writing a context field is aFrozenInstanceError. Everything else is bitwise.Files
miles/backends/fsdp_utils/loss_hub/types.py: three fields removed; context frozenmiles/backends/fsdp_utils/actor.py: the two loops pass the counters; one list per train callmiles/backends/fsdp_utils/loss_hub/{sft,nft,flow_grpo,dmd2}.py: hook signaturesmiles/backends/fsdp_utils/loss_hub/__init__.py:LossHookscontractmiles/utils/arguments.py: dmd2 rejects the recompute flag; help stringsdocs/user-guide/customization.md: hook signatures and the numbering ruletests/fast/...: call sites updated; new wiring, accumulator-identity and frozen-context tests;tests/fast-gpu/backends/fsdp_utils/_reference_models_worker.pycall sites updatedChecklist
pre-commit run --all-filespassespytest -xis green (tests/faston CPU;tests/fast-gpuwas updated but not run on a GPU)python3 train.py --helpstill parses (train_diffusion.py --help)