[jinyouzhi]

Analyze affine transfer batches before execution

Stacked on #8809 (executor) and #8966 (calibration path labels). Only the last commit is new.

Why

Plans that move the same bytes can differ a lot in time (up to 1.75× on 8× B70 between replica choices), and executor measurements show that the round count under a StagingBudget dominates sync time. A policy (#8935/#8936) or cost model (#8252) needs these facts before it runs a plan.

What

deepspeed/checkpoint/affine_transfer_analysis.py:

• analyze_transfers(plans, endpoints, element_size, budget, path_of) is a pure function with no communication. It replays the executor's build_operations and round packing and reports:
  • volume;
  • segment, chunk and message size distributions;
  • rounds;
  • per process: sent/received bytes, peers, messages, pack/unpack bytes, staging arena;
  • per-pair and per-path bytes.
• expected_stats(report, process) predicts that process's TransferStats.
• report_to_dict returns a JSON-ready summary.
• Messages use the same path_of labels as calibrate_links, so per-path totals can be priced with fit_links.

It does not choose replicas, rank plans, or model time.

Note: it imports the executor's private _packed_rounds / build_operations. @0z5a, would a public dry-run schedule API on the executor be preferable?

Tests

• Hand-counted rounds and bytes on an uneven handoff.
• Staging of strided sources.
• A batch with a shared plan.
• Path and pair labels in the JSON summary.
• 4 ranks: expected_stats == the executor's TransferStats, for a strided and a replicated parameter under budgets of 3 and 64 elements.

Results: 6 passed on 4× L20 (nccl) and on 4× Intel Arc Pro B70 (xccl). Mutation checks (dropping the receiver's round count, or the local bytes in the path summary) are caught.

[0z5a]

Read the last commit (72081c1b). The report is what a policy or cost model needs: per-process sent and received bytes, rounds and messages give the bandwidth and per-message terms, and expected_stats matching the executor's TransferStats on 4 ranks is a good anchor. It passes on CPU (gloo, 4 ranks) here.

I checked the schedule reconstruction against the executor at e38477d7. The direct-send test (_is_packed versus region.is_contiguous()), the per-peer, per-direction arena sized by the largest staged round, and rounds as the maximum over a process's peer queues all match today. Two structural suggestions:

• Share the schedule instead of re-deriving it. The module imports the executor's private _packed_rounds, and build_operations isn't in its __all__ either. It then re-implements the direct-versus-staged decision and the arena sizing. Those will drift when the executor's staging changes. staging_bytes in particular isn't covered by the expected_stats comparison, since TransferStats has no staging field. Would a public schedule helper in the executor work for you, returning each peer's rounds and whether each round is sent directly? Both sides could then call it.
• Ordering. This stacks on #8966 only for LOCAL and default_path_of. Defining those here, or in a small shared module that calibration imports, would let this land first.

Minor: the report numbers processes by transport-group rank, while calibration uses default-group ranks, so the path labels join only when the two groups coincide. The docstring says so, but recording that assumption in the report itself would stop a consumer from mixing the two.