Repository navigation
GPU Metal: do not let Metal track hazards on the memory buffers - #16068
Merged
Merged
Conversation
All memory of the reconstruction lives in three MTLBuffers, so with Metal's default hazard tracking every encoder that writes device memory conflicts with every other one, and Metal ordered them even across the streams' command queues: in a Metal System Trace the per-sector kernels the chain spreads over 8 streams did not overlap at all (kernel time summed to the GPU busy time). The backend orders the work itself, as CUDA streams do: a fence between the encoders of a stream, an event between its command buffers, and the chain's events between streams. So create the buffers untracked; the kernels of different streams now overlap by about 150 ms per event. On an M1 Max, interleaved over 3 rounds of 10 runs (load average 4 to 7), the 30-collision Pb-Pb time frame takes 975 ms per event instead of 1046, and a 10-collision Pb-Pb time frame dumped from the full system test 754 ms instead of 850. The clusters are byte-identical to the CPU's on the 30-collision time frame and on sim5pp.
Member
Author
|
Metal only. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
All memory of the reconstruction lives in three MTLBuffers, so with
Metal's default hazard tracking every encoder that writes device
memory conflicts with every other one, and Metal ordered them even
across the streams' command queues: in a Metal System Trace the
per-sector kernels the chain spreads over 8 streams did not overlap at
all (kernel time summed to the GPU busy time). The backend orders the
work itself, as CUDA streams do: a fence between the encoders of a
stream, an event between its command buffers, and the chain's events
between streams. So create the buffers untracked; the kernels of
different streams now overlap by about 150 ms per event.
On an M1 Max, interleaved over 3 rounds of 10 runs (load average 4 to
7), the 30-collision Pb-Pb time frame takes 975 ms per event instead
of 1046, and a 10-collision Pb-Pb time frame dumped from the full
system test 754 ms instead of 850. The clusters are byte-identical to
the CPU's on the 30-collision time frame and on sim5pp.