Repository navigation
GPU: more Metal bringup changes - #16064
Merged
Merged
Conversation
A stream's command buffer is committed only where something waits for it, so while the host drained one stream the GPU did not see what the other streams had encoded. In the merging across sector borders, which sorts on the host between kernels on alternating streams, each drain let the GPU run only the stream waited for, then idle while the host sorted: about 70 ms per event of 30-collision Pb-Pb. Commit the open buffers of all streams before waiting. In a Metal System Trace this also lets the streams overlap on the GPU, which they did not before. On an M1 Max the 30-collision Pb-Pb time frame takes 1204 ms per event instead of 1364 (4 interleaved rounds of 10 runs, load average 3 to 4), and the clusters are byte-identical to the CPU's.
DrainStream slept in waitUntilCompleted, which returns once the completion of the buffer has been processed, well after the GPU is done. With about 800 such waits per event of 30-collision Pb-Pb, the GPU sat idle for 58 to 71 ms per event while the host was still waiting in it, as os_signpost intervals around the wait show against the GPU state in a Metal System Trace. The last submitted buffer of a stream signals the stream's order event when all the work before it is done, so watch that event for up to 20 ms first, and sleep only for longer waits. On an M1 Max the 30-collision Pb-Pb time frame takes 1184 ms per event instead of 1210 (4 interleaved rounds of 10 runs, load average 4 to 5; waiting at most 5 ms gave 1192, at most 1 ms no gain), and the clusters are byte-identical to the CPU's.
GPUTPCGMMergerFollowLoopers follows each looper in one thread through a long serial loop of propagation steps and cluster searches. There are few loopers, about 206 per event of 30-collision Pb-Pb, so with 256 threads per group they all landed in one threadgroup, where the lanes of each SIMD group take different paths and run one after the other: 33 to 41 ms per event in a Metal System Trace, against 0.6 ms on the CPU. Without the cluster searches the kernel took 0.6 ms, and 32 or 64 threads per group did not help. With one thread per group every looper runs in its own SIMD group; 16 groups per GPU core leave room for twice as many loopers as here. On an M1 Max the kernel took 3.6 to 8.5 ms per event in a trace with 8 groups per core, and the 30-collision Pb-Pb time frame takes 1058 ms per event instead of 1088 with 8 or 16 groups per core alike (4 interleaved rounds of 10 runs, load average 4 to 7). The clusters are byte-identical to the CPU's.
ktf
changed the title
ktf
changed the title
ktf
changed the title
Member
Author
|
Metal only changes, not compiled and not touching common parts. Merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.