Achieving Computation-Communication Overlap with Overdecomposition on GPU Systems

The landscape of high performance computing is shifting towards a collection of multi-GPU nodes, widening the gap between on-node compute and off-node communication capabilities. Consequently, the ability to tolerate communication latencies and maximize utilization of the compute hardware are becomi...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2020 IEEE/ACM Fifth International Workshop on Extreme Scale Programming Models and Middleware (ESPM2) s. 1 - 10
Hlavní autoři: Choi, Jaemin, Richards, David F., Kale, Laxmikant V.
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.11.2020
Témata:
application program interfaces
asynchronous task-based runtime
Charm++ parallel programming system
communication latencies
computation-communication overlap
compute hardware
computer graphic equipment
coprocessors
Delays
GPU computing
graphics processing units
Hardware
high performance computing
Jacobian matrices
Kernel
message passing
modern GPU systems
multiGPU nodes
off-node communication capabilities
on-node compute
overde-composition
overdecomposition
parallel architectures
parallel processing
parallel programming
Runtime
Task analysis
traditional CPU-based systems
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The landscape of high performance computing is shifting towards a collection of multi-GPU nodes, widening the gap between on-node compute and off-node communication capabilities. Consequently, the ability to tolerate communication latencies and maximize utilization of the compute hardware are becoming increasingly important in achieving high performance. Overdecomposition has been successfully adopted on traditional CPU-based systems to achieve computation-communication overlap, significantly reducing the impact of communication on application performance. However, it has been unclear whether overdecomposition can provide the same benefits on modern GPU systems. In this work, we address the challenges in achieving computation-communication overlap with overdecomposition on GPU systems using the Charm++ parallel programming system. By prioritizing communication with CUDA streams in the application and supporting asynchronous progress of GPU operations in the Charm++ runtime system, we obtain improvements in overall performance of up to 50% and 47% with proxy applications Jacobi3D and MiniMD, respectively.
DOI:10.1109/ESPM251964.2020.00006