Achieving Computation-Communication Overlap with Overdecomposition on GPU Systems

The landscape of high performance computing is shifting towards a collection of multi-GPU nodes, widening the gap between on-node compute and off-node communication capabilities. Consequently, the ability to tolerate communication latencies and maximize utilization of the compute hardware are becomi...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2020 IEEE/ACM Fifth International Workshop on Extreme Scale Programming Models and Middleware (ESPM2) s. 1 - 10
Hlavní autori: Choi, Jaemin, Richards, David F., Kale, Laxmikant V.
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 01.11.2020
Predmet:
application program interfaces
asynchronous task-based runtime
Charm++ parallel programming system
communication latencies
computation-communication overlap
compute hardware
computer graphic equipment
coprocessors
Delays
GPU computing
graphics processing units
Hardware
high performance computing
Jacobian matrices
Kernel
message passing
modern GPU systems
multiGPU nodes
off-node communication capabilities
on-node compute
overde-composition
overdecomposition
parallel architectures
parallel processing
parallel programming
Runtime
Task analysis
traditional CPU-based systems
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:The landscape of high performance computing is shifting towards a collection of multi-GPU nodes, widening the gap between on-node compute and off-node communication capabilities. Consequently, the ability to tolerate communication latencies and maximize utilization of the compute hardware are becoming increasingly important in achieving high performance. Overdecomposition has been successfully adopted on traditional CPU-based systems to achieve computation-communication overlap, significantly reducing the impact of communication on application performance. However, it has been unclear whether overdecomposition can provide the same benefits on modern GPU systems. In this work, we address the challenges in achieving computation-communication overlap with overdecomposition on GPU systems using the Charm++ parallel programming system. By prioritizing communication with CUDA streams in the application and supporting asynchronous progress of GPU operations in the Charm++ runtime system, we obtain improvements in overall performance of up to 50% and 47% with proxy applications Jacobi3D and MiniMD, respectively.
DOI:10.1109/ESPM251964.2020.00006