Achieving Computation-Communication Overlap with Overdecomposition on GPU Systems

The landscape of high performance computing is shifting towards a collection of multi-GPU nodes, widening the gap between on-node compute and off-node communication capabilities. Consequently, the ability to tolerate communication latencies and maximize utilization of the compute hardware are becomi...

Full description

Saved in:
Bibliographic Details
Published in:2020 IEEE/ACM Fifth International Workshop on Extreme Scale Programming Models and Middleware (ESPM2) pp. 1 - 10
Main Authors: Choi, Jaemin, Richards, David F., Kale, Laxmikant V.
Format: Conference Proceeding
Language:English
Published: IEEE 01.11.2020
Subjects:
application program interfaces
asynchronous task-based runtime
Charm++ parallel programming system
communication latencies
computation-communication overlap
compute hardware
computer graphic equipment
coprocessors
Delays
GPU computing
graphics processing units
Hardware
high performance computing
Jacobian matrices
Kernel
message passing
modern GPU systems
multiGPU nodes
off-node communication capabilities
on-node compute
overde-composition
overdecomposition
parallel architectures
parallel processing
parallel programming
Runtime
Task analysis
traditional CPU-based systems
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The landscape of high performance computing is shifting towards a collection of multi-GPU nodes, widening the gap between on-node compute and off-node communication capabilities. Consequently, the ability to tolerate communication latencies and maximize utilization of the compute hardware are becoming increasingly important in achieving high performance. Overdecomposition has been successfully adopted on traditional CPU-based systems to achieve computation-communication overlap, significantly reducing the impact of communication on application performance. However, it has been unclear whether overdecomposition can provide the same benefits on modern GPU systems. In this work, we address the challenges in achieving computation-communication overlap with overdecomposition on GPU systems using the Charm++ parallel programming system. By prioritizing communication with CUDA streams in the application and supporting asynchronous progress of GPU operations in the Charm++ runtime system, we obtain improvements in overall performance of up to 50% and 47% with proxy applications Jacobi3D and MiniMD, respectively.
DOI:10.1109/ESPM251964.2020.00006