Scalable reduction collectives with data partitioning-based multi-leader design

Existing designs for MPI_Allreduce do not take advantage of the vast parallelism available in modern multi-/many-core processors like Intel Xeon/Xeon Phis or the increases in communication throughput and recent advances in high-end features seen with modern interconnects like InfiniBand and Omni-Pat...

Full description

Saved in:
Bibliographic Details
Published in:International Conference for High Performance Computing, Networking, Storage and Analysis (Online) pp. 1 - 11
Main Authors: Bayatpour, Mohammadreza, Chakraborty, Sourav, Subramoni, Hari, Lu, Xiaoyi, Panda, Dhabaleswar K. (DK)
Format: Conference Proceeding
Language:English
Published: New York, NY, USA ACM 12.11.2017
Series:ACM Conferences
Subjects:
Analytical models
Collectives
Computer architecture
Computer systems organization > Architectures > Parallel architectures
Computing methodologies > Parallel computing methodologies > Parallel algorithms > Massively parallel algorithms
Costs
Data Partitioning
High performance computing
MPI
MPI_Allreduce
Multi-Leader
Organizations
Program processors
SHArP
Throughput
SHArP
data partitioning
collectives
multi-leader
MPI
MPI_allreduce
ISBN:9781450351140, 145035114X
ISSN:2167-4337
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Existing designs for MPI_Allreduce do not take advantage of the vast parallelism available in modern multi-/many-core processors like Intel Xeon/Xeon Phis or the increases in communication throughput and recent advances in high-end features seen with modern interconnects like InfiniBand and Omni-Path. In this paper, we propose a high-performance and scalable Data Partitioning-based Multi-Leader (DPML) solution for MPI_Allreduce that can take advantage of the parallelism offered by multi-/many-core architectures in conjunction with the high throughput and high-end features offered by InfiniBand and Omni-Path to significantly enhance the performance of MPI_Allreduce on modern HPC systems. We also model DPML-based designs to analyze the communication costs theoretically. Microbenchmark level evaluations show that the proposed DPML-based designs are able to deliver up to 3.5 times performance improvement for MPI_Allreduce for multiple HPC systems at scale. At the application-level, up to 35% and 60% improvement is seen in communication for HPCG and miniAMR respectively.
ISBN:9781450351140
145035114X
ISSN:2167-4337
DOI:10.1145/3126908.3126954