Scalable reduction collectives with data partitioning-based multi-leader design

Existing designs for MPI_Allreduce do not take advantage of the vast parallelism available in modern multi-/many-core processors like Intel Xeon/Xeon Phis or the increases in communication throughput and recent advances in high-end features seen with modern interconnects like InfiniBand and Omni-Pat...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:International Conference for High Performance Computing, Networking, Storage and Analysis (Online) s. 1 - 11
Hlavní autoři: Bayatpour, Mohammadreza, Chakraborty, Sourav, Subramoni, Hari, Lu, Xiaoyi, Panda, Dhabaleswar K. (DK)
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: New York, NY, USA ACM 12.11.2017
Edice:ACM Conferences
Témata:
Analytical models
Collectives
Computer architecture
Computer systems organization > Architectures > Parallel architectures
Computing methodologies > Parallel computing methodologies > Parallel algorithms > Massively parallel algorithms
Costs
Data Partitioning
High performance computing
MPI
MPI_Allreduce
Multi-Leader
Organizations
Program processors
SHArP
Throughput
SHArP
data partitioning
collectives
multi-leader
MPI
MPI_allreduce
ISBN:9781450351140, 145035114X
ISSN:2167-4337
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Existing designs for MPI_Allreduce do not take advantage of the vast parallelism available in modern multi-/many-core processors like Intel Xeon/Xeon Phis or the increases in communication throughput and recent advances in high-end features seen with modern interconnects like InfiniBand and Omni-Path. In this paper, we propose a high-performance and scalable Data Partitioning-based Multi-Leader (DPML) solution for MPI_Allreduce that can take advantage of the parallelism offered by multi-/many-core architectures in conjunction with the high throughput and high-end features offered by InfiniBand and Omni-Path to significantly enhance the performance of MPI_Allreduce on modern HPC systems. We also model DPML-based designs to analyze the communication costs theoretically. Microbenchmark level evaluations show that the proposed DPML-based designs are able to deliver up to 3.5 times performance improvement for MPI_Allreduce for multiple HPC systems at scale. At the application-level, up to 35% and 60% improvement is seen in communication for HPCG and miniAMR respectively.
ISBN:9781450351140
145035114X
ISSN:2167-4337
DOI:10.1145/3126908.3126954