On the Parallel I/O Optimality of Linear Algebra Kernels: Near-Optimal Matrix Factorizations

Matrix factorizations are among the most important building blocks of scientific computing. However, state-of-the-art libraries are not communication-optimal, underutilizing current parallel architectures. We present novel algorithms for Cholesky and LU factorizations that utilize an asymptotically...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:SC21: International Conference for High Performance Computing, Networking, Storage and Analysis s. 1 - 15
Hlavní autoři: Kwasniewski, Grzegorz, Kabic, Marko, Ben-Nun, Tal, Ziogas, Alexandros Nikolaos, Saethre, Jens Eirik, Gaillard, Andre, Schneider, Timo, Besta, Maciej, Kozhevnikov, Anton, VandeVondele, Joost, Hoefler, Torsten
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: ACM 14.11.2021
Témata:
Codes
communication complexity
Distributed linear algebra algorithms
Layout
Libraries
matrix factorization
Schedules
Scientific computing
Supercomputers
Three-dimensional displays
ISSN:2167-4337
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Matrix factorizations are among the most important building blocks of scientific computing. However, state-of-the-art libraries are not communication-optimal, underutilizing current parallel architectures. We present novel algorithms for Cholesky and LU factorizations that utilize an asymptotically communication-optimal 2.5D decomposition. We first establish a theoretical framework for deriving parallel I/O lower bounds for linear algebra kernels, and then utilize its insights to derive Cholesky and LU schedules, both communicating N^{3}/(P\sqrt{M}) elements per processor, where M is the local memory size. The empirical results match our theoretical analysis: our implementations communicate significantly less than Intel MKL, SLATE, and the asymptotically communication-optimal CANDMC and CAPITAL libraries. Our code outperforms these state-of-the-art libraries in almost all tested scenarios, with matrix sizes ranging from 2,048 to 524,288 on up to 512 CPU nodes of the Piz Daint supercomputer, decreasing the time-to-solution by up to three times. Our code is ScaLAPAck-compatible and available as an open-source library.
ISSN:2167-4337
DOI:10.1145/3458817.3476167