MPI detach — Towards automatic asynchronous local completion

When aiming for large-scale parallel computing, waiting time due to network latency, synchronization, and load imbalance are the primary opponents of high parallel efficiency. A common approach to hide latency with computation is the use of non-blocking communication. In the presence of a consistent...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Parallel computing Ročník 109; s. 102859
Hlavní autori: Protze, Joachim, Hermanns, Marc-André, Müller, Matthias S., Nguyen, Van Man, Jaeger, Julien, Saillard, Emmanuelle, Carribault, Patrick, Barthou, Denis
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier B.V 01.03.2022
Elsevier
Predmet:
Asynchronous communication
Code transformation
Computer Science
Distributed, Parallel, and Cluster Computing
Hybrid parallelism
Message Passing Interface
OpenMP tasking
Static analysis
Asynchronous communication
OpenMP tasking
Hybrid parallelism
Message Passing Interface
Code transformation
Static analysis
ISSN:0167-8191, 1872-7336
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:When aiming for large-scale parallel computing, waiting time due to network latency, synchronization, and load imbalance are the primary opponents of high parallel efficiency. A common approach to hide latency with computation is the use of non-blocking communication. In the presence of a consistent load imbalance, synchronization cost is just the visible symptom of the load imbalance. Tasking approaches as in OpenMP, TBB, OmpSs, or C++20 coroutines promise to expose a higher degree of concurrency, which can be distributed on available execution units and significantly increase load balance. Available MPI non-blocking functionality does not integrate seamlessly into such tasking parallelization. In this work, we present a slim extension of the MPI interface to allow seamless integration of non-blocking communication with available concepts of asynchronous execution in OpenMP and C++. Using our concept allows to span task dependency graphs for asynchronous execution over the full distributed memory application. We furthermore investigate compile-time analysis necessary to transform an application using blocking MPI communication into an application integrating OpenMP tasks with our proposed MPI interface extension. •MPI interface extensions to transfer request completion back to the MPI library.•callback-driven notification of asynchronous completion back to the application.•prototype implementation of the interface independent of the MPI implementation.•integration of MPI communication into OpenMP task programming.•compile-time analysis to convert blocking communication into non-blocking.
ISSN:0167-8191
1872-7336
DOI:10.1016/j.parco.2021.102859