AutoParallel: Automatic parallelisation and distributed execution of affine loop nests in Python.
Gespeichert in:
| Titel: | AutoParallel: Automatic parallelisation and distributed execution of affine loop nests in Python. |
|---|---|
| Autoren: | Ramon-Cortes, Cristian, Amela, Ramon, Ejarque, Jorge, Clauss, Philippe, Badia, Rosa M. |
| Quelle: | International Journal of High Performance Computing Applications; Nov2020, Vol. 34 Issue 6, p659-675, 17p |
| Schlagwörter: | PYTHON programming language, DISTRIBUTED computing, PARALLEL programming, PROGRAMMING languages, ALGORITHMS |
| Abstract: | The last improvements in programming languages and models have focused on simplicity and abstraction; leading Python to the top of the list of the programming languages. However, there is still room for improvement when preventing users from dealing directly with distributed and parallel computing issues. This paper proposes and evaluates AutoParallel, a Python module to automatically find an appropriate task-based parallelisation of affine loop nests and execute them in parallel in a distributed computing infrastructure. It is based on sequential programming and contains one single annotation (in the form of a Python decorator) so that anyone with intermediate-level programming skills can scale up an application to hundreds of cores. The evaluation demonstrates that AutoParallel goes one step further in easing the development of distributed applications. On the one hand, the programmability evaluation highlights the benefits of using a single Python decorator instead of manually annotating each task and its parameters or, even worse, having to develop the parallel code explicitly (e.g., using OpenMP, MPI). On the other hand, the performance evaluation demonstrates that AutoParallel is capable of automatically generating task-based workflows from sequential Python code while achieving the same performances than manually taskified versions of established state-of-the-art algorithms (i.e., Cholesky, LU, and QR decompositions). Finally, AutoParallel is also capable of automatically building data blocks to increase the tasks' granularity; freeing the user from creating the data chunks, and re-designing the algorithm. For advanced users, we believe that this feature can be useful as a baseline to design blocked algorithms. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of High Performance Computing Applications is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Datenbank: | Complementary Index |
Nájsť tento článok vo Web of Science Schreiben Sie den ersten Kommentar!