Performance Meets Programmabilty: Enabling Native Python MPI Tasks In PyCOMPSs

The increasing complexity of modern and future computing systems makes it challenging to develop applications that aim for maximum performance. Hybrid parallel programming models offer new ways to exploit the capabilities of the underlying infrastructure. However, the performance gain is sometimes a...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings - Euromicro Workshop on Parallel and Distributed Processing pp. 63 - 66
Main Authors: Elshazly, Hatem, Lordan, Francesc, Ejarque, Jorge, Badia, Rosa M.
Format: Conference Proceeding
Language:English
Published: IEEE 01.03.2020
Subjects:
Computational modeling
Distributed Computing
High Performance Computing
Hybrid Programming Models
MPI
Parallel processing
Parallel programming
Productivity
Python
Runtime
Task analysis
Task-based Parallel Programming Models
ISSN:2377-5750
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The increasing complexity of modern and future computing systems makes it challenging to develop applications that aim for maximum performance. Hybrid parallel programming models offer new ways to exploit the capabilities of the underlying infrastructure. However, the performance gain is sometimes accompanied by increased programming complexity. We introduce an extension to PyCOMPSs, a high-level task-based parallel programming model for Python applications, to support tasks that use MPI natively as part of the task model. Without compromising application's programmability, using Native MPI tasks in PyCOMPSs offers up to 3x improvement in total performance for compute intensive applications and up to 1.9x improvement in total performance for I/O intensive applications over sequential implementation of the tasks.
ISSN:2377-5750
DOI:10.1109/PDP50117.2020.00016