Integrating Asynchronous Task Parallelism with MPI

Effective combination of inter-node and intra-node parallelism is recognized to be a major challenge for future extreme-scale systems. Many researchers have demonstrated the potential benefits of combining both levels of parallelism, including increased communication-computation overlap, improved me...

Full description

Saved in:
Bibliographic Details
Published in:2013 IEEE 27th International Symposium on Parallel and Distributed Processing pp. 712 - 725
Main Authors: Chatterjee, Sanjay, Tasırlar, Sagnak, Budimlic, Zoran, Cave, Vincent, Chabbi, Milind, Grossman, Max, Sarkar, Vivek, Yonghong Yan
Format: Conference Proceeding
Language:English
Published: IEEE 01.05.2013
Subjects:
asynchronous task parallelism
Computational modeling
data flow
Data models
data-driven tasks
Electronics packaging
MPI
Parallel processing
phasers
Programming
Runtime
Synchronization
ISBN:146736066X, 9781467360661
ISSN:1530-2075
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Effective combination of inter-node and intra-node parallelism is recognized to be a major challenge for future extreme-scale systems. Many researchers have demonstrated the potential benefits of combining both levels of parallelism, including increased communication-computation overlap, improved memory utilization, and effective use of accelerators. However, current "hybrid programming" approaches often require significant rewrites of application code and assume a high level of programmer expertise. Dynamic task parallelism has been widely regarded as a programming model that combines the best of performance and programmability for shared-memory programs. For distributed-memory programs, most users rely on efficient implementations of MPI. In this paper, we propose HCMPI (Habanero-C MPI), an integration of the Habanero-C dynamic task-parallel programming model with the widely used MPI message-passing interface. All MPI calls are treated as asynchronous tasks in this model, thereby enabling unified handling of messages and tasking constructs. For programmers unfamiliar with MPI, we introduce distributed data-driven futures (DDDFs), a new data-flow programming model that seamlessly integrates intra-node and inter-node data-flow parallelism without requiring any knowledge of MPI. Our novel runtime design for HCMPI and DDDFs uses a combination of dedicated communication and computation specific worker threads. We evaluate our approach on a set of micro-benchmarks as well as larger applications and demonstrate better scalability compared to the most efficient MPI implementations, while offering a unified programming model to integrate asynchronous task parallelism with distributed-memory parallelism.
ISBN:146736066X
9781467360661
ISSN:1530-2075
DOI:10.1109/IPDPS.2013.78