Integrating External Resources with a Task-Based Programming Model

Accessing external resources (e.g., loading input data, checkpointing snapshots, and out-of-core processing) can have a significant impact on the performance of supercomputer applications. However, no existing programming systems for high-performance computing directly manage and optimize these exte...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings, 24th IEEE International Conference on High Performance Computing : 18-21 December 2017, Jaipur, India pp. 307 - 316
Main Authors: Jia, Zhihao, Treichler, Sean, Shipman, Galen, Bauer, Michael, Watkins, Noah, Maltzahn, Carlos, McCormick, Patrick, Aiken, Alex
Format: Conference Proceeding
Language:English
Published: IEEE 01.12.2017
Subjects:
Coherence
Computational modeling
Deferred Execution
External Resource
I/O
Iris recognition
Programming
Runtime
Semantics
Task analysis
Task Based Programming Model
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Accessing external resources (e.g., loading input data, checkpointing snapshots, and out-of-core processing) can have a significant impact on the performance of supercomputer applications. However, no existing programming systems for high-performance computing directly manage and optimize these external accesses. As a result, users must explicitly manage external accesses alongside their computation at the application level, which can result in both correctness and performance issues. We address this limitation by introducing Iris, a task-based programming model with semantics for external resources. Iris allows applications to describe their access requirements to external resources and the relationship of those accesses to the computation. Iris incorporates external I/O into a deferred execution model, reschedules external I/O to overlap I/O with computation, and reduces external I/O when possible. We evaluate Iris on three microbenchmarks representative of important workloads in HPC and a full combustion simulation, S3D. We demonstrate that the Iris implementation of S3D reduces the external I/O overhead by up to 20×, compared to the Legion and the Fortran implementations.
DOI:10.1109/HiPC.2017.00043