Multi-level checkpointing and silent error detection for linear workflows

•A brand new section to deal with multi-level fail-stop errors.•A brand new section to deal with the combination of multilevel fail-stop errors.•Silent errors (with in memory checkpointing, and partial/guaranteed verifications). We focus on High Performance Computing (HPC) workflows whose dependency...

Full description

Saved in:
Bibliographic Details
Published in:Journal of computational science Vol. 28; pp. 398 - 415
Main Authors: Benoit, Anne, Cavelan, Aurélien, Robert, Yves, Sun, Hongyang
Format: Journal Article
Language:English
Published: Elsevier B.V 01.09.2018
Elsevier
Subjects:
Computer Science
Distributed, Parallel, and Cluster Computing
Dynamic programming algorithm
Error detection
Fail-stop errors
Fault-tolerance
Linear workflows
Multi-level checkpointing
Silent errors
Verification mechanism
Dynamic programming algorithm
Fail-stop errors
Multi-level checkpointing
Linear workflows
Error detection
Verification mechanism
Fault-tolerance
Silent errors
ISSN:1877-7503, 1877-7511
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•A brand new section to deal with multi-level fail-stop errors.•A brand new section to deal with the combination of multilevel fail-stop errors.•Silent errors (with in memory checkpointing, and partial/guaranteed verifications). We focus on High Performance Computing (HPC) workflows whose dependency graph forms a linear chain, and we extend single-level checkpointing in two important directions. Our first contribution targets silent errors, and combines in-memory checkpoints with both partial and guaranteed verifications. Our second contribution deals with multi-level checkpointing for fail-stop errors. We present sophisticated dynamic programming algorithms that return the optimal solution for each problem in polynomial time. We also show how to combine all these techniques and solve the problem with both fail-stop and silent errors. Simulation results demonstrate that these extensions lead to significantly improved performance compared to the standard single-level checkpointing algorithm.
ISSN:1877-7503
1877-7511
DOI:10.1016/j.jocs.2017.03.024