Scalable Resilience Against Node Failures for Communication-Hiding Preconditioned Conjugate Gradient and Conjugate Residual Methods

The observed and expected continued growth in the number of nodes in large-scale parallel computers gives rise to two major challenges: global communication operations are becoming major bottlenecks due to their limited scalability, and the likelihood of node failures is increasing. We study an appr...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org
Main Authors: Levonyak, Markus, Pacher, Christina, Gansterer, Wilfried N
Format: Paper
Language:English
Published: Ithaca Cornell University Library, arXiv.org 19.12.2019
Subjects:
Algorithms
Communication
Conjugates
Linear systems
Nodes
Parallel computers
Resilience
ISSN:2331-8422
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract The observed and expected continued growth in the number of nodes in large-scale parallel computers gives rise to two major challenges: global communication operations are becoming major bottlenecks due to their limited scalability, and the likelihood of node failures is increasing. We study an approach for addressing these challenges in the context of solving large sparse linear systems. In particular, we focus on the pipelined preconditioned conjugate gradient (PPCG) method, which has been shown to successfully deal with the first of these challenges. In this paper, we address the second challenge. We present extensions to the PPCG solver and two of its variants which make them resilient against the failure of a compute node while fully preserving their communication-hiding properties and thus their scalability. The basic idea is to efficiently communicate a few redundant copies of local vector elements to neighboring nodes with very little overhead. In case a node fails, these redundant copies are gathered at a replacement node, which can then accurately reconstruct the lost parts of the solver's state. After that, the parallel solver can continue as in the failure-free scenario. Experimental evaluations of our approach illustrate on average very low runtime overheads compared to the standard non-resilient algorithms. This shows that scalable algorithmic resilience can be achieved at low extra cost.
AbstractList The observed and expected continued growth in the number of nodes in large-scale parallel computers gives rise to two major challenges: global communication operations are becoming major bottlenecks due to their limited scalability, and the likelihood of node failures is increasing. We study an approach for addressing these challenges in the context of solving large sparse linear systems. In particular, we focus on the pipelined preconditioned conjugate gradient (PPCG) method, which has been shown to successfully deal with the first of these challenges. In this paper, we address the second challenge. We present extensions to the PPCG solver and two of its variants which make them resilient against the failure of a compute node while fully preserving their communication-hiding properties and thus their scalability. The basic idea is to efficiently communicate a few redundant copies of local vector elements to neighboring nodes with very little overhead. In case a node fails, these redundant copies are gathered at a replacement node, which can then accurately reconstruct the lost parts of the solver's state. After that, the parallel solver can continue as in the failure-free scenario. Experimental evaluations of our approach illustrate on average very low runtime overheads compared to the standard non-resilient algorithms. This shows that scalable algorithmic resilience can be achieved at low extra cost.
Author Pacher, Christina
Gansterer, Wilfried N
Levonyak, Markus
Author_xml – sequence: 1
  givenname: Markus
  surname: Levonyak
  fullname: Levonyak, Markus
– sequence: 2
  givenname: Christina
  surname: Pacher
  fullname: Pacher, Christina
– sequence: 3
  givenname: Wilfried
  surname: Gansterer
  middlename: N
  fullname: Gansterer, Wilfried N
BookMark eNpNj09LAzEUxIMoWGs_gLeA563Zl-xmcyzFtkL9g_Ze3iavNWWb6GZXvPvFXdGDp2GGYX7MBTsNMRBjV7mYqqooxA22n_5jmpscpsKAFCdsBFLmWaUAztkkpYMQAkoNRSFH7OvFYoN1Q_yZkm88BUt8tkcfUscfoiO-QN_0LSW-iy2fx-OxD95i52PIVt75sOdPLdkYnP_JyA2dcOj32BFftuiGxY5j-B__kFyPDb-n7jW6dMnOdtgkmvzpmG0Wt5v5Kls_Lu_ms3WGBUCmc2UJynIwWomds65WQkpT1ICotSkI0A7Pdqo02qBy2tQVKJ3XOrdCSzlm17-zb2187yl120Ps2zAQtyChMhVIDfIb5ZVklw
ContentType Paper
Copyright 2019. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2019. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.48550/arxiv.1912.09230
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials - QC
ProQuest Central
Technology Collection
ProQuest One
ProQuest Central
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
ProQuest Central Premium
ProQuest One Academic (New)
Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-LOGICAL-a522-714ce266a52740fdcdb403395b2aa7795e2ac002f46979a4d79b82471b71c0733
IEDL.DBID BENPR
IngestDate Mon Jun 30 09:18:59 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a522-714ce266a52740fdcdb403395b2aa7795e2ac002f46979a4d79b82471b71c0733
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/docview/2328982372?pq-origsite=%requestingapplication%
PQID 2328982372
PQPubID 2050157
ParticipantIDs proquest_journals_2328982372
PublicationCentury 2000
PublicationDate 20191219
PublicationDateYYYYMMDD 2019-12-19
PublicationDate_xml – month: 12
  year: 2019
  text: 20191219
  day: 19
PublicationDecade 2010
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2019
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.7086289
SecondaryResourceType preprint
Snippet The observed and expected continued growth in the number of nodes in large-scale parallel computers gives rise to two major challenges: global communication...
SourceID proquest
SourceType Aggregation Database
SubjectTerms Algorithms
Communication
Conjugates
Linear systems
Nodes
Parallel computers
Resilience
Title Scalable Resilience Against Node Failures for Communication-Hiding Preconditioned Conjugate Gradient and Conjugate Residual Methods
URI https://www.proquest.com/docview/2328982372
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3dS8MwEA-6KfjkN37MkQdfs7VptjRPorKp4ErZhsynkSatVKSb7TZ89x_3knV-IPjiW9MWGi7N3e8ud_dD6Fw5QoFdkIR5ShPW9hSRgCNgQbQvWEQVtY3nH-55EPijkQjLgFtRplWudKJV1HqiTIy8CZbfF6azCr2YvhLDGmVOV0sKjXVUNZ3KWAVVrzpB2P-MstA2B8zsLY8zbfOupszf0kUD3BTacGBWzi8lbC1Ld_u_c9pB1VBO43wXrcXZHtq0GZ2q2EfvA5C-qYvC_bhIX-wOxpdPMgU8iIOJjnFXpiYnvcCAW_GPQhFymxqThkPrLutlOyMN72TPcxN2wze5zRSbYZl9v22-ZGq7cM_yUhcHaNjtDK9vScm4QCTgMMJdpmKw2DDgzEm00hFzPE-0Iiol56IVU6lAqAn41FxIprmIfArmLeKuMuyPh6iSwYSOEE5cqqNEMqoSzQAyAAqSOokEXPiuUu4xqq1EOi53TTH-kufJ349P0RYAF8vj4IoaqszyeXyGNtRilhZ5vfwJ6iaPcwCj8K4XPn4Arr_BNg
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3JTsMwEB0BBcGJXez4AMdA4rh1fEAIAaVVF1VQIW6VYycoCKUlaVnOfA__yNhtWYTEjQO3bEocj_XmzXgWgD3lCoV6QTrMV9phJV85EnkECkQHgoVUUVt4_rrOm83g5ka0JuBtnAtjwirHmGiBWneV8ZEfouYPhKmsQo97D47pGmV2V8ctNIbLoha9PKHJlh9Vz1C--5SWz9unFWfUVcCRyDUc7jEVoVbCE87cWCsdMtf3RTGkUnIuihGVCmEiRruRC8k0F2FAEcJD7inT4RBfOwkFZsDfRgpefbh0aIkjQfeHe6e2UtihzJ6TxwO0ieiBi1Pg_kB8q8bK8_9sAhag0JK9KFuEiShdghkbraryZXi9wpVlcr7IZZQn9xadyMmtTJDrkmZXR6QsExNvnxPk5ORbEoxTSYy6Ji3rCtDDUk0an0nvBsalSC4yGwXXJzL9etl8yeStkYbtuZ2vQPsvfnsVplIc0BqQ2KM6jCWjKtYM6RAyPKnjUOBB4CnlrcPWWIKdESLknU_xbfx-exdmK-1GvVOvNmubMIcEzfar8MQWTPWzQbQN0-qxn-TZjl19BDp_LOx3wp0WLg
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Scalable+Resilience+Against+Node+Failures+for+Communication-Hiding+Preconditioned+Conjugate+Gradient+and+Conjugate+Residual+Methods&rft.jtitle=arXiv.org&rft.au=Levonyak%2C+Markus&rft.au=Pacher%2C+Christina&rft.au=Gansterer%2C+Wilfried+N&rft.date=2019-12-19&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.1912.09230