Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method

Pipelined Krylov subspace methods (also referred to as communication-hiding methods) have been proposed in the literature as a scalable alternative to classic Krylov subspace algorithms for iteratively computing the solution to a large linear system in parallel. For symmetric and positive definite s...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on parallel and distributed systems Jg. 30; H. 11; S. 2507 - 2522
Hauptverfasser:	Cools, Siegfried, Cornelis, Jeffrey, Vanroose, Wim
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 01.11.2019 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Algorithms attainable accuracy Communication Conjugate gradient method conjugate gradients Conjugates Distributed memory Distributed processing Global communication Gradient methods Hardware inexact computations Krylov subspace methods latency hiding Mathematical analysis Matrix algebra Matrix methods Methods Numerical stability parallel performance Pipeline processing pipelining Pipelining (computers) Rounding Stability Subspace methods Subspaces Symmetric matrices
ISSN:	1045-9219, 1558-2183
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	Pipelined Krylov subspace methods (also referred to as communication-hiding methods) have been proposed in the literature as a scalable alternative to classic Krylov subspace algorithms for iteratively computing the solution to a large linear system in parallel. For symmetric and positive definite system matrices the pipelined Conjugate Gradient method, p(ll)-CG, outperforms its classic Conjugate Gradient counterpart on large scale distributed memory hardware by overlapping global communication with essential computations like the matrix-vector product, thus "hiding" global communication. A well-known drawback of the pipelining technique is the (possibly significant) loss of numerical stability. In this work a numerically stable variant of the pipelined Conjugate Gradient algorithm is presented that avoids the propagation of local rounding errors in the finite precision recurrence relations that construct the Krylov subspace basis. The multi-term recurrence relation for the basis vector is replaced by ℓ three-term recurrences, improving stability without increasing the overall computational cost of the algorithm. The proposed modification ensures that the pipelined Conjugate Gradient method is able to attain a highly accurate solution independently of the pipeline length. Numerical experiments demonstrate a combination of excellent parallel performance and improved maximal attainable accuracy for the new pipelined Conjugate Gradient algorithm. This work thus resolves one of the major practical restrictions for the useability of pipelined Krylov subspace methods.
AbstractList	Pipelined Krylov subspace methods (also referred to as communication-hiding methods) have been proposed in the literature as a scalable alternative to classic Krylov subspace algorithms for iteratively computing the solution to a large linear system in parallel. For symmetric and positive definite system matrices the pipelined Conjugate Gradient method, p(ll)-CG, outperforms its classic Conjugate Gradient counterpart on large scale distributed memory hardware by overlapping global communication with essential computations like the matrix-vector product, thus "hiding" global communication. A well-known drawback of the pipelining technique is the (possibly significant) loss of numerical stability. In this work a numerically stable variant of the pipelined Conjugate Gradient algorithm is presented that avoids the propagation of local rounding errors in the finite precision recurrence relations that construct the Krylov subspace basis. The multi-term recurrence relation for the basis vector is replaced by ℓ three-term recurrences, improving stability without increasing the overall computational cost of the algorithm. The proposed modification ensures that the pipelined Conjugate Gradient method is able to attain a highly accurate solution independently of the pipeline length. Numerical experiments demonstrate a combination of excellent parallel performance and improved maximal attainable accuracy for the new pipelined Conjugate Gradient algorithm. This work thus resolves one of the major practical restrictions for the useability of pipelined Krylov subspace methods. Pipelined Krylov subspace methods (also referred to as communication-hiding methods) have been proposed in the literature as a scalable alternative to classic Krylov subspace algorithms for iteratively computing the solution to a large linear system in parallel. For symmetric and positive definite system matrices the pipelined Conjugate Gradient method, p($l$l)-CG, outperforms its classic Conjugate Gradient counterpart on large scale distributed memory hardware by overlapping global communication with essential computations like the matrix-vector product, thus “hiding” global communication. A well-known drawback of the pipelining technique is the (possibly significant) loss of numerical stability. In this work a numerically stable variant of the pipelined Conjugate Gradient algorithm is presented that avoids the propagation of local rounding errors in the finite precision recurrence relations that construct the Krylov subspace basis. The multi-term recurrence relation for the basis vector is replaced by $\ell$ℓ three-term recurrences, improving stability without increasing the overall computational cost of the algorithm. The proposed modification ensures that the pipelined Conjugate Gradient method is able to attain a highly accurate solution independently of the pipeline length. Numerical experiments demonstrate a combination of excellent parallel performance and improved maximal attainable accuracy for the new pipelined Conjugate Gradient algorithm. This work thus resolves one of the major practical restrictions for the useability of pipelined Krylov subspace methods.
Author	Cools, Siegfried Vanroose, Wim Cornelis, Jeffrey
Author_xml	– sequence: 1 givenname: Siegfried orcidid: 0000-0001-7065-4729 surname: Cools fullname: Cools, Siegfried email: siegfried.cools@uantwerp.be organization: Department of Mathematics and Computer Science, Applied Mathematics Group, University of Antwerp, Antwerp, Belgium – sequence: 2 givenname: Jeffrey surname: Cornelis fullname: Cornelis, Jeffrey email: jeffrey.cornelis@uantwerp.be organization: Department of Mathematics and Computer Science, Applied Mathematics Group, University of Antwerp, Antwerp, Belgium – sequence: 3 givenname: Wim surname: Vanroose fullname: Vanroose, Wim email: wim.vanroose@uantwerp.be organization: Department of Mathematics and Computer Science, Applied Mathematics Group, University of Antwerp, Antwerp, Belgium
BookMark	eNp9kU1PwzAMhiMEEp8_AHGJxLkjbpq2OaIBGxJfYuNcJanHMrXJSNMD_56WIQ4cONmy38eWXx-TfecdEnIObALA5NXy5WYxSRnISSqhyHO-R45AiDJJoeT7Q84ykcgU5CE57roNY5AJlh2R9VPfYrBGNc0nXUSlG6SvaPoQ0JkxbVS03nV05QONa6RT37a9G4CxTOe2tu6dvtgtNtZhPbTdpn9XEeksqNqii_QR49rXp-RgpZoOz37iCXm7u11O58nD8-x-ev2QmFTymNQGGIAuMuRCS61qlgpUtRa1yfNcap2KrESTSSNKw4qMaWOKnCnNGZdcCn5CLndzt8F_9NjFauP74IaVVTpqCikBBlWxU5nguy7gqjI2fp8Ug7JNBawaba1GW6vR1urH1oGEP-Q22FaFz3-Zix1jEfFXXxbDa4DzL3ZEhmY
CODEN	ITDSEO
CitedBy_id	crossref_primary_10_1137_19M1276856 crossref_primary_10_1016_j_laa_2022_01_004 crossref_primary_10_3390_su13147933 crossref_primary_10_1016_j_cie_2022_108656 crossref_primary_10_1007_s40314_025_03243_6 crossref_primary_10_1109_TPDS_2021_3084104 crossref_primary_10_1007_s40314_024_02867_4 crossref_primary_10_1007_s40314_025_03390_w crossref_primary_10_1016_j_jfranklin_2025_107870 crossref_primary_10_3390_app112110102 crossref_primary_10_1137_20M1346249
Cites_doi	10.1137/120893057 10.1137/S1064827599353865 10.1177/1094342010391989 10.1137/12086563X 10.1137/17M1117872 10.1016/0024-3795(80)90167-6 10.1016/0167-8191(87)90037-8 10.1137/1.9780898718027 10.1109/SC.2016.17 10.1016/0377-0427(89)90045-9 10.1137/16M1103361 10.1137/S0895479897323087 10.2172/7172467 10.1002/nla.643 10.1137/15M1049130 10.1093/imamat/18.3.341 10.1137/1.9780898718003 10.1017/S096249290626001X 10.1137/S0895479895284944 10.1093/acprof:oso/9780199655410.001.0001 10.1137/110834512 10.6028/jres.049.044 10.1137/S0895479897331862 10.1016/j.parco.2017.04.005 10.1016/j.parco.2013.06.001 10.1137/0613011 10.1137/140989492 10.1088/1742-6596/78/1/012066 10.1137/120881191 10.1007/s10543-005-0032-1 10.1017/S096249290000235X 10.1016/j.parco.2019.05.002 10.1145/3079079.3079091 10.1016/0024-3795(89)90285-1 10.1007/BF02309342 10.1016/j.parco.2013.10.001 10.1016/0167-8191(87)90051-2 10.1109/IPDPSW.2017.65 10.1017/CBO9780511615115
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/TPDS.2019.2917663
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Computer Science
EISSN	1558-2183
EndPage	2522
ExternalDocumentID	10_1109_TPDS_2019_2917663 8718313
Genre	orig-research
GrantInformation_xml	– fundername: University of Antwerp Research Council – fundername: Flemish Research Foundation (FWO Flanders) grantid: 12H4617N – fundername: University Research Fund
GroupedDBID	--Z -~X .DC 0R~ 29I 4.4 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ IEDLZ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS TN5 TWZ UHB AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c293t-dc1011b74e35b9bad025eadb5dc6669bb2548ec49c58c0740bcc760ab30393953
IEDL.DBID	RIE
ISICitedReferencesCount	20
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000492450900010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1045-9219
IngestDate	Sun Nov 30 05:23:52 EST 2025 Tue Nov 18 21:52:23 EST 2025 Sat Nov 29 06:06:47 EST 2025 Wed Aug 27 02:44:46 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	11
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c293t-dc1011b74e35b9bad025eadb5dc6669bb2548ec49c58c0740bcc760ab30393953
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0001-7065-4729
PQID	2303979911
PQPubID	85437
PageCount	16
ParticipantIDs	ieee_primary_8718313 crossref_primary_10_1109_TPDS_2019_2917663 crossref_citationtrail_10_1109_TPDS_2019_2917663 proquest_journals_2303979911
PublicationCentury	2000
PublicationDate	2019-11-01
PublicationDateYYYYMMDD	2019-11-01
PublicationDate_xml	– month: 11 year: 2019 text: 2019-11-01 day: 01
PublicationDecade	2010
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on parallel and distributed systems
PublicationTitleAbbrev	TPDS
PublicationYear	2019
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref35 ref13 ref34 ref12 ref37 ref36 ref14 ref31 ref30 ref33 ref32 ref10 cornelis (ref11) 2018 ref2 ref39 ref38 ref16 ref19 ref18 erhel (ref17) 1995; 3 balay (ref1) 2015 wilkinson (ref45) 1994 ref46 ref24 ref23 ref26 ref47 ref25 ref20 ref41 ref22 ref44 ref21 ref43 hoemmen (ref27) 2010 ref29 ref8 ref7 ref9 ref4 dongarra (ref15) 2015 ref3 ref6 ref5 ref40 imberti (ref28) 2017; 47 swirydowicz (ref42) 2018
References_xml	– ident: ref3 doi: 10.1137/120893057 – ident: ref44 doi: 10.1137/S1064827599353865 – ident: ref14 doi: 10.1177/1094342010391989 – ident: ref18 doi: 10.1137/12086563X – ident: ref10 doi: 10.1137/17M1117872 – year: 2018 ident: ref11 article-title: The communication-hiding conjugate gradient method with deep pipelines publication-title: Submitted to SIAM J Sci Comput – ident: ref34 doi: 10.1016/0024-3795(80)90167-6 – ident: ref31 doi: 10.1016/0167-8191(87)90037-8 – ident: ref26 doi: 10.1137/1.9780898718027 – ident: ref16 doi: 10.1109/SC.2016.17 – year: 1994 ident: ref45 publication-title: Rounding Errors in Algebraic Processes – ident: ref6 doi: 10.1016/0377-0427(89)90045-9 – ident: ref5 doi: 10.1137/16M1103361 – ident: ref39 doi: 10.1137/S0895479897323087 – ident: ref12 doi: 10.2172/7172467 – ident: ref7 doi: 10.1002/nla.643 – ident: ref36 doi: 10.1137/15M1049130 – year: 2015 ident: ref1 article-title: PETSc Web page – ident: ref33 doi: 10.1093/imamat/18.3.341 – ident: ref35 doi: 10.1137/1.9780898718003 – ident: ref32 doi: 10.1017/S096249290626001X – ident: ref21 doi: 10.1137/S0895479895284944 – ident: ref29 doi: 10.1093/acprof:oso/9780199655410.001.0001 – ident: ref2 doi: 10.1137/110834512 – ident: ref25 doi: 10.6028/jres.049.044 – year: 2018 ident: ref42 article-title: Low synchronization GMRES algorithms publication-title: Proceedings of the Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems – volume: 47 start-page: 206 year: 2017 ident: ref28 article-title: Varying the s in your s-step GMRES publication-title: Electron Trans Numerical Anal – ident: ref24 doi: 10.1137/S0895479897331862 – ident: ref9 doi: 10.1016/j.parco.2017.04.005 – ident: ref19 doi: 10.1016/j.parco.2013.06.001 – ident: ref22 doi: 10.1137/0613011 – ident: ref23 doi: 10.1137/140989492 – year: 2015 ident: ref15 article-title: HPCG benchmark: A new metric for ranking high performance computing systems – volume: 3 start-page: 160 year: 1995 ident: ref17 article-title: A parallel GMRES version for general sparse matrices publication-title: Electron Trans Numerical Anal – ident: ref37 doi: 10.1088/1742-6596/78/1/012066 – ident: ref4 doi: 10.1137/120881191 – ident: ref41 doi: 10.1007/s10543-005-0032-1 – ident: ref13 doi: 10.1017/S096249290000235X – ident: ref8 doi: 10.1016/j.parco.2019.05.002 – ident: ref47 doi: 10.1145/3079079.3079091 – year: 2010 ident: ref27 article-title: Communication-avoiding Krylov subspace methods – ident: ref20 doi: 10.1016/0024-3795(89)90285-1 – ident: ref38 doi: 10.1007/BF02309342 – ident: ref30 doi: 10.1016/j.parco.2013.10.001 – ident: ref40 doi: 10.1016/0167-8191(87)90051-2 – ident: ref46 doi: 10.1109/IPDPSW.2017.65 – ident: ref43 doi: 10.1017/CBO9780511615115
SSID	ssj0014504
Score	2.39489
Snippet	Pipelined Krylov subspace methods (also referred to as communication-hiding methods) have been proposed in the literature as a scalable alternative to classic...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	2507
SubjectTerms	Algorithms attainable accuracy Communication Conjugate gradient method conjugate gradients Conjugates Distributed memory Distributed processing Global communication Gradient methods Hardware inexact computations Krylov subspace methods latency hiding Mathematical analysis Matrix algebra Matrix methods Methods Numerical stability parallel performance Pipeline processing pipelining Pipelining (computers) Rounding Stability Subspace methods Subspaces Symmetric matrices
Title	Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method
URI	https://ieeexplore.ieee.org/document/8718313 https://www.proquest.com/docview/2303979911
Volume	30
WOSCitedRecordID	wos000492450900010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2183 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014504 issn: 1045-9219 databaseCode: RIE dateStart: 19900101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB5q8aAHq61itcoePIlp0zya7FF89aClaIXeQnazwUpIS5sI_ntnNmmwKIK3QHaTJd9O5pudF8CFROnjoa5wGaOBgozDEMh0jUjZrrBi6YVF15JHbzTyp1M-rsFVlQujlNLBZ6pLl9qXH81lTkdlPST3vk0tarc8zytytSqPAb7GKSoPuAZHMSw9mH2T9ybj2xcK4uJdi1M9RHtDB-mmKj_-xFq93Df-t7B92CtpJLsucD-Amkqb0Fi3aGClxDZh91u9wRa8jfLCQZMknwxppkgUe6YDd53yx6rAOIZMliEzZBvpI2w4I0XHxrMFJbGrCG-n7zmdw7GHpQ4dy9iT7kh9CK_3d5OboVG2WjAk6vvMiCSKZl94DkHERRghFcI9JtxIon3DhUA70lfS4dL1JbIOU0jpDcxQ2JTby137COrpPFXHwAahq0wzDp2-iB0RxwIpo6NExAfcl64VtsFcf_xAlnXIqR1GEmh7xOQB4RUQXkGJVxsuqymLogjHX4NbBFA1sMSmDZ01wkEppqvAouV7SJH7J7_POoUdenaRfNiBerbM1Rlsy49stlqe6x34BXUO2Z0
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB6KCurBaqtYrboHT2LaPJvsUdRasS1FK_QWspsNVkpb-hD8985s0mBRBG-B7LJLvp3MNzsvgEuJ0scjXeEyQQMFGYchkOkasXI8YSfSj9KuJW2_2w0GA94rwHWeC6OU0sFnqkaP2pcfT-SSrsrqSO4Dh1rUbnqua1tptlbuM8CF3LT2gGdwFMTMh2mZvN7v3b1QGBev2ZwqIjprWki3VfnxL9YKpln839b2YS8jkuwmRf4ACmpcguKqSQPLZLYEu98qDpbhrbtMXTSj0SdDoilGij3TlbtO-mN5aBxDLsuQG7K1BBLWGpKqY73hlNLYVYyvx-9LuoljDzMdPLZgHd2T-hBem_f925aRNVswJGr8hRFLFE5L-C6BxEUUIxnCUya8WKKFw4VASzJQ0uXSCyTyDlNI6TfMSDiU3cs95wg2xpOxOgbWiDxlmknkWiJxRZIIJI2uEjFv8EB6dlQBc_XxQ5lVIqeGGKNQWyQmDwmvkPAKM7wqcJVPmaZlOP4aXCaA8oEZNhWorhAOM0GdhzZt30eSbJ38PusCtlv9TjtsP3afTmGH1klTEauwsZgt1RlsyY_FcD4716fxC-uB3OQ
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Numerically+Stable+Recurrence+Relations+for+the+Communication+Hiding+Pipelined+Conjugate+Gradient+Method&rft.jtitle=IEEE+transactions+on+parallel+and+distributed+systems&rft.au=Cools%2C+Siegfried&rft.au=Cornelis%2C+Jeffrey&rft.au=Vanroose%2C+Wim&rft.date=2019-11-01&rft.issn=1045-9219&rft.eissn=1558-2183&rft.volume=30&rft.issue=11&rft.spage=2507&rft.epage=2522&rft_id=info:doi/10.1109%2FTPDS.2019.2917663&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TPDS_2019_2917663
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1045-9219&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1045-9219&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1045-9219&client=summon