Exploiting Multiple Levels of Parallelism in Sparse Matrix-Matrix Multiplication

Sparse matrix-matrix multiplication (or SpGEMM) is a key primitive for many high-performance graph algorithms as well as for some linear solvers, such as algebraic multigrid. The scaling of existing parallel implementations of SpGEMM is heavily bound by communication. Even though 3D (or 2.5D) algori...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org
Main Authors: Azad, Ariful, Ballard, Grey, Aydin Buluc, Demmel, James, Grigori, Laura, Schwartz, Oded, Toledo, Sivan, Williams, Samuel
Format: Paper
Language:English
Published: Ithaca Cornell University Library, arXiv.org 16.11.2016
Subjects:
Algorithms
Matrices (mathematics)
Multiplication
Solvers
Sparsity
ISSN:2331-8422
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Sparse matrix-matrix multiplication (or SpGEMM) is a key primitive for many high-performance graph algorithms as well as for some linear solvers, such as algebraic multigrid. The scaling of existing parallel implementations of SpGEMM is heavily bound by communication. Even though 3D (or 2.5D) algorithms have been proposed and theoretically analyzed in the flat MPI model on Erdos-Renyi matrices, those algorithms had not been implemented in practice and their complexities had not been analyzed for the general case. In this work, we present the first ever implementation of the 3D SpGEMM formulation that also exploits multiple (intra-node and inter-node) levels of parallelism, achieving significant speedups over the state-of-the-art publicly available codes at all levels of concurrencies. We extensively evaluate our implementation and identify bottlenecks that should be subject to further research.
AbstractList Sparse matrix-matrix multiplication (or SpGEMM) is a key primitive for many high-performance graph algorithms as well as for some linear solvers, such as algebraic multigrid. The scaling of existing parallel implementations of SpGEMM is heavily bound by communication. Even though 3D (or 2.5D) algorithms have been proposed and theoretically analyzed in the flat MPI model on Erdos-Renyi matrices, those algorithms had not been implemented in practice and their complexities had not been analyzed for the general case. In this work, we present the first ever implementation of the 3D SpGEMM formulation that also exploits multiple (intra-node and inter-node) levels of parallelism, achieving significant speedups over the state-of-the-art publicly available codes at all levels of concurrencies. We extensively evaluate our implementation and identify bottlenecks that should be subject to further research.
Author Azad, Ariful
Toledo, Sivan
Williams, Samuel
Schwartz, Oded
Aydin Buluc
Ballard, Grey
Demmel, James
Grigori, Laura
Author_xml – sequence: 1
  givenname: Ariful
  surname: Azad
  fullname: Azad, Ariful
– sequence: 2
  givenname: Grey
  surname: Ballard
  fullname: Ballard, Grey
– sequence: 3
  fullname: Aydin Buluc
– sequence: 4
  givenname: James
  surname: Demmel
  fullname: Demmel, James
– sequence: 5
  givenname: Laura
  surname: Grigori
  fullname: Grigori, Laura
– sequence: 6
  givenname: Oded
  surname: Schwartz
  fullname: Schwartz, Oded
– sequence: 7
  givenname: Sivan
  surname: Toledo
  fullname: Toledo, Sivan
– sequence: 8
  givenname: Samuel
  surname: Williams
  fullname: Williams, Samuel
BookMark eNo1jV1LwzAYRoMoOOd-gHcBrzvffKeXMuYHdDhw9-Ntm0hGbGvSjf58B9OrAw-c59yR667vHCEPDJbSKgVPmKZwWjJ1HgCslFdkxoVghZWc35JFzgcA4NpwpcSMbNfTEPswhu6Lbo5xDEN0tHInFzPtPd1iwhhdDPmbho5-DpiyoxscU5iKC_610OAY-u6e3HiM2S3-OCe7l_Vu9VZUH6_vq-eqQMVF0dS8Zcq60liUYJRpnWhEi76sW-W5kSgVYltrqb1quNBlLZhnqjQtGA9WzMnj5XZI_c_R5XF_6I-pOxf3HCwDrTUI8QtcTlLf
ContentType Paper
Copyright 2016. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2016. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.48550/arxiv.1510.00844
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
ProQuest SciTech Premium Collection Technology Collection Materials Science & Engineering Database
ProQuest Central (Alumni Edition)
ProQuest Central UK/Ireland
ProQuest Central Essentials
ProQuest Central
ProQuest Technology Collection
ProQuest One Community College
ProQuest Central Korea
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
Proquest Central Premium
ProQuest One Academic (New)
Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-LOGICAL-a523-cb2d158e978a40757de3c3daf9bd5f274a45aadb646f5c2369b31f1597d07f083
IEDL.DBID M7S
IngestDate Mon Jun 30 09:29:19 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a523-cb2d158e978a40757de3c3daf9bd5f274a45aadb646f5c2369b31f1597d07f083
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/docview/2081066603?pq-origsite=%requestingapplication%
PQID 2081066603
PQPubID 2050157
ParticipantIDs proquest_journals_2081066603
PublicationCentury 2000
PublicationDate 20161116
PublicationDateYYYYMMDD 2016-11-16
PublicationDate_xml – month: 11
  year: 2016
  text: 20161116
  day: 16
PublicationDecade 2010
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2016
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.6078007
SecondaryResourceType preprint
Snippet Sparse matrix-matrix multiplication (or SpGEMM) is a key primitive for many high-performance graph algorithms as well as for some linear solvers, such as...
SourceID proquest
SourceType Aggregation Database
SubjectTerms Algorithms
Matrices (mathematics)
Multiplication
Solvers
Sparsity
Title Exploiting Multiple Levels of Parallelism in Sparse Matrix-Matrix Multiplication
URI https://www.proquest.com/docview/2081066603
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3LSgMxFA3aKrjyjY9asnAbbSaZJLMSlBaFtgy2SF2VTB4woNM6U0s_3ySd6kJw4yqEJBAScnPOzcm9AFyTTqKkFhbFjBtELdNIUCwQNYI7-Gy4Cf-4X_p8OBSTSZLWDreqllVubGIw1HqmvI_ckXSBPdbukLv5B_JZo_zrap1CYxs0fZQEHKR7o28fS8S4Q8xk_ZgZQnfdynKVL29cV6_oEpT-MsHhXunt_3dGB6CZyrkpD8GWKY7AbtBzquoYpEFbl3tNMxzUmkHY9wKhCs4sTGXpc6i85dU7zAs4mjt6a-DAh-tfoXWxGVb79E7AuNcdPzyiOnkCko5bIpVFGsfCOJIoHWeLuTZEES1tkunYOioqaSylzhhlNlYRYUlGsHXYhusOtw6XnYJGMSvMGYCRzlTkOiZcEaqwyjKBmbE4Uloax9bOQWuzPtP6AFTTn8W5-Lv5Euw5DML89z7MWqCxKD_NFdhRy0VelW3QvO8O0-d22FdXS58G6esXtNSvAw
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3JTsMwEB2VTXBiFzs-wNHQOIntHBAHFrXqokpUiFvleJEiQVsS1o_iHxmnDRyQuHHglIMTS8k4M--N33gAjsJ6opWRjsZcWBo5bqiMAkkjKwXCZytsWcd92xbdrry7S3o1-KhqYbyssvKJpaM2I-1z5EjSZeCxdj08Hz9S3zXK765WLTQmy6Jl31-RshVnzUu07zFj11f9iwaddhWgCkkX1SkzQSwtsieFZCYWxoY6NMolqYkdcjQVxUqZlEfcxZqFPEnDwGHQF6YuHAIWnHYG5hBFsKRUCt58pXQYFwjQw8neaXlS2KnK37KXE4yqXkAmo-iHxy_D2PXyP_sAKzDXU2Obr0LNDtdgoVSr6mIdeqVyMPOKbdKZKiJJ28ufCjJypKdy3yHmPiseSDYkN2Mk75Z0fDOCNzq5VI9NM5Yb0P-Ld9iE2eFoaLeAMJNqhjcmQoeRDnSayoBbFzBtlEUuug17lTkG09-7GHzbYuf34UNYbPQ77UG72W3twhKiLe4LGQO-B7NP-bPdh3n98pQV-UG5lAgM_thyn-V4B7I
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Exploiting+Multiple+Levels+of+Parallelism+in+Sparse+Matrix-Matrix+Multiplication&rft.jtitle=arXiv.org&rft.au=Azad%2C+Ariful&rft.au=Ballard%2C+Grey&rft.au=Aydin+Buluc&rft.au=Demmel%2C+James&rft.date=2016-11-16&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.1510.00844