HCL: Distributing Parallel Data Structures in Extreme Scales

Most parallel programs use irregular control flow and data structures, which are perfect for one-sided communication paradigms such as MPI or PGAS programming languages. However, these environments lack the presence of efficient function-based application libraries that can utilize popular communica...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Proceedings / IEEE International Conference on Cluster Computing s. 248 - 258
Hlavní autori: Devarajan, Hariharan, Kougkas, Anthony, Bateman, Keith, Sun, Xian-He
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 01.09.2020
Predmet:
Computational modeling
Containers
Data models
Data structures
Distributed Data Structure
Distributed databases
HPC Data Containers
Hybrid Data Access Model
Libraries
Programming
RPC over RDMA
ISSN:2168-9253
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract Most parallel programs use irregular control flow and data structures, which are perfect for one-sided communication paradigms such as MPI or PGAS programming languages. However, these environments lack the presence of efficient function-based application libraries that can utilize popular communication fabrics such as TCP, Infinity Band (IB), and RDMA over Converged Ethernet (RoCE). Additionally, there is a lack of high-performance data structure interfaces. We present Hermes Container Library (HCL), a high-performance distributed data structures library that offers high-level abstractions including hash-maps, sets, and queues. HCL uses a RPC over RDMA technology that implements a novel procedural programming paradigm. In this paper, we argue a RPC over RDMA technology can serve as a high-performance, flexible, and co-ordination free backend for implementing complex data structures. Evaluation results from testing real workloads shows that HCL programs are 2x to 12x faster compared to BCL, a state-of-the-art distributed data structure library.
AbstractList Most parallel programs use irregular control flow and data structures, which are perfect for one-sided communication paradigms such as MPI or PGAS programming languages. However, these environments lack the presence of efficient function-based application libraries that can utilize popular communication fabrics such as TCP, Infinity Band (IB), and RDMA over Converged Ethernet (RoCE). Additionally, there is a lack of high-performance data structure interfaces. We present Hermes Container Library (HCL), a high-performance distributed data structures library that offers high-level abstractions including hash-maps, sets, and queues. HCL uses a RPC over RDMA technology that implements a novel procedural programming paradigm. In this paper, we argue a RPC over RDMA technology can serve as a high-performance, flexible, and co-ordination free backend for implementing complex data structures. Evaluation results from testing real workloads shows that HCL programs are 2x to 12x faster compared to BCL, a state-of-the-art distributed data structure library.
Author Kougkas, Anthony
Bateman, Keith
Sun, Xian-He
Devarajan, Hariharan
Author_xml – sequence: 1
  givenname: Hariharan
  surname: Devarajan
  fullname: Devarajan, Hariharan
  email: hdevarajan@hawk.iit.edu
  organization: Illinois Institute of Technology,Department of Computer Science
– sequence: 2
  givenname: Anthony
  surname: Kougkas
  fullname: Kougkas, Anthony
  email: akougkas@iit.edu
  organization: Illinois Institute of Technology,Department of Computer Science
– sequence: 3
  givenname: Keith
  surname: Bateman
  fullname: Bateman, Keith
  email: kbateman@hawk.iit.edu
  organization: Illinois Institute of Technology,Department of Computer Science
– sequence: 4
  givenname: Xian-He
  surname: Sun
  fullname: Sun, Xian-He
  email: sun@iit.edu
  organization: Illinois Institute of Technology,Department of Computer Science
BookMark eNotzM1Kw0AUQOFRFGxrn8CF8wKJd36SuSNuJI1WCCimXZdJciMjaZTJBPTtFXR1Nh9nyc7Gj5EYuxaQCgH2pqj29a581RaETCVISAFAZSdsbQ0KI1HkuTHqlC2kyDGxMlMXbDlN77_KKMgX7G5bVLd846cYfDNHP77xFxfcMNDANy46Xscwt3EONHE_8vIrBjoSr1s30HTJzns3TLT-74rtH8pdsU2q58en4r5KvAQVk761bWd666QDR9QhSovQkOl07tB2wqDW2CA4TYqw1bIjQIGdUSRaq9SKXf19PREdPoM_uvB9sFLazGbqB7kASxs
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/CLUSTER49012.2020.00035
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9781728166773
1728166772
EISSN 2168-9253
EndPage 258
ExternalDocumentID 9229595
Genre orig-research
GrantInformation_xml – fundername: National Science Foundation
  funderid: 10.13039/100000001
GroupedDBID 29O
6IE
6IF
6IH
6IK
6IL
6IN
AAJGR
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
M43
OCL
RIE
RIL
RNS
ID FETCH-LOGICAL-i203t-fc9cd7f9a2a0aeed882980be7d46a89d178448b80a4e3e8c42de0818d73e1c933
IEDL.DBID RIE
ISICitedReferencesCount 5
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000698696500025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 06:00:22 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-fc9cd7f9a2a0aeed882980be7d46a89d178448b80a4e3e8c42de0818d73e1c933
PageCount 11
ParticipantIDs ieee_primary_9229595
PublicationCentury 2000
PublicationDate 2020-Sept.
PublicationDateYYYYMMDD 2020-09-01
PublicationDate_xml – month: 09
  year: 2020
  text: 2020-Sept.
PublicationDecade 2020
PublicationTitle Proceedings / IEEE International Conference on Cluster Computing
PublicationTitleAbbrev CLUSTER
PublicationYear 2020
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0037306
Score 2.1534793
Snippet Most parallel programs use irregular control flow and data structures, which are perfect for one-sided communication paradigms such as MPI or PGAS programming...
SourceID ieee
SourceType Publisher
StartPage 248
SubjectTerms Computational modeling
Containers
Data models
Data structures
Distributed Data Structure
Distributed databases
HPC Data Containers
Hybrid Data Access Model
Libraries
Programming
RPC over RDMA
Title HCL: Distributing Parallel Data Structures in Extreme Scales
URI https://ieeexplore.ieee.org/document/9229595
WOSCitedRecordID wos000698696500025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT8IwFH4B4sETKhjFH-nBo5Vu7dbWeONHOBBCgiTcSNd2hsQMA8P459uWgR68eFuWLN3e1n19r-_7PoAHmWuTJYRgplKGWRYpLGwksUvifGU_zaUJkvljPpmIxUJOa_B45MJYa0PzmX3yh2Ev36z1zpfKutJ7T8ukDnXO0z1X6_DXpe5LTav-rYjIbm88n7n1IHNw5-lWsW_gIt7T7ZeHSoCQYfN_g59B-4eLh6ZHlDmHmi0uoHkwY0DV3GzBy6g3fkZ9r4MbLKyKNzRVG--U8o76qlRoFpRidy69RqsCDb5KXxl01zuE2LZhPhy89ka4skbAq5jQEudaasNzqWJFlLsBt06WgmSWG5YqIU3Ehcu7MkEUs9QKzWJjvXid4dRGWlJ6CY1iXdgrQDFLhHGJSZZyB_W5yFKRaRrFufY0U5NcQ8sHY_mxV79YVnHo_H36Bk59tPddWLfQcI9m7-BEf5ar7eY-vLJviT-W_w
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT8IwFH5BNNETKhh_24NHJ13bba3xxo9gnIQESLiRru0IiRkGhvHPtx0DPXjxtixZur2t-_pe3_d9APciVToJMPaYDJnHEl963PjCs0mcq-yHqdCFZH4c9ft8MhGDCjzsuDDGmKL5zDy6w2IvXy_U2pXKmsJ5T4tgD_YDxgjesLW2_11qv9Ww7ODysWi24vHQrgiZBTxHuCKuhQs7V7dfLioFiHRr_xv-GBo_bDw02OHMCVRMdgq1rR0DKmdnHZ57rfgJtZ0SbmFilc3QQC6dV8o7astcomGhFbu2CTaaZ6jzlbvaoL3eYsSqAeNuZ9TqeaU5gjcnmOZeqoTSUSokkVjaG7ArZcFxYiLNQsmF9iNuM6-EY8kMNVwxoo2Tr9MRNb4SlJ5BNVtk5hwQYQHXNjVJwsiCfcqTkCeK-iRVjmiqgwuou2BMPzb6F9MyDpd_n76Dw97oLZ7GL_3XKzhykd_0ZF1D1T6muYED9ZnPV8vb4vV9A5XimkY
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%2F+IEEE+International+Conference+on+Cluster+Computing&rft.atitle=HCL%3A+Distributing+Parallel+Data+Structures+in+Extreme+Scales&rft.au=Devarajan%2C+Hariharan&rft.au=Kougkas%2C+Anthony&rft.au=Bateman%2C+Keith&rft.au=Sun%2C+Xian-He&rft.date=2020-09-01&rft.pub=IEEE&rft.eissn=2168-9253&rft.spage=248&rft.epage=258&rft_id=info:doi/10.1109%2FCLUSTER49012.2020.00035&rft.externalDocID=9229595