HCL: Distributing Parallel Data Structures in Extreme Scales

Most parallel programs use irregular control flow and data structures, which are perfect for one-sided communication paradigms such as MPI or PGAS programming languages. However, these environments lack the presence of efficient function-based application libraries that can utilize popular communica...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings / IEEE International Conference on Cluster Computing S. 248 - 258
Hauptverfasser: Devarajan, Hariharan, Kougkas, Anthony, Bateman, Keith, Sun, Xian-He
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.09.2020
Schlagworte:
Computational modeling
Containers
Data models
Data structures
Distributed Data Structure
Distributed databases
HPC Data Containers
Hybrid Data Access Model
Libraries
Programming
RPC over RDMA
ISSN:2168-9253
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Most parallel programs use irregular control flow and data structures, which are perfect for one-sided communication paradigms such as MPI or PGAS programming languages. However, these environments lack the presence of efficient function-based application libraries that can utilize popular communication fabrics such as TCP, Infinity Band (IB), and RDMA over Converged Ethernet (RoCE). Additionally, there is a lack of high-performance data structure interfaces. We present Hermes Container Library (HCL), a high-performance distributed data structures library that offers high-level abstractions including hash-maps, sets, and queues. HCL uses a RPC over RDMA technology that implements a novel procedural programming paradigm. In this paper, we argue a RPC over RDMA technology can serve as a high-performance, flexible, and co-ordination free backend for implementing complex data structures. Evaluation results from testing real workloads shows that HCL programs are 2x to 12x faster compared to BCL, a state-of-the-art distributed data structure library.
ISSN:2168-9253
DOI:10.1109/CLUSTER49012.2020.00035