HCL: Distributing Parallel Data Structures in Extreme Scales

Most parallel programs use irregular control flow and data structures, which are perfect for one-sided communication paradigms such as MPI or PGAS programming languages. However, these environments lack the presence of efficient function-based application libraries that can utilize popular communica...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings / IEEE International Conference on Cluster Computing s. 248 - 258
Hlavní autoři: Devarajan, Hariharan, Kougkas, Anthony, Bateman, Keith, Sun, Xian-He
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.09.2020
Témata:
Computational modeling
Containers
Data models
Data structures
Distributed Data Structure
Distributed databases
HPC Data Containers
Hybrid Data Access Model
Libraries
Programming
RPC over RDMA
ISSN:2168-9253
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Most parallel programs use irregular control flow and data structures, which are perfect for one-sided communication paradigms such as MPI or PGAS programming languages. However, these environments lack the presence of efficient function-based application libraries that can utilize popular communication fabrics such as TCP, Infinity Band (IB), and RDMA over Converged Ethernet (RoCE). Additionally, there is a lack of high-performance data structure interfaces. We present Hermes Container Library (HCL), a high-performance distributed data structures library that offers high-level abstractions including hash-maps, sets, and queues. HCL uses a RPC over RDMA technology that implements a novel procedural programming paradigm. In this paper, we argue a RPC over RDMA technology can serve as a high-performance, flexible, and co-ordination free backend for implementing complex data structures. Evaluation results from testing real workloads shows that HCL programs are 2x to 12x faster compared to BCL, a state-of-the-art distributed data structure library.
ISSN:2168-9253
DOI:10.1109/CLUSTER49012.2020.00035