LogECMem: Coupling Erasure-Coded In-Memory Key-Value Stores with Parity Logging

In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low perfo...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:SC21: International Conference for High Performance Computing, Networking, Storage and Analysis S. 01 - 14
Hauptverfasser: Cheng, Liangfeng, Hu, Yuchong, Ke, Zhaokang, Xu, Jia, Yao, Qiaori, Feng, Dan, Wang, Weichun, Chen, Wei
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: ACM 14.11.2021
Schlagworte:
Costs
Couplings
Erasure coding
High performance computing
Key-value stores
Maintenance engineering
Memory management
Parity logging
Prototypes
Redundancy
Update
ISSN:2167-4337
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low performance or high memory overhead. In this paper, we propose a novel parity logging-based architecture, HybridPL, which creates a hybrid of in-place update (for data and XOR parity chunks) and log-based update (for the remaining parity chunks), so as to balance the update performance and memory cost, while maintaining efficient single-failure repairs. We realize HybridPL as an in-memory key-value store called LogECMem, and further design efficient repair schemes for multiple failures. We prototype LogECMem and conduct experiments on different workloads. We show that LogECMem achieves better up-date performance over existing erasure-coded update schemes with low memory overhead, while maintaining high basic I/O and repair performance.
ISSN:2167-4337
DOI:10.1145/3458817.3480852