LogECMem: Coupling Erasure-Coded In-Memory Key-Value Stores with Parity Logging

In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low perfo...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:SC21: International Conference for High Performance Computing, Networking, Storage and Analysis s. 01 - 14
Hlavní autoři: Cheng, Liangfeng, Hu, Yuchong, Ke, Zhaokang, Xu, Jia, Yao, Qiaori, Feng, Dan, Wang, Weichun, Chen, Wei
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: ACM 14.11.2021
Témata:
Costs
Couplings
Erasure coding
High performance computing
Key-value stores
Maintenance engineering
Memory management
Parity logging
Prototypes
Redundancy
Update
ISSN:2167-4337
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low performance or high memory overhead. In this paper, we propose a novel parity logging-based architecture, HybridPL, which creates a hybrid of in-place update (for data and XOR parity chunks) and log-based update (for the remaining parity chunks), so as to balance the update performance and memory cost, while maintaining efficient single-failure repairs. We realize HybridPL as an in-memory key-value store called LogECMem, and further design efficient repair schemes for multiple failures. We prototype LogECMem and conduct experiments on different workloads. We show that LogECMem achieves better up-date performance over existing erasure-coded update schemes with low memory overhead, while maintaining high basic I/O and repair performance.
ISSN:2167-4337
DOI:10.1145/3458817.3480852