LogECMem: Coupling Erasure-Coded In-Memory Key-Value Stores with Parity Logging - Cheng, Liangfeng; Hu, Yuchong; Ke, Zhaokang; Xu, Jia; Yao, Qiaori; Feng, Dan; Wang, Weichun; Chen, Wei (SC21: International Conference for High Performance Computing, Networking, Storage and Analysis, 2021)
In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low performance or high memory overhead. In this paper, we propose a novel parity logging-based architecture, HybridPL, which creates a hybrid of in-place update (for data and XOR parity chunks) and log-based update (for the remaining parity chunks), so as to balance the update performance and memory cost, while maintaining efficient single-failure repairs. We realize HybridPL as an in-memory key-value store called LogECMem, and further design efficient repair schemes for multiple failures. We prototype LogECMem and conduct experiments on different workloads. We show that LogECMem achieves better up-date performance over existing erasure-coded update schemes with low memory overhead, while maintaining high basic I/O and repair performance.