LogECMem: Coupling Erasure-Coded In-Memory Key-Value Stores with Parity Logging
In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low perfo...
Gespeichert in:
| Veröffentlicht in: | SC21: International Conference for High Performance Computing, Networking, Storage and Analysis S. 01 - 14 |
|---|---|
| Hauptverfasser: | , , , , , , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
ACM
14.11.2021
|
| Schlagworte: | |
| ISSN: | 2167-4337 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low performance or high memory overhead. In this paper, we propose a novel parity logging-based architecture, HybridPL, which creates a hybrid of in-place update (for data and XOR parity chunks) and log-based update (for the remaining parity chunks), so as to balance the update performance and memory cost, while maintaining efficient single-failure repairs. We realize HybridPL as an in-memory key-value store called LogECMem, and further design efficient repair schemes for multiple failures. We prototype LogECMem and conduct experiments on different workloads. We show that LogECMem achieves better up-date performance over existing erasure-coded update schemes with low memory overhead, while maintaining high basic I/O and repair performance. |
|---|---|
| AbstractList | In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low performance or high memory overhead. In this paper, we propose a novel parity logging-based architecture, HybridPL, which creates a hybrid of in-place update (for data and XOR parity chunks) and log-based update (for the remaining parity chunks), so as to balance the update performance and memory cost, while maintaining efficient single-failure repairs. We realize HybridPL as an in-memory key-value store called LogECMem, and further design efficient repair schemes for multiple failures. We prototype LogECMem and conduct experiments on different workloads. We show that LogECMem achieves better up-date performance over existing erasure-coded update schemes with low memory overhead, while maintaining high basic I/O and repair performance. |
| Author | Ke, Zhaokang Chen, Wei Yao, Qiaori Cheng, Liangfeng Xu, Jia Hu, Yuchong Feng, Dan Wang, Weichun |
| Author_xml | – sequence: 1 givenname: Liangfeng surname: Cheng fullname: Cheng, Liangfeng email: lenfungcheng@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 2 givenname: Yuchong surname: Hu fullname: Hu, Yuchong email: yuchonghu@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 3 givenname: Zhaokang surname: Ke fullname: Ke, Zhaokang email: 2020612167@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 4 givenname: Jia surname: Xu fullname: Xu, Jia email: helenxu@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 5 givenname: Qiaori surname: Yao fullname: Yao, Qiaori email: yaoqr@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 6 givenname: Dan surname: Feng fullname: Feng, Dan email: dfeng@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 7 givenname: Weichun surname: Wang fullname: Wang, Weichun email: wangweichun@hikvision.com organization: HIKVISION,Hangzhou,China – sequence: 8 givenname: Wei surname: Chen fullname: Chen, Wei email: chenwei7@hikvision.com organization: HIKVISION,Hangzhou,China |
| BookMark | eNotjMtKw0AUQEdRsNasXbiZH5h678xkHu4kVC1GKvjYlqRzUyNtUiYJkr83oKuzOJxzyc6atiHGrhEWiDq9VTp1Du1CaQculScs8dZNApTTWuIpm0k0Vmil7AVLuu4bAKSzqCTM2Dpvd8vshQ53PGuH475udnwZi26IJLI2UOCrRky6jSN_plF8FvuB-FvfRur4T91_8dci1v3Ip89uiq_YeVXsO0r-OWcfD8v37Enk68dVdp-LQtq0Fwq03XodUgALIaiydNqVRoOByuMWfTB6S6ayVKKnVGGobCidKTXpylRKzdnN37cmos0x1ocijhvvEVBa9Qv9BU94 |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1145/3458817.3480852 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9781450384421 1450384420 |
| EISSN | 2167-4337 |
| EndPage | 14 |
| ExternalDocumentID | 9910127 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Natural Science Foundation of China grantid: 61872414 funderid: 10.13039/501100001809 |
| GroupedDBID | 6IE 6IF 6IH 6IK 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI OCL RIE RIL |
| ID | FETCH-LOGICAL-a275t-3047c94d50070dd3bb848b64060f91c19d64ce6f7eb19e531df7db86b4e4f6f33 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 12 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000946520100087&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:18:35 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a275t-3047c94d50070dd3bb848b64060f91c19d64ce6f7eb19e531df7db86b4e4f6f33 |
| PageCount | 14 |
| ParticipantIDs | ieee_primary_9910127 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-Nov.-14 |
| PublicationDateYYYYMMDD | 2021-11-14 |
| PublicationDate_xml | – month: 11 year: 2021 text: 2021-Nov.-14 day: 14 |
| PublicationDecade | 2020 |
| PublicationTitle | SC21: International Conference for High Performance Computing, Networking, Storage and Analysis |
| PublicationTitleAbbrev | SC |
| PublicationYear | 2021 |
| Publisher | ACM |
| Publisher_xml | – name: ACM |
| SSID | ssj0002871320 ssj0003204180 |
| Score | 1.8563764 |
| Snippet | In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 01 |
| SubjectTerms | Costs Couplings Erasure coding High performance computing Key-value stores Maintenance engineering Memory management Parity logging Prototypes Redundancy Update |
| Title | LogECMem: Coupling Erasure-Coded In-Memory Key-Value Stores with Parity Logging |
| URI | https://ieeexplore.ieee.org/document/9910127 |
| WOSCitedRecordID | wos000946520100087&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NT8JAEN0A8eAJFYzf2YNHCy3ddrdeG4hGQBKVcCP7MUtMtDVATfj37pSKMfHibdqkTbPN9k3fzJtHyLUDVSmVRKLKSo8FPnjSSBfFVvvCBCooJ95Mh3w8FrNZMqmRm50WBgDK5jPoYFjW8k2uC6TKui6XwUppndQ5j7darR2fgpl_WKU-eOxiFgi_muYTsKgboigz4J2QCZdn9H7ZqZRoMmj-7zkOSPtHlkcnO8A5JDXIjkjz25eBVtu0RR6H-aKfjuD9lqZ5gZLbBe0vSy7QS3MDht5n3ghbbDf0ATbeVL4VQJ_c3zesKBKzdCLR0466-6CHUZu8DPrP6Z1XGSd4ssejtYelNJ0wE-EwH2NCpQQTKnbY7dsk0EFiYqYhttx9qBNwu9BYbpSIFQNmYxuGx6SR5RmcEMoNiyKtQQC37nqVJMo60Adjw0RL4Z-SFq7P_GM7G2NeLc3Z36fPyX4Pe0KwjY5dkMZ6WcAl2dOf69fV8qp8oV9126AZ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEF5qFfRUtRXf7sGjafPYJBuvoaWlDwvW0lvZx2wRtJE-hP57d9JYEbx4mwQSwobNN_lmvvkIubegKoQUSFQZ4TDPBUdoYaPIKJdrT3r5xJtxLx4M-GSSDEvkYaeFAYC8-QzqGOa1fJ2pNVJlDZvLYKV0j-yHjPnuVq21Y1Qw9w-K5AePbcw87hbzfDwWNgKUZXpxPWDcZhr-L0OVHE9alf89yTGp_Qjz6HAHOSekBPNTUvl2ZqDFRq2Sp142a6Z9eH-kabZG0e2MNhc5G-ikmQZNO3Onj022G9qFjTMWb2ugz_b_G5YUqVk6FOhqR-190MWoRl5azVHadgrrBEf4cbhysJimEqZDHOejdSAlZ1xGFr1dk3jKS3TEFEQmtp_qBOw-1CbWkkeSATORCYIzUp5nczgnNNYsDJUCDrGx18skkcbCPmgTJEpw94JUcX2mH9vpGNNiaS7_Pn1HDtujfm_a6wy6V-TIxw4RbKpj16S8Wqzhhhyoz9XrcnGbv9wv-5ijYA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=SC21%3A+International+Conference+for+High+Performance+Computing%2C+Networking%2C+Storage+and+Analysis&rft.atitle=LogECMem%3A+Coupling+Erasure-Coded+In-Memory+Key-Value+Stores+with+Parity+Logging&rft.au=Cheng%2C+Liangfeng&rft.au=Hu%2C+Yuchong&rft.au=Ke%2C+Zhaokang&rft.au=Xu%2C+Jia&rft.date=2021-11-14&rft.pub=ACM&rft.eissn=2167-4337&rft.spage=01&rft.epage=14&rft_id=info:doi/10.1145%2F3458817.3480852&rft.externalDocID=9910127 |