LogECMem: Coupling Erasure-Coded In-Memory Key-Value Stores with Parity Logging
In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low perfo...
Uloženo v:
| Vydáno v: | SC21: International Conference for High Performance Computing, Networking, Storage and Analysis s. 01 - 14 |
|---|---|
| Hlavní autoři: | , , , , , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
ACM
14.11.2021
|
| Témata: | |
| ISSN: | 2167-4337 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low performance or high memory overhead. In this paper, we propose a novel parity logging-based architecture, HybridPL, which creates a hybrid of in-place update (for data and XOR parity chunks) and log-based update (for the remaining parity chunks), so as to balance the update performance and memory cost, while maintaining efficient single-failure repairs. We realize HybridPL as an in-memory key-value store called LogECMem, and further design efficient repair schemes for multiple failures. We prototype LogECMem and conduct experiments on different workloads. We show that LogECMem achieves better up-date performance over existing erasure-coded update schemes with low memory overhead, while maintaining high basic I/O and repair performance. |
|---|---|
| AbstractList | In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low performance or high memory overhead. In this paper, we propose a novel parity logging-based architecture, HybridPL, which creates a hybrid of in-place update (for data and XOR parity chunks) and log-based update (for the remaining parity chunks), so as to balance the update performance and memory cost, while maintaining efficient single-failure repairs. We realize HybridPL as an in-memory key-value store called LogECMem, and further design efficient repair schemes for multiple failures. We prototype LogECMem and conduct experiments on different workloads. We show that LogECMem achieves better up-date performance over existing erasure-coded update schemes with low memory overhead, while maintaining high basic I/O and repair performance. |
| Author | Ke, Zhaokang Chen, Wei Yao, Qiaori Cheng, Liangfeng Xu, Jia Hu, Yuchong Feng, Dan Wang, Weichun |
| Author_xml | – sequence: 1 givenname: Liangfeng surname: Cheng fullname: Cheng, Liangfeng email: lenfungcheng@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 2 givenname: Yuchong surname: Hu fullname: Hu, Yuchong email: yuchonghu@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 3 givenname: Zhaokang surname: Ke fullname: Ke, Zhaokang email: 2020612167@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 4 givenname: Jia surname: Xu fullname: Xu, Jia email: helenxu@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 5 givenname: Qiaori surname: Yao fullname: Yao, Qiaori email: yaoqr@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 6 givenname: Dan surname: Feng fullname: Feng, Dan email: dfeng@hust.edu.cn organization: Huazhong University of Science and Technology,Wuhan,China – sequence: 7 givenname: Weichun surname: Wang fullname: Wang, Weichun email: wangweichun@hikvision.com organization: HIKVISION,Hangzhou,China – sequence: 8 givenname: Wei surname: Chen fullname: Chen, Wei email: chenwei7@hikvision.com organization: HIKVISION,Hangzhou,China |
| BookMark | eNotjMtKw0AUQEdRsNasXbiZH5h678xkHu4kVC1GKvjYlqRzUyNtUiYJkr83oKuzOJxzyc6atiHGrhEWiDq9VTp1Du1CaQculScs8dZNApTTWuIpm0k0Vmil7AVLuu4bAKSzqCTM2Dpvd8vshQ53PGuH475udnwZi26IJLI2UOCrRky6jSN_plF8FvuB-FvfRur4T91_8dci1v3Ip89uiq_YeVXsO0r-OWcfD8v37Enk68dVdp-LQtq0Fwq03XodUgALIaiydNqVRoOByuMWfTB6S6ayVKKnVGGobCidKTXpylRKzdnN37cmos0x1ocijhvvEVBa9Qv9BU94 |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1145/3458817.3480852 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9781450384421 1450384420 |
| EISSN | 2167-4337 |
| EndPage | 14 |
| ExternalDocumentID | 9910127 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Natural Science Foundation of China grantid: 61872414 funderid: 10.13039/501100001809 |
| GroupedDBID | 6IE 6IF 6IH 6IK 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI OCL RIE RIL |
| ID | FETCH-LOGICAL-a275t-3047c94d50070dd3bb848b64060f91c19d64ce6f7eb19e531df7db86b4e4f6f33 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 12 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000946520100087&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:18:35 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a275t-3047c94d50070dd3bb848b64060f91c19d64ce6f7eb19e531df7db86b4e4f6f33 |
| PageCount | 14 |
| ParticipantIDs | ieee_primary_9910127 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-Nov.-14 |
| PublicationDateYYYYMMDD | 2021-11-14 |
| PublicationDate_xml | – month: 11 year: 2021 text: 2021-Nov.-14 day: 14 |
| PublicationDecade | 2020 |
| PublicationTitle | SC21: International Conference for High Performance Computing, Networking, Storage and Analysis |
| PublicationTitleAbbrev | SC |
| PublicationYear | 2021 |
| Publisher | ACM |
| Publisher_xml | – name: ACM |
| SSID | ssj0002871320 ssj0003204180 |
| Score | 1.8562721 |
| Snippet | In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 01 |
| SubjectTerms | Costs Couplings Erasure coding High performance computing Key-value stores Maintenance engineering Memory management Parity logging Prototypes Redundancy Update |
| Title | LogECMem: Coupling Erasure-Coded In-Memory Key-Value Stores with Parity Logging |
| URI | https://ieeexplore.ieee.org/document/9910127 |
| WOSCitedRecordID | wos000946520100087&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA_b8OBp6iZ-k4NHs7VN2iZey4aimwM_2G2kzcsQtJVtFfbfm9fVieDFW9JDCSmv7_c-fu9HyKUxDiQLtDStXYACQcQ0cMWUMqCEAN9oU4lNxOOxnE7VpEGutlwYAKiaz6CHy6qWb4qsxFRZ32EZrJQ2STOOow1Xa5tPQeTPa-iDe7cWvvTqaT6-CPscSZl-3ONCOpwR_JJTqbzJsP2_c-yR7g8tj062DmefNCA_IO1vXQZam2mHPNwX80EygvdrmhQlUm7ndLCocoEsKQwYepuzEbbYrukdrNmLfiuBPrroG5YUE7N0olHTjrr3oIZRlzwPB0_JDauFE5gO4nDFsJSWKWFCHOZjDE9TKWQaOd_tWeVnvjKRyCCysftRK3BWaGxsUhmlAoSNLOeHpJUXORwRmlkRgASZaq6FTQPNbSBTL5TWCBdcecekg_cz-9jMxpjVV3Py9-NTshtgTwi20Ykz0lotSjgnO9nn6nW5uKg-6BfX5J-_ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEF5qFfRUn_h2Dx6NJtlJsuu1tFj6sOADb2WTnRVBE6mN0H_vThorghdvuzmEZcNkvnl88zF2bowDyUCWprULUDCMPY1CeUoZVAAYGG0qsYlkNJJPT2rcYBdLLgwiVs1neEnLqpZviqykVNmVwzJUKV1hqxFA6C_YWsuMCmF_UYMf2rs1BNKv5_kEEF0JomUGyaUA6ZBG-EtQpfIn3db_TrLJdn-IeXy8dDlbrIH5Nmt9KzPw2lB32O2geO60h_h2zdtFSaTbZ96ZVtlAr10YNLyXe0Nqsp3zPs69R_1aIr9z8Td-cErN8rEmVTvu3kMqRrvsodu5b994tXSCp8MkmnlUTMsUmIjG-Rgj0lSCTGPnvX2rgixQJoYMY5u4X7VCZ4fGJiaVcQoINrZC7LFmXuS4z3hmIUSJMtVCg01DLWwoUz-S1oALr_wDtkP3M3lfTMeY1Fdz-PfjM7Z-cz8cTAa9Uf-IbYTUIUJNdXDMmrNpiSdsLfucvXxMT6uP-wVY7qMG |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=SC21%3A+International+Conference+for+High+Performance+Computing%2C+Networking%2C+Storage+and+Analysis&rft.atitle=LogECMem%3A+Coupling+Erasure-Coded+In-Memory+Key-Value+Stores+with+Parity+Logging&rft.au=Cheng%2C+Liangfeng&rft.au=Hu%2C+Yuchong&rft.au=Ke%2C+Zhaokang&rft.au=Xu%2C+Jia&rft.date=2021-11-14&rft.pub=ACM&rft.eissn=2167-4337&rft.spage=01&rft.epage=14&rft_id=info:doi/10.1145%2F3458817.3480852&rft.externalDocID=9910127 |