Accelerating Cloud-Native Databases with Distributed PMem Stores
Relational databases have gone through a phase of architectural transition from a monolithic to a distributed architecture to take full advantage of cloud technology. These distributed databases can leverage remote storage to maintain larger amounts of data than monolithic databases at the cost of i...
Uložené v:
| Vydané v: | Data engineering s. 3043 - 3057 |
|---|---|
| Hlavní autori: | , , , , , , , , , , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
01.04.2023
|
| Predmet: | |
| ISSN: | 2375-026X |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Relational databases have gone through a phase of architectural transition from a monolithic to a distributed architecture to take full advantage of cloud technology. These distributed databases can leverage remote storage to maintain larger amounts of data than monolithic databases at the cost of increased latency. At ByteDance, we have built a distributed database called veDB based on the popular compute-storage separation architecture, however we have observed the system is unable to provide both low latency and high throughput required by some business critical applications, such as batched order processing.In this paper we present our novel approaches to tackle this problem. We have modified our system's storage to utilize persistent memory (PMem) coupled with a remote direct memory access (RDMA) network to reduce read/write latency and increase the throughput. We also propose a query push-down framework to push partial computations to the PMem storage layer to accelerate analytical queries and reduce the impact of the transaction workload in the computation layer. Our experiments show that our methods improve the throughput by up to 1.5× and reduce latency by up to 20× for standard benchmarks and real-world applications. |
|---|---|
| AbstractList | Relational databases have gone through a phase of architectural transition from a monolithic to a distributed architecture to take full advantage of cloud technology. These distributed databases can leverage remote storage to maintain larger amounts of data than monolithic databases at the cost of increased latency. At ByteDance, we have built a distributed database called veDB based on the popular compute-storage separation architecture, however we have observed the system is unable to provide both low latency and high throughput required by some business critical applications, such as batched order processing.In this paper we present our novel approaches to tackle this problem. We have modified our system's storage to utilize persistent memory (PMem) coupled with a remote direct memory access (RDMA) network to reduce read/write latency and increase the throughput. We also propose a query push-down framework to push partial computations to the PMem storage layer to accelerate analytical queries and reduce the impact of the transaction workload in the computation layer. Our experiments show that our methods improve the throughput by up to 1.5× and reduce latency by up to 20× for standard benchmarks and real-world applications. |
| Author | Wu, Kai Bruhwiler, Kevin Liang, Yuming Zhang, Li Zhu, Cheng Luo, Shangyu Zhang, Lei Nie, Yuanyuan Ma, Haoxiang Liu, Huicong Shi, Haiyang Chen, Jianjun Sun, Jason |
| Author_xml | – sequence: 1 givenname: Jason surname: Sun fullname: Sun, Jason email: jason.sun@bytedance.com – sequence: 2 givenname: Haoxiang surname: Ma fullname: Ma, Haoxiang – sequence: 3 givenname: Li surname: Zhang fullname: Zhang, Li – sequence: 4 givenname: Huicong surname: Liu fullname: Liu, Huicong – sequence: 5 givenname: Haiyang surname: Shi fullname: Shi, Haiyang organization: ByteDance Inc.,ByteDance US Infrastructure System Lab – sequence: 6 givenname: Shangyu surname: Luo fullname: Luo, Shangyu organization: ByteDance Inc.,ByteDance US Infrastructure System Lab – sequence: 7 givenname: Kai surname: Wu fullname: Wu, Kai email: wukai@microsoft.com organization: ByteDance Inc.,ByteDance US Infrastructure System Lab – sequence: 8 givenname: Kevin surname: Bruhwiler fullname: Bruhwiler, Kevin email: kevin.bruhwiler@gmail.com – sequence: 9 givenname: Cheng surname: Zhu fullname: Zhu, Cheng – sequence: 10 givenname: Yuanyuan surname: Nie fullname: Nie, Yuanyuan – sequence: 11 givenname: Jianjun surname: Chen fullname: Chen, Jianjun organization: ByteDance Inc.,ByteDance US Infrastructure System Lab – sequence: 12 givenname: Lei surname: Zhang fullname: Zhang, Lei – sequence: 13 givenname: Yuming surname: Liang fullname: Liang, Yuming |
| BookMark | eNotjMtOwzAQRQ0CiVLyB13kB1L8iGPPjiopUKk8JLpgV02cCRilCYpdEH9PJLiLc3Q295Kd9UNPjC0EXwrB4XpTVmuttdBLyaVa8gnqhCVgwCrNlZTSwCmbSWV0xmXxesGSED74NMiF0HzGblbOUUcjRt-_pWU3HJvscYovSiuMWGOgkH77-J5WPsTR18dITfr8QIf0JQ4jhSt23mIXKPn3nO1u17vyPts-3W3K1TbzIGLmZFMAYGtqqyy2VnGTAymrUTgDEhpNrUMLtUZpisI6k-coDIJAa3nj1Jwt_m49Ee0_R3_A8WcvuLB5oUD9AuCiTBs |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/ICDE55515.2023.00233 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798350322279 |
| EISSN | 2375-026X |
| EndPage | 3057 |
| ExternalDocumentID | 10184639 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i91t-c2d699af7b838af830749e385a1c7929d5efca89b5a27668c744a17a91a880dc3 |
| IEDL.DBID | RIE |
| IngestDate | Wed Aug 27 02:07:22 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i91t-c2d699af7b838af830749e385a1c7929d5efca89b5a27668c744a17a91a880dc3 |
| PageCount | 15 |
| ParticipantIDs | ieee_primary_10184639 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-April |
| PublicationDateYYYYMMDD | 2023-04-01 |
| PublicationDate_xml | – month: 04 year: 2023 text: 2023-April |
| PublicationDecade | 2020 |
| PublicationTitle | Data engineering |
| PublicationTitleAbbrev | ICDE |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0000941150 |
| Score | 2.23651 |
| Snippet | Relational databases have gone through a phase of architectural transition from a monolithic to a distributed architecture to take full advantage of cloud... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 3043 |
| SubjectTerms | Benchmark testing cloud computing cloud-native database Computer architecture distributed database Distributed databases PMem Query processing query push-down RDMA Relational databases Technological innovation Throughput |
| Title | Accelerating Cloud-Native Databases with Distributed PMem Stores |
| URI | https://ieeexplore.ieee.org/document/10184639 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA62ePBUHxXf5OB12-4jj7kpfaCgpWCR3so0yUpBt9Ju_f1m0m314sHbEpZdmDB8M5N838fYbYY5oFZAFGXfoCCISGcoI5GCf1-ajgks19cnNRzqyQRGFVk9cGGcc-HymWvRYzjLtwuzplFZm9SlMg-pNVZTSm7IWruBiu9TqLqp6HFxB9qP3V5f-IpAtMgjnEYnZI_7y0QlYMig8c-_H7LmDxuPj3Y4c8T2XHHMGls7Bl5l5wm7uzfGgwhtafHGu--LtY2GQdeb97BEwqsVp7kr75FaLhldOctHz-6Dv_jO262abDzoj7sPUWWQEM0hLiOTWAmAuZrpVGOufbpm4FItMDbKlz1WuNyghpnAREmpjcoyjBVCjD5rrUlPWb1YFO6McSnQGgkmn7k8cyRZCjpB8N-XQqRJfM6aFJDp50YCY7qNxcUf65fsgGK-ueJyxerlcu2u2b75Kuer5U3YuG-x4piQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8JAEN4omugJHxjf7sFrgT72MTcNSCBCQyIx3Miw3RoSLQaKv9-dUtCLB2_NpmmT2Uy-mdn9vo-x-whTQK2AKMquQUEQno5QeiIE9740TVOwXF_7Ko71eAzDkqxecGGstcXlM1unx-IsP5mbFY3KGqQuFTlI3WV7ZJ3lr-la25GK61SovikJcn4TGr1W-0m4mkDUySWchidkkPvLRqVAkU71n_8_YrUfPh4fbpHmmO3Y7IRVN4YMvMzPU_bwaIyDEdrU7I233uerxIsLZW_exhwJsZacJq-8TXq5ZHVlEz4c2A_-4npvu6yxUedp1Op6pUWCNwM_90yQSABM1VSHGlPtEjYCG2qBvlGu8EmETQ1qmAoMlJTaqChCXyH46PI2MeEZq2TzzJ4zLgUmRoJJpzaNLImWgg4Q3PelEGHgX7AaBWTyuRbBmGxicfnH-h076I4G_Um_Fz9fsUOK__rCyzWr5IuVvWH75iufLRe3xSZ-A9Ngm9c |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Data+engineering&rft.atitle=Accelerating+Cloud-Native+Databases+with+Distributed+PMem+Stores&rft.au=Sun%2C+Jason&rft.au=Ma%2C+Haoxiang&rft.au=Zhang%2C+Li&rft.au=Liu%2C+Huicong&rft.date=2023-04-01&rft.pub=IEEE&rft.eissn=2375-026X&rft.spage=3043&rft.epage=3057&rft_id=info:doi/10.1109%2FICDE55515.2023.00233&rft.externalDocID=10184639 |