A Comparative Study of Consistent Snapshot Algorithms for Main-Memory Database Systems
Saved in:
| Title: | A Comparative Study of Consistent Snapshot Algorithms for Main-Memory Database Systems |
|---|---|
| Authors: | Liang Li, Guoren Wang, Gang Wu, Ye Yuan, Lei Chen, Xiang Lian |
| Source: | IEEE Transactions on Knowledge and Data Engineering. 33:316-330 |
| Publication Status: | Preprint |
| Publisher Information: | Institute of Electrical and Electronics Engineers (IEEE), 2021. |
| Publication Year: | 2021 |
| Subject Terms: | Checkpoints, FOS: Computer and information sciences, Computer Science - Databases, HTAP, Snapshot algorithms, 0202 electrical engineering, electronic engineering, information engineering, Databases (cs.DB), 02 engineering and technology, In-memory database systems |
| Description: | In-memory databases (IMDBs) are gaining increasing popularity in big data applications, where clients commit updates intensively. Specifically, it is necessary for IMDBs to have efficient snapshot performance to support certain special applications (e.g., consistent checkpoint, HTAP). Formally, the in-memory consistent snapshot problem refers to taking an in-memory consistent time-in-point snapshot with the constraints that 1) clients can read the latest data items and 2) any data item in the snapshot should not be overwritten. Various snapshot algorithms have been proposed in academia to trade off throughput and latency, but industrial IMDBs such as Redis adhere to the simple fork algorithm. To understand this phenomenon, we conduct comprehensive performance evaluations on mainstream snapshot algorithms. Surprisingly, we observe that the simple fork algorithm indeed outperforms the state-of-the-arts in update-intensive workload scenarios. On this basis, we identify the drawbacks of existing research and propose two lightweight improvements. Extensive evaluations on synthetic data and Redis show that our lightweight improvements yield better performance than fork, the current industrial standard, and the representative snapshot algorithms from academia. Finally, we have opensourced the implementation of all the above snapshot algorithms so that practitioners are able to benchmark the performance of each algorithm and select proper methods for different application scenarios. |
| Document Type: | Article |
| ISSN: | 2326-3865 1041-4347 |
| DOI: | 10.1109/tkde.2019.2930987 |
| DOI: | 10.48550/arxiv.1810.04915 |
| Access URL: | http://arxiv.org/pdf/1810.04915 http://arxiv.org/abs/1810.04915 https://arxiv.org/abs/1810.04915 https://ieeexplore.ieee.org/document/8772140/ https://arxiv.org/pdf/1810.04915.pdf https://dblp.uni-trier.de/db/journals/tkde/tkde33.html#LiWWYCL21 https://doi.org/10.1109/TKDE.2019.2930987 |
| Rights: | IEEE Copyright arXiv Non-Exclusive Distribution |
| Accession Number: | edsair.doi.dedup.....75b8a7c455d7a9c13aa4c89605c57a2c |
| Database: | OpenAIRE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstract: | In-memory databases (IMDBs) are gaining increasing popularity in big data applications, where clients commit updates intensively. Specifically, it is necessary for IMDBs to have efficient snapshot performance to support certain special applications (e.g., consistent checkpoint, HTAP). Formally, the in-memory consistent snapshot problem refers to taking an in-memory consistent time-in-point snapshot with the constraints that 1) clients can read the latest data items and 2) any data item in the snapshot should not be overwritten. Various snapshot algorithms have been proposed in academia to trade off throughput and latency, but industrial IMDBs such as Redis adhere to the simple fork algorithm. To understand this phenomenon, we conduct comprehensive performance evaluations on mainstream snapshot algorithms. Surprisingly, we observe that the simple fork algorithm indeed outperforms the state-of-the-arts in update-intensive workload scenarios. On this basis, we identify the drawbacks of existing research and propose two lightweight improvements. Extensive evaluations on synthetic data and Redis show that our lightweight improvements yield better performance than fork, the current industrial standard, and the representative snapshot algorithms from academia. Finally, we have opensourced the implementation of all the above snapshot algorithms so that practitioners are able to benchmark the performance of each algorithm and select proper methods for different application scenarios. |
|---|---|
| ISSN: | 23263865 10414347 |
| DOI: | 10.1109/tkde.2019.2930987 |