Research and implementation of asynchronous compaction mechanism of distributed database based on LSM-Tree
Uložené v:
| Názov: | Research and implementation of asynchronous compaction mechanism of distributed database based on LSM-Tree |
|---|---|
| Autori: | DU Yide, LIU Wenjie |
| Zdroj: | Xibei Gongye Daxue Xuebao, Vol 42, Iss 2, Pp 303-309 (2024) |
| Informácie o vydavateľovi: | EDP Sciences, 2024. |
| Rok vydania: | 2024 |
| Predmety: | data partitioning, lsm-tree, distributed database, 0202 electrical engineering, electronic engineering, information engineering, data merging, TL1-4050, 02 engineering and technology, asynchronous compaction, Motor vehicles. Aeronautics. Astronautics |
| Popis: | With the continuous development of information technology, distributed database has become a research hotspot. Due to the limit support for SQL and defects in transaction processing and consistency of distributed databases based on NoSQL architecture, NewSQL databases based on LSM-Tree become gradually the mainstream of applications, such as TiDB and OceanBase. The distributed LSM-Tree storage architecture divides the data into baseline data and incremental data. Through the compaction operation, the incremental data of different partitions and the baseline data are continuously merged and stored on the disk, thereby reducing memory pressure. However, compaction will occupy a large amount of system resources and seriously affect system availability. This paper proposes an asynchronous compaction mechanism based on LSM-Tree architecture. By subdividing the compaction process, the data merging is asynchronous, which effectively shortens the time for a single compaction operation. Experiments show that the asynchronous compaction mechanism proposed in this paper can significantly shorten the data merging time and improve the robustness and usability of the system in high-frequency writing scenarios. |
| Druh dokumentu: | Article |
| ISSN: | 2609-7125 1000-2758 |
| DOI: | 10.1051/jnwpu/20244220303 |
| Prístupová URL adresa: | https://doaj.org/article/18c5e7f8a5624f6ba2fcb9cf7f2ef5e0 |
| Rights: | CC BY |
| Prístupové číslo: | edsair.doi.dedup.....74aa876573f2ed0b52f5d8488daf1009 |
| Databáza: | OpenAIRE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstrakt: | With the continuous development of information technology, distributed database has become a research hotspot. Due to the limit support for SQL and defects in transaction processing and consistency of distributed databases based on NoSQL architecture, NewSQL databases based on LSM-Tree become gradually the mainstream of applications, such as TiDB and OceanBase. The distributed LSM-Tree storage architecture divides the data into baseline data and incremental data. Through the compaction operation, the incremental data of different partitions and the baseline data are continuously merged and stored on the disk, thereby reducing memory pressure. However, compaction will occupy a large amount of system resources and seriously affect system availability. This paper proposes an asynchronous compaction mechanism based on LSM-Tree architecture. By subdividing the compaction process, the data merging is asynchronous, which effectively shortens the time for a single compaction operation. Experiments show that the asynchronous compaction mechanism proposed in this paper can significantly shorten the data merging time and improve the robustness and usability of the system in high-frequency writing scenarios. |
|---|---|
| ISSN: | 26097125 10002758 |
| DOI: | 10.1051/jnwpu/20244220303 |