Zobrazit v EDS

Aion: Live Migration for In-Memory Databases with Zero Downtime and Reduced Redundant Data Transfer

Uloženo v:
Podrobná bibliografie
Název: Aion: Live Migration for In-Memory Databases with Zero Downtime and Reduced Redundant Data Transfer
Autoři: Huijie Cao, Chenfeng Huang, Shengchi Liu, Huiqi Hu, Minghao Zhao, Xuan Zhou, Yaofeng Tu, Weining Qian
Zdroj: Data Science and Engineering, Vol 10, Iss 2, Pp 212-229 (2025)
Informace o vydavateli: Springer Science and Business Media LLC, 2025.
Rok vydání: 2025
Témata: Electronic computers. Computer science, Distributed database, Information technology, QA75.5-76.95, Live migration, In-memory database, T58.5-58.64, Concurrency control
Popis: Distributed in-memory databases are widely adopted to achieve low latency and high bandwidth for data-intensive applications. They support scale-out by sharding and distributing data across multiple nodes. To efficiently adapt to various workloads, distributed in-memory databases must be capable of migrating shards across nodes. In this paper, we demonstrate that state-of-the-art approaches experience significant performance degradation during migration due to service downtime and redundant data transfer. Furthermore, our findings indicate that the presence of service downtime constrains the scalability of migration strategies, while the transfer of redundant data during the snapshot transfer phase limits their adaptability to dynamic workloads. To this end, this paper proposes Aion, a live migration strategy designed for distributed in-memory databases. Aion eliminates any potential service downtime by immediately switching transaction routing to the destination node. To ensure data consistency between the source and destination nodes, as well as serializable execution during migration, Aion proposes the mutual validation phase. Moreover, Aion introduces an analysis phase before the snapshot transfer phase to identify dynamically changing hotspots in workloads. The analysis phase identifies and transfers tuples and versions accessed less frequently to the destination node, reducing the amount of data transferred. Aion is implemented on a distributed in-memory database and evaluated using various OLTP workloads. The results demonstrate that Aion can fundamentally eliminate service downtime, adapt effectively to various workloads and exhibit robust scalability. Compared to state-of-the-art approaches, Aion achieves up to 2.25x–6.57x higher throughput during migration and shortens the migration duration by 53.7–68.2%.
Druh dokumentu: Article
Jazyk: English
ISSN: 2364-1541
2364-1185
DOI: 10.1007/s41019-024-00276-5
Přístupová URL adresa: https://doaj.org/article/86845ebd676e4146bf8ce977048d26e0
Rights: CC BY
Přístupové číslo: edsair.doi.dedup.....58cb50e2bdd414b691cf739c9d7e6768
Databáze: OpenAIRE
Popis
Abstrakt:Distributed in-memory databases are widely adopted to achieve low latency and high bandwidth for data-intensive applications. They support scale-out by sharding and distributing data across multiple nodes. To efficiently adapt to various workloads, distributed in-memory databases must be capable of migrating shards across nodes. In this paper, we demonstrate that state-of-the-art approaches experience significant performance degradation during migration due to service downtime and redundant data transfer. Furthermore, our findings indicate that the presence of service downtime constrains the scalability of migration strategies, while the transfer of redundant data during the snapshot transfer phase limits their adaptability to dynamic workloads. To this end, this paper proposes Aion, a live migration strategy designed for distributed in-memory databases. Aion eliminates any potential service downtime by immediately switching transaction routing to the destination node. To ensure data consistency between the source and destination nodes, as well as serializable execution during migration, Aion proposes the mutual validation phase. Moreover, Aion introduces an analysis phase before the snapshot transfer phase to identify dynamically changing hotspots in workloads. The analysis phase identifies and transfers tuples and versions accessed less frequently to the destination node, reducing the amount of data transferred. Aion is implemented on a distributed in-memory database and evaluated using various OLTP workloads. The results demonstrate that Aion can fundamentally eliminate service downtime, adapt effectively to various workloads and exhibit robust scalability. Compared to state-of-the-art approaches, Aion achieves up to 2.25x–6.57x higher throughput during migration and shortens the migration duration by 53.7–68.2%.
ISSN:23641541
23641185
DOI:10.1007/s41019-024-00276-5