View in EDS

Aion: Live Migration for In-Memory Databases with Zero Downtime and Reduced Redundant Data Transfer

Saved in:
Bibliographic Details
Title: Aion: Live Migration for In-Memory Databases with Zero Downtime and Reduced Redundant Data Transfer
Authors: Huijie Cao, Chenfeng Huang, Shengchi Liu, Huiqi Hu, Minghao Zhao, Xuan Zhou, Yaofeng Tu, Weining Qian
Source: Data Science and Engineering, Vol 10, Iss 2, Pp 212-229 (2025)
Publisher Information: Springer Science and Business Media LLC, 2025.
Publication Year: 2025
Subject Terms: Electronic computers. Computer science, Distributed database, Information technology, QA75.5-76.95, Live migration, In-memory database, T58.5-58.64, Concurrency control
Description: Distributed in-memory databases are widely adopted to achieve low latency and high bandwidth for data-intensive applications. They support scale-out by sharding and distributing data across multiple nodes. To efficiently adapt to various workloads, distributed in-memory databases must be capable of migrating shards across nodes. In this paper, we demonstrate that state-of-the-art approaches experience significant performance degradation during migration due to service downtime and redundant data transfer. Furthermore, our findings indicate that the presence of service downtime constrains the scalability of migration strategies, while the transfer of redundant data during the snapshot transfer phase limits their adaptability to dynamic workloads. To this end, this paper proposes Aion, a live migration strategy designed for distributed in-memory databases. Aion eliminates any potential service downtime by immediately switching transaction routing to the destination node. To ensure data consistency between the source and destination nodes, as well as serializable execution during migration, Aion proposes the mutual validation phase. Moreover, Aion introduces an analysis phase before the snapshot transfer phase to identify dynamically changing hotspots in workloads. The analysis phase identifies and transfers tuples and versions accessed less frequently to the destination node, reducing the amount of data transferred. Aion is implemented on a distributed in-memory database and evaluated using various OLTP workloads. The results demonstrate that Aion can fundamentally eliminate service downtime, adapt effectively to various workloads and exhibit robust scalability. Compared to state-of-the-art approaches, Aion achieves up to 2.25x–6.57x higher throughput during migration and shortens the migration duration by 53.7–68.2%.
Document Type: Article
Language: English
ISSN: 2364-1541
2364-1185
DOI: 10.1007/s41019-024-00276-5
Access URL: https://doaj.org/article/86845ebd676e4146bf8ce977048d26e0
Rights: CC BY
Accession Number: edsair.doi.dedup.....58cb50e2bdd414b691cf739c9d7e6768
Database: OpenAIRE
Description
Abstract:Distributed in-memory databases are widely adopted to achieve low latency and high bandwidth for data-intensive applications. They support scale-out by sharding and distributing data across multiple nodes. To efficiently adapt to various workloads, distributed in-memory databases must be capable of migrating shards across nodes. In this paper, we demonstrate that state-of-the-art approaches experience significant performance degradation during migration due to service downtime and redundant data transfer. Furthermore, our findings indicate that the presence of service downtime constrains the scalability of migration strategies, while the transfer of redundant data during the snapshot transfer phase limits their adaptability to dynamic workloads. To this end, this paper proposes Aion, a live migration strategy designed for distributed in-memory databases. Aion eliminates any potential service downtime by immediately switching transaction routing to the destination node. To ensure data consistency between the source and destination nodes, as well as serializable execution during migration, Aion proposes the mutual validation phase. Moreover, Aion introduces an analysis phase before the snapshot transfer phase to identify dynamically changing hotspots in workloads. The analysis phase identifies and transfers tuples and versions accessed less frequently to the destination node, reducing the amount of data transferred. Aion is implemented on a distributed in-memory database and evaluated using various OLTP workloads. The results demonstrate that Aion can fundamentally eliminate service downtime, adapt effectively to various workloads and exhibit robust scalability. Compared to state-of-the-art approaches, Aion achieves up to 2.25x–6.57x higher throughput during migration and shortens the migration duration by 53.7–68.2%.
ISSN:23641541
23641185
DOI:10.1007/s41019-024-00276-5