DOMe: A deduplication optimization method for the NewSQL database backups

Reducing duplicated data of database backups is an important application scenario for data deduplication technology. NewSQL is an emerging database system and is now being used more and more widely. NewSQL systems need to improve data reliability by periodically backing up in-memory data, resulting...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:PloS one Ročník 12; číslo 10; s. e0185189
Hlavní autoři: Wang, Longxiang, Zhu, Zhengdong, Zhang, Xingjun, Dong, Xiaoshe, Wang, Yinfeng
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States Public Library of Science 19.10.2017
Public Library of Science (PLoS)
Témata:
Algorithms
Architectural engineering
Back up systems
Backups
Computer and Information Sciences
Cores
Data base management systems
Data bases
Data processing
Data structures
Databases, Factual
Engineering
Engineering and Technology
Hard disks
Hardware
Hash based algorithms
International conferences
Operating systems
Optimization
Parallel processing
Physical Sciences
Programming Languages
Reproduction (copying)
Research and Analysis Methods
Researchers
SQL (Programming language)
Studies
China
ISSN:1932-6203, 1932-6203
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Reducing duplicated data of database backups is an important application scenario for data deduplication technology. NewSQL is an emerging database system and is now being used more and more widely. NewSQL systems need to improve data reliability by periodically backing up in-memory data, resulting in a lot of duplicated data. The traditional deduplication method is not optimized for the NewSQL server system and cannot take full advantage of hardware resources to optimize deduplication performance. A recent research pointed out that the future NewSQL server will have thousands of CPU cores, large DRAM and huge NVRAM. Therefore, how to utilize these hardware resources to optimize the performance of data deduplication is an important issue. To solve this problem, we propose a deduplication optimization method (DOMe) for NewSQL system backup. To take advantage of the large number of CPU cores in the NewSQL server to optimize deduplication performance, DOMe parallelizes the deduplication method based on the fork-join framework. The fingerprint index, which is the key data structure in the deduplication process, is implemented as pure in-memory hash table, which makes full use of the large DRAM in NewSQL system, eliminating the performance bottleneck problem of fingerprint index existing in traditional deduplication method. The H-store is used as a typical NewSQL database system to implement DOMe method. DOMe is experimentally analyzed by two representative backup data. The experimental results show that: 1) DOMe can reduce the duplicated NewSQL backup data. 2) DOMe significantly improves deduplication performance by parallelizing CDC algorithms. In the case of the theoretical speedup ratio of the server is 20.8, the speedup ratio of DOMe can achieve up to 18; 3) DOMe improved the deduplication throughput by 1.5 times through the pure in-memory index optimization method.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
Competing Interests: The authors have declared that no competing interests exist.
ISSN:1932-6203
1932-6203
DOI:10.1371/journal.pone.0185189