Research on Join Operation of Temporal Big Data in Distributed Environment

Distributed system is an ideal choice for processing temporal large data join operation,but the existing distributed system cannot support the original temporal join query and cannot meet the processing requirements of temporal large data with low latency and high throughput.Therefore,a two-level in...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Ji suan ji gong cheng Ročník 45; číslo 3; s. 20 - 25,31
Hlavní autor: ZHANG Wei,WANG Zhijie
Médium: Journal Article
Jazyk:čínština
angličtina
Vydáno: Editorial Office of Computer Engineering 01.03.2019
Témata:
temporal big data| distributed memory computing| temporal join| two-level index| partition method| spark framework
ISSN:1000-3428
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Distributed system is an ideal choice for processing temporal large data join operation,but the existing distributed system cannot support the original temporal join query and cannot meet the processing requirements of temporal large data with low latency and high throughput.Therefore,a two-level index memory solution scheme based on Spark is proposed.The global index is used to prune the distributed partitions,and the local temporal index is used to query the partitions in order to improve the efficiency of data retrieval.A partition method is designed for temporal data to optimize global pruning.Experimental results based on real and synthetic datasets show that the scheme can significantly improve the processing efficiency of temporal join operation.
ISSN:1000-3428
DOI:10.19678/j.issn.1000-3428.0052626