Efficient Processing of Skyline Queries Using MapReduce

The skyline operator has attracted considerable attention recently due to its broad applications. However, computing a skyline is challenging today since we have to deal with big data. For data-intensive applications, the MapReduce framework has been widely used recently. In this paper, we propose t...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on knowledge and data engineering Ročník 29; číslo 5; s. 1031 - 1044
Hlavní autoři: Park, Yoonjae, Min, Jun-Ki, Shim, Kyuseok
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.05.2017
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithms
Big data
Buildings
Data management
distributed and parallel algorithms
Histograms
Indexes
MapReduce algorithms
Parallel algorithms
Partitioning algorithms
Partitions
Query processing
Scalability
Skyline queries
State of the art
Two dimensional displays
ISSN:1041-4347, 1558-2191
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The skyline operator has attracted considerable attention recently due to its broad applications. However, computing a skyline is challenging today since we have to deal with big data. For data-intensive applications, the MapReduce framework has been widely used recently. In this paper, we propose the efficient parallel algorithm SKY-MR + for processing skyline queries using MapReduce. We first build a quadtree-based histogram for space partitioning by deciding whether to split each leaf node judiciously based on the benefit of splitting in terms of the estimated execution time. In addition, we apply the dominance power filtering method to effectively prune non-skyline points in advance. We next partition data based on the regions divided by the quadtree and compute candidate skyline points for each partition using MapReduce. Finally, we check whether each skyline candidate point is actually a skyline point in every partition using MapReduce. We also develop the workload balancing methods to make the estimated execution times of all available machines to be similar. We did experiments to compare SKY-MR + with the state-of-the-art algorithms using MapReduce and confirmed the effectiveness as well as the scalability of SKY-MR + .
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1041-4347
1558-2191
DOI:10.1109/TKDE.2017.2654459