Efficient Processing of Skyline Queries Using MapReduce

The skyline operator has attracted considerable attention recently due to its broad applications. However, computing a skyline is challenging today since we have to deal with big data. For data-intensive applications, the MapReduce framework has been widely used recently. In this paper, we propose t...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on knowledge and data engineering Ročník 29; číslo 5; s. 1031 - 1044
Hlavní autori:	Park, Yoonjae, Min, Jun-Ki, Shim, Kyuseok
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York IEEE 01.05.2017 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Algorithms Big data Buildings Data management distributed and parallel algorithms Histograms Indexes MapReduce algorithms Parallel algorithms Partitioning algorithms Partitions Query processing Scalability Skyline queries State of the art Two dimensional displays
ISSN:	1041-4347, 1558-2191
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	The skyline operator has attracted considerable attention recently due to its broad applications. However, computing a skyline is challenging today since we have to deal with big data. For data-intensive applications, the MapReduce framework has been widely used recently. In this paper, we propose the efficient parallel algorithm SKY-MR + for processing skyline queries using MapReduce. We first build a quadtree-based histogram for space partitioning by deciding whether to split each leaf node judiciously based on the benefit of splitting in terms of the estimated execution time. In addition, we apply the dominance power filtering method to effectively prune non-skyline points in advance. We next partition data based on the regions divided by the quadtree and compute candidate skyline points for each partition using MapReduce. Finally, we check whether each skyline candidate point is actually a skyline point in every partition using MapReduce. We also develop the workload balancing methods to make the estimated execution times of all available machines to be similar. We did experiments to compare SKY-MR + with the state-of-the-art algorithms using MapReduce and confirmed the effectiveness as well as the scalability of SKY-MR + .
AbstractList	The skyline operator has attracted considerable attention recently due to its broad applications. However, computing a skyline is challenging today since we have to deal with big data. For data-intensive applications, the MapReduce framework has been widely used recently. In this paper, we propose the efficient parallel algorithm SKY-MR + for processing skyline queries using MapReduce. We first build a quadtree-based histogram for space partitioning by deciding whether to split each leaf node judiciously based on the benefit of splitting in terms of the estimated execution time. In addition, we apply the dominance power filtering method to effectively prune non-skyline points in advance. We next partition data based on the regions divided by the quadtree and compute candidate skyline points for each partition using MapReduce. Finally, we check whether each skyline candidate point is actually a skyline point in every partition using MapReduce. We also develop the workload balancing methods to make the estimated execution times of all available machines to be similar. We did experiments to compare SKY-MR + with the state-of-the-art algorithms using MapReduce and confirmed the effectiveness as well as the scalability of SKY-MR + .
Author	Kyuseok Shim Yoonjae Park Jun-Ki Min
Author_xml	– sequence: 1 givenname: Yoonjae surname: Park fullname: Park, Yoonjae – sequence: 2 givenname: Jun-Ki surname: Min fullname: Min, Jun-Ki – sequence: 3 givenname: Kyuseok surname: Shim fullname: Shim, Kyuseok
BookMark	eNp9kMtOwzAQRS0EEm3hAxCbSKxT_IztJSrlIYp4tWvLcSfIpTjFThb9exJasWDBbGYsnzMj3SE6DHUAhM4IHhOC9eX84Xo6ppjIMS0E50IfoAERQuWUaHLYzZiTnDMuj9EwpRXGWElFBkhOq8o7D6HJnmPtICUf3rO6yt4-tmsfIHtpIXpI2eLn49FuXmHZOjhBR5VdJzjd9xFa3Eznk7t89nR7P7ma5Y5q1uRSlFpxuSxwBa4suGaaOGYV5ppzxZel1v1LF9pSUGWhLDhqhbSqoFxQx0boYrd3E-uvFlJjVnUbQ3fSUCJ5V0SwjpI7ysU6pQiVcb6xja9DE61fG4JNn5LpUzJ9SmafUmeSP-Ym-k8bt_865zvHA8AvL1UHKcq-AX2JcrM
CODEN	ITKEEH
CitedBy_id	crossref_primary_10_1016_j_knosys_2019_06_003 crossref_primary_10_1007_s44443_025_00183_3 crossref_primary_10_1109_TKDE_2020_3021470 crossref_primary_10_1007_s10619_020_07297_9 crossref_primary_10_1007_s11704_020_9246_2 crossref_primary_10_1007_s10115_018_1310_y crossref_primary_10_1109_JIOT_2018_2882820 crossref_primary_10_1007_s11704_020_0122_x crossref_primary_10_1109_TC_2022_3140884 crossref_primary_10_1007_s11227_017_2171_y crossref_primary_10_1016_j_is_2019_101443 crossref_primary_10_1007_s12293_018_0271_8 crossref_primary_10_1007_s10586_017_1070_6 crossref_primary_10_1007_s41019_024_00261_y crossref_primary_10_1016_j_chb_2018_10_009 crossref_primary_10_1002_cpe_4848 crossref_primary_10_3390_app10051708
Cites_doi	10.1109/TKDE.2011.64 10.1145/1559845.1559899 10.1145/2463676.2465324 10.1109/TPDS.2015.2472016 10.1109/ICDE.2011.5767896 10.1145/2274576.2274605 10.1145/1376616.1376642 10.1109/BigData.Congress.2013.35 10.1145/1412331.1412343 10.1016/j.adhoc.2015.07.006 10.1109/ICDE.2010.5447780 10.1145/1989323.1989333 10.1137/0117039 10.1109/ICDE.2001.914855 10.1145/872757.872814 10.1016/B978-155860869-6/50032-9 10.14778/2824032.2824040 10.14778/2556549.2556580 10.1109/ICDE.2003.1260846 10.1145/1327452.1327492 10.1007/978-3-642-12098-5_5 10.1098/rspl.1895.0041 10.2307/2332226 10.1016/j.is.2013.05.005 10.1109/TKDE.2006.48 10.1109/TKDE.2008.142 10.1007/978-3-642-15883-4_13 10.1016/j.is.2014.11.005
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2017
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2017
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/TKDE.2017.2654459
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Computer Science
EISSN	1558-2191
EndPage	1044
ExternalDocumentID	10_1109_TKDE_2017_2654459 7820182
Genre	orig-research
GrantInformation_xml	– fundername: Next-Generation Information Computing Development Program – fundername: IITP grantid: IITP-2016-H8501-16-1013 – fundername: Ministry of Education grantid: 2015R1D1A1A01058909 funderid: 10.13039/501100002701 – fundername: Basic Science Research Program – fundername: Ministry of Science, ICT & Future Planning grantid: NRF-2012M3C4A7033342 funderid: 10.13039/501100003621 – fundername: ITRC – fundername: National Research Foundation of Korea (NRF) funderid: 10.13039/501100003725
GroupedDBID	-~X .DC 0R~ 29I 4.4 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ IEDLZ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS RXW TAE TN5 UHB AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D RIG
ID	FETCH-LOGICAL-c293t-75b9847d60fecb649391c3a80494484db993a80969a2e8b68aec2a57a862452c3
IEDL.DBID	RIE
ISICitedReferencesCount	24
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000399289300008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1041-4347
IngestDate	Mon Jun 30 03:45:38 EDT 2025 Sat Nov 29 04:46:39 EST 2025 Tue Nov 18 22:27:35 EST 2025 Wed Aug 27 02:52:16 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	5
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c293t-75b9847d60fecb649391c3a80494484db993a80969a2e8b68aec2a57a862452c3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
PQID	2174444153
PQPubID	85438
PageCount	14
ParticipantIDs	proquest_journals_2174444153 crossref_citationtrail_10_1109_TKDE_2017_2654459 crossref_primary_10_1109_TKDE_2017_2654459 ieee_primary_7820182
PublicationCentury	2000
PublicationDate	2017-05-01
PublicationDateYYYYMMDD	2017-05-01
PublicationDate_xml	– month: 05 year: 2017 text: 2017-05-01 day: 01
PublicationDecade	2010
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on knowledge and data engineering
PublicationTitleAbbrev	TKDE
PublicationYear	2017
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref35 ref13 ref15 ref14 ref31 ref30 ref33 ref32 ref10 ref2 ref1 ref16 ref19 ref18 (ref17) 2006 tao (ref22) 2006; 18 zhang (ref12) 2011 huang (ref26) 2006 ref24 ref23 pearson (ref34) 1895; 58 ref21 tan (ref20) 2001 mullesgaard (ref11) 2014 ref28 ref27 ref29 ref8 ref7 ref9 ref4 dellis (ref3) 2007 ref6 ref5 bartolini (ref25) 2006
References_xml	– year: 2006 ident: ref17 – ident: ref6 doi: 10.1109/TKDE.2011.64 – ident: ref33 doi: 10.1145/1559845.1559899 – ident: ref9 doi: 10.1145/2463676.2465324 – ident: ref13 doi: 10.1109/TPDS.2015.2472016 – ident: ref23 doi: 10.1109/ICDE.2011.5767896 – start-page: 66 year: 2006 ident: ref26 article-title: Skyline queries against mobile lightweight devices in MANETs publication-title: Proc 2nd Int Conf Data Eng – ident: ref27 doi: 10.1145/2274576.2274605 – ident: ref32 doi: 10.1145/1376616.1376642 – ident: ref30 doi: 10.1109/BigData.Congress.2013.35 – ident: ref21 doi: 10.1145/1412331.1412343 – ident: ref31 doi: 10.1016/j.adhoc.2015.07.006 – start-page: 403 year: 2011 ident: ref12 article-title: Adapting skyline computation to the MapReduce framework: Algorithms and experiments publication-title: Proc Int Conf Database Syst Adv Appl – ident: ref4 doi: 10.1109/ICDE.2010.5447780 – ident: ref28 doi: 10.1145/1989323.1989333 – ident: ref15 doi: 10.1137/0117039 – ident: ref1 doi: 10.1109/ICDE.2001.914855 – ident: ref2 doi: 10.1145/872757.872814 – ident: ref19 doi: 10.1016/B978-155860869-6/50032-9 – start-page: 37 year: 2014 ident: ref11 article-title: Efficient skyline computation in MapReduce publication-title: Proc Intl Conf Extending Database Technology – ident: ref16 doi: 10.14778/2824032.2824040 – ident: ref14 doi: 10.14778/2556549.2556580 – ident: ref18 doi: 10.1109/ICDE.2003.1260846 – start-page: 291 year: 2007 ident: ref3 article-title: Efficient computation of reverse skyline queries publication-title: Proc Int Conf On Very Large Data Bases – ident: ref10 doi: 10.1145/1327452.1327492 – ident: ref7 doi: 10.1007/978-3-642-12098-5_5 – start-page: 405 year: 2006 ident: ref25 article-title: SaLSa: Computing the skyline without scanning the whole sky publication-title: Proc 15th ACM Int Conf Inf Knowl Manage – start-page: 301 year: 2001 ident: ref20 article-title: Efficient progressive skyline computation publication-title: Proc 27th Int Conf Very Large Data Bases – volume: 58 start-page: 240 year: 1895 ident: ref34 article-title: Note on regression and inheritance in the case of two parents publication-title: Proc Roy Soc London doi: 10.1098/rspl.1895.0041 – ident: ref35 doi: 10.2307/2332226 – ident: ref24 doi: 10.1016/j.is.2013.05.005 – volume: 18 start-page: 377 year: 2006 ident: ref22 article-title: Maintaining sliding window skylines on data streams publication-title: IEEE Trans Knowl Data Eng doi: 10.1109/TKDE.2006.48 – ident: ref29 doi: 10.1109/TKDE.2008.142 – ident: ref5 doi: 10.1007/978-3-642-15883-4_13 – ident: ref8 doi: 10.1016/j.is.2014.11.005
SSID	ssj0008781
Score	2.341384
Snippet	The skyline operator has attracted considerable attention recently due to its broad applications. However, computing a skyline is challenging today since we...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1031
SubjectTerms	Algorithms Big data Buildings Data management distributed and parallel algorithms Histograms Indexes MapReduce algorithms Parallel algorithms Partitioning algorithms Partitions Query processing Scalability Skyline queries State of the art Two dimensional displays
Title	Efficient Processing of Skyline Queries Using MapReduce
URI	https://ieeexplore.ieee.org/document/7820182 https://www.proquest.com/docview/2174444153
Volume	29
WOSCitedRecordID	wos000399289300008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2191 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0008781 issn: 1041-4347 databaseCode: RIE dateStart: 19890101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5q8aAHq61itUoOnsTtYzebbI6iLYJYfFTobclrQZS29CH4782kaVEUwdvukoTlyyYzs5n5PoCzREmeWllEsbU6ooqaSBVxESXaxR9UdURqMi82wfv9bDgU9yW4WNfCWGt98plt4qU_yzdjvcBfZS3kdnP-8AZscM6WtVrrXTfjXpDURRcuJkooDyeYnbZoDW6vu5jExZsxQ-4Z8c0GeVGVHzuxNy-9yv9ebBd2ghtJLpfzvgclO6pCZSXRQMKKrcL2F77BGvCuJ4xwY5FQIOAek3FBnl4_0N0kDwvkPZ4Rn0hA7uTkEZld7T4897qDq5soKCdE2pnvecRTJZzZMaxdWK0YFYno6ERmSAZDM2qU80rcnWBCxjZTLJNWxzLlEstF0lgnB1AejUf2EIi0SCpXMGaUpoayTBnDRZEaYVM30Z06tFdY5jrQiqO6xVvuw4u2yBH-HOHPA_x1OF93mSw5Nf5qXEO81w0D1HVorCYsD6tulmN4RTFATI5-73UMWzj2MmGxAeX5dGFPYFO_z19m01P_QX0CJTHHDg
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD7MKagP3qY4r3nwSey2tmnTPIpOlM3hZYJvJbeCKJu4TfDfm5NlRVEE39qStOWkyTlfc873ARzFUrDEiCKIjFEBlVQHsoiKIFYWf1AZ8kRnTmyC9XrZ4yO_qcBJWQtjjHHJZ6aBh24vXw_VBH-VNZHbzcbDczCfUIt7ptVa5bqbMSdJavGFRUUxZX4PM2zxZr9z3sY0LtaIUmSf4d-8kJNV-bEWOwdzsfq_V1uDFR9IktPpyK9DxQw2YHUm0kD8nN2A5S-MgzVgbUcZYe9FfImAvUyGBbl__sCAk9xOkPl4RFwqAbkWr3fI7Wo24eGi3T-7DLx2QqCsAx8HLJHcOh6dtgqjZEp5zEMViwzpYGhGtbRxiT3jKReRyWSaCaMikTCBBSNJpOItqA6GA7MNRBiklSvSVEtFNU0zqTXjRaK5SexQh3VozWyZK08sjvoWL7kDGC2eo_lzNH_uzV-H47LL65RV46_GNbR32dCbug57swHL_bwb5QiwKELEeOf3XoeweNm_7ubdq15nF5bwOdP0xT2ojt8mZh8W1Pv4afR24D6uT61eylU
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficient+Processing+of+Skyline+Queries+Using+MapReduce&rft.jtitle=IEEE+transactions+on+knowledge+and+data+engineering&rft.au=Park%2C+Yoonjae&rft.au=Jun-Ki%2C+Min&rft.au=Shim%2C+Kyuseok&rft.date=2017-05-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1041-4347&rft.eissn=1558-2191&rft.volume=29&rft.issue=5&rft.spage=1031&rft_id=info:doi/10.1109%2FTKDE.2017.2654459&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1041-4347&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1041-4347&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1041-4347&client=summon