Towards Fast Theta-join: A Prefiltering and Amalgamated Partitioning Approach
As one of the most useful online processing techniques, the theta-join operation has been utilized by many applications to fully excavate the relationships between data streams in various scenarios. As such, constant research efforts have been put to optimize its performance in the distributed envir...
Gespeichert in:
| Veröffentlicht in: | arXiv.org |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Paper |
| Sprache: | Englisch |
| Veröffentlicht: |
Ithaca
Cornell University Library, arXiv.org
05.08.2022
|
| Schlagworte: | |
| ISSN: | 2331-8422 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | As one of the most useful online processing techniques, the theta-join operation has been utilized by many applications to fully excavate the relationships between data streams in various scenarios. As such, constant research efforts have been put to optimize its performance in the distributed environment, which is typically characterized by reducing the number of Cartesian products as much as possible. In this article, we design and implement a novel fast theta-join algorithm, called Prefap, by developing two distinct techniques - prefiltering and amalgamated partitioning-based on the state-of-the-art FastThetaJoin algorithm to optimize the efficiency of the theta-join operation. Firstly, we develop a prefiltering strategy before data streams are partitioned to reduce the amount of data to be involved and benefit a more fine-grained partitioning. Secondly, to avoid the data streams being partitioned in a coarse-grained isolated manner and improve the quality of the partition-level filtering, we introduce an amalgamated partitioning mechanism that can amalgamate the partitioning boundaries of two data streams to assist a fine-grained partitioning. With the integration of these two techniques into the existing FastThetaJoin algorithm, we design and implement a new framework to achieve a decreased number of Cartesian products and a higher theta-join efficiency. By comparing with existing algorithms, FastThetaJoin in particular, we evaluate the performance of Prefap on both synthetic and real data streams from two-way to multiway theta-join to demonstrate its superiority. |
|---|---|
| AbstractList | As one of the most useful online processing techniques, the theta-join operation has been utilized by many applications to fully excavate the relationships between data streams in various scenarios. As such, constant research efforts have been put to optimize its performance in the distributed environment, which is typically characterized by reducing the number of Cartesian products as much as possible. In this article, we design and implement a novel fast theta-join algorithm, called Prefap, by developing two distinct techniques - prefiltering and amalgamated partitioning-based on the state-of-the-art FastThetaJoin algorithm to optimize the efficiency of the theta-join operation. Firstly, we develop a prefiltering strategy before data streams are partitioned to reduce the amount of data to be involved and benefit a more fine-grained partitioning. Secondly, to avoid the data streams being partitioned in a coarse-grained isolated manner and improve the quality of the partition-level filtering, we introduce an amalgamated partitioning mechanism that can amalgamate the partitioning boundaries of two data streams to assist a fine-grained partitioning. With the integration of these two techniques into the existing FastThetaJoin algorithm, we design and implement a new framework to achieve a decreased number of Cartesian products and a higher theta-join efficiency. By comparing with existing algorithms, FastThetaJoin in particular, we evaluate the performance of Prefap on both synthetic and real data streams from two-way to multiway theta-join to demonstrate its superiority. |
| Author | Wang, Yang Xu, Chengzhong Ye, Kejiang Wu, Jiashu Fan, Xiaopeng |
| Author_xml | – sequence: 1 givenname: Jiashu surname: Wu fullname: Wu, Jiashu – sequence: 2 givenname: Yang surname: Wang fullname: Wang, Yang – sequence: 3 givenname: Xiaopeng surname: Fan fullname: Fan, Xiaopeng – sequence: 4 givenname: Kejiang surname: Ye fullname: Ye, Kejiang – sequence: 5 givenname: Chengzhong surname: Xu fullname: Xu, Chengzhong |
| BookMark | eNotjV1LwzAYRoMoOOd-gHcBr1vTNx9NvCvDqTBxF70f79Jka-nSmWbqz7eicOC5OPCcG3IZhuAIuStYLrSU7AHjd_uZAzCdMzC6uCAz4LzItAC4Jotx7BhjoEqQks_IWz18YWxGusIx0frgEmbd0IZHWtFNdL7tk4tt2FMMDa2O2O_xiMk1dIMxtakdwq-sTqc4oD3ckiuP_egW_zsn9eqpXr5k6_fn12W1zlCCzKzcWYFWGeG4L51tUHhVupJpv_OSGyiVBiacYRPe26IohXJGeAlFIy3nc3L_dztVP85uTNtuOMcwFbegjNGgGEj-A6smUQg |
| ContentType | Paper |
| Copyright | 2022. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: 2022. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| DOI | 10.48550/arxiv.2208.02981 |
| DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central UK/Ireland ProQuest Central Essentials - QC ProQuest Central ProQuest Technology Collection ProQuest One ProQuest Central SciTech Premium Collection ProQuest Engineering Collection Engineering Database ProQuest Central Premium ProQuest One Academic (New) Publicly Available Content Database ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
| DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest One Academic Middle East (New) ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest Central (New) ProQuest One Academic ProQuest One Academic (New) Engineering Collection |
| DatabaseTitleList | Publicly Available Content Database |
| Database_xml | – sequence: 1 dbid: PIMPY name: Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Physics |
| EISSN | 2331-8422 |
| Genre | Working Paper/Pre-Print |
| GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| ID | FETCH-LOGICAL-a525-c5bc4ac694e3f7ecda4f67e708fbf5392768204e90e90ffc11746e94f521d5c33 |
| IEDL.DBID | BENPR |
| IngestDate | Mon Jun 30 09:20:10 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a525-c5bc4ac694e3f7ecda4f67e708fbf5392768204e90e90ffc11746e94f521d5c33 |
| Notes | SourceType-Working Papers-1 ObjectType-Working Paper/Pre-Print-1 content type line 50 |
| OpenAccessLink | https://www.proquest.com/docview/2699826025?pq-origsite=%requestingapplication% |
| PQID | 2699826025 |
| PQPubID | 2050157 |
| ParticipantIDs | proquest_journals_2699826025 |
| PublicationCentury | 2000 |
| PublicationDate | 20220805 |
| PublicationDateYYYYMMDD | 2022-08-05 |
| PublicationDate_xml | – month: 08 year: 2022 text: 20220805 day: 05 |
| PublicationDecade | 2020 |
| PublicationPlace | Ithaca |
| PublicationPlace_xml | – name: Ithaca |
| PublicationTitle | arXiv.org |
| PublicationYear | 2022 |
| Publisher | Cornell University Library, arXiv.org |
| Publisher_xml | – name: Cornell University Library, arXiv.org |
| SSID | ssj0002672553 |
| Score | 1.802771 |
| SecondaryResourceType | preprint |
| Snippet | As one of the most useful online processing techniques, the theta-join operation has been utilized by many applications to fully excavate the relationships... |
| SourceID | proquest |
| SourceType | Aggregation Database |
| SubjectTerms | Algorithms Amalgamation Cartesian coordinates Data transmission Partitioning Performance evaluation |
| Title | Towards Fast Theta-join: A Prefiltering and Amalgamated Partitioning Approach |
| URI | https://www.proquest.com/docview/2699826025 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8NAEF60VfDkGx-17MFrbLPZ3SRepEqLgi1Bi9RT2ewDIpjUJBZ_vrNxqwfBi5DLspdlM8x838y3MwidK6kpp9L3JPc5EBSeerEKAciRgBlbFuPNOKCn-3AyiWazOHEJt8rJKlc-sXHUqpA2R94jHIgBgG_CrhZvnp0aZaurboTGOmrbTmVg5-3r4SR5-M6yEB4CZg6-yplN866eKD-y5QUhVkRJ4sj_5YSbyDLa_u-ZdlA7EQtd7qI1ne-hzUbRKat9NJ42gtgKj0RVY7CGWngvRZZf4gFOICxmtkwOcQuLXOHBq33SAeBVK5xYY3JpWjxwLccP0HQ0nN7cem52gicYYZ5kqaRC8pjqwIRaKkEND3XYj0xqGGAiYBmkT3Xch88Y6QMx4TqmBqK5YjIIDlErL3J9hDCBpYgM7KiQRpLGlAYiZZYYCh4xfYw6q8uZO_uv5j83c_L39inaIvZBgRVhsA5q1eW7PkMbcllnVdl1v7NrFZmPsEruxsnzJ94NrLc |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V07T8MwELaqFgQTb_Eo4AHG0NaxnQQJoQqoWvWhDBEqU-Q6thQk0pKUAj-K_8g5TWBAYmNAyhJ5su98993ddz6EziKpKKeyZUne4hCg8InlRQ4AOWIzbcpiPB8HdD9wRiN3PPb8Cvooe2EMrbK0ibmhjqbS5MgbhENgAOCbsOvZs2WmRpnqajlCY6kWffX-CiFbdtW7BfmeE9K5C266VjFVwBKMMEuyiaRCco8qWztKRoJq7iin6eqJZoAWAH-TJlVeEz6tZQsgO1ce1eDnIiZN_hMsfo2CrjerqOb3hv7DV1KHcAcgur2snuZvhTVE-hYvLggxnE3iua0fNj93ZJ2Nf3YEm7B1MVPpFqqoZBut5nxVme2gYZDTfTPcEdkcg67PhfU4jZNL3MY-OP3YkADAK2ORRLj9ZBpWAJqrCPvmqhRJaNwuHlTfRcFfbGAPVZNpovYRJvArXA0rkUNdST1KbTFhJuwV3GXqANVLWYTF7c7Cb0Ec_r58ita6wXAQDnqj_hFaJ6Z1wtBNWB1V5-mLOkYrcjGPs_Sk0CSMwj8W3CdjMwZa |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Towards+Fast+Theta-join%3A+A+Prefiltering+and+Amalgamated+Partitioning+Approach&rft.jtitle=arXiv.org&rft.au=Wu%2C+Jiashu&rft.au=Wang%2C+Yang&rft.au=Fan%2C+Xiaopeng&rft.au=Ye%2C+Kejiang&rft.date=2022-08-05&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2208.02981 |