LSH SimilarityJoin Pattern in FastFlow

Similarity joins are recognized to be among the most used data processing and analysis operations. We introduce a C++-based high-level parallel pattern implemented on top of FastFlow Building Blocks to provide the programmer with ready-to-use similarity join computations. The SimilarityJoin pattern...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:International journal of parallel programming Ročník 52; číslo 3; s. 207 - 230
Hlavní autoři: Tonci, Nicolò, Rivault, Sébastien, Bamha, Mostafa, Robert, Sophie, Limet, Sébastien, Torquati, Massimo
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.06.2024
Springer Nature B.V
Springer Verlag
Témata:
Algorithms
C++ (programming language)
Computer Science
Data processing
Data structures
Datasets
Distributed memory
Performance assessment
Processor Architectures
Similarity
Software Engineering/Programming and Operating Systems
Theory of Computation
LSH Similarity join
Distributed programming
Big data MapReduce
Parallel patterns
High-level parallel programming
Big data
MapReduce
ISSN:0885-7458, 1573-7640
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Similarity joins are recognized to be among the most used data processing and analysis operations. We introduce a C++-based high-level parallel pattern implemented on top of FastFlow Building Blocks to provide the programmer with ready-to-use similarity join computations. The SimilarityJoin pattern is implemented according to the MapReduce paradigm enriched with locality sensitive hashing (LSH) to optimize the whole computation. The new parallel pattern can be used with any C++ serializable data structure and executed on shared- and distributed-memory machines. We present experimental validations of the proposed solution considering two different clusters and small and large input datasets to evaluate in-core and out-of-core executions. The performance assessment of the SimilarityJoin pattern has been conducted by comparing the execution time against the one obtained from the original hand-tuned Hadoop-based implementation of the LSH-based similarity join algorithms as well as a Spark-based version. The experiments show that the SimilarityJoin pattern: (1) offers a significant performance improvement for small and medium datasets; (2) is competitive also for computations using large input datasets producing out-of-core executions.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-024-00772-1