Similarity Joins: Their implementation and interactions with other database operators

Similarity Joins are extensively used in multiple application domains and are recognized among the most useful data processing and analysis operations. They retrieve all data pairs whose distances are smaller than a predefined threshold ε. While several standalone implementations have been proposed,...

Full description

Saved in:
Bibliographic Details
Published in:Information systems (Oxford) Vol. 52; pp. 149 - 162
Main Authors: Silva, Yasin N., Pearson, Spencer S., Chon, Jaime, Roberts, Ryan
Format: Journal Article
Language:English
Published: Elsevier Ltd 01.08.2015
Subjects:
Data processing
Database operator
Information systems
Operators
Optimization
PostgreSQL
Query processing
Query processing and optimization
Similarity
Similarity Join
Similarity queries
Transformations
Similarity queries
Query processing and optimization
Database operator
PostgreSQL
Similarity Join
ISSN:0306-4379, 1873-6076
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Similarity Joins are extensively used in multiple application domains and are recognized among the most useful data processing and analysis operations. They retrieve all data pairs whose distances are smaller than a predefined threshold ε. While several standalone implementations have been proposed, very little work has addressed the implementation of Similarity Joins as physical database operators. In this paper, we focus on the study, design, implementation, and optimization of a Similarity Join database operator for metric spaces. We present DBSimJoin, a physical database operator that integrates techniques to: enable a non-blocking behavior, prioritize the early generation of results, and fully support the database iterator interface. The proposed operator can be used with multiple distance functions and data types. We describe the changes in each query engine module to implement DBSimJoin and provide details of our implementation in PostgreSQL. We also study ways in which DBSimJoin can be combined with other similarity and non-similarity operators to answer more complex queries, and how DBSimJoin can be used in query transformation rules to improve query performance. The extensive performance evaluation shows that DBSimJoin significantly outperforms alternative approaches and scales very well when important parameters like ε, data size, and number of dimensions increase.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0306-4379
1873-6076
DOI:10.1016/j.is.2015.01.008