Submodularity of Distributed Join Computation

We study distributed equi-join computation in the presence of join-attribute skew, which causes load imbalance. Skew can be addressed by more fine-grained partitioning, at the cost of input duplication. For random load assignment, e.g., using a hash function, fine-grained partitioning creates a trad...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings - ACM-SIGMOD International Conference on Management of Data Jg. 2018; S. 1237
Hauptverfasser: Li, Rundong, Riedewald, Mirek, Deng, Xinyan
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States 01.06.2018
Schlagworte:
Information systems → Parallel and distributed DBMSs
load balancing
Distributed join
MapReduce algorithms
Computing methodologies → Distributed algorithms
load variance minimization
Theory of computation → Distributed algorithms
ISSN:0730-8078
Online-Zugang:Weitere Angaben
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We study distributed equi-join computation in the presence of join-attribute skew, which causes load imbalance. Skew can be addressed by more fine-grained partitioning, at the cost of input duplication. For random load assignment, e.g., using a hash function, fine-grained partitioning creates a tradeoff between load expectation and variance. We show that minimizing load variance subject to a constraint on expectation is a monotone submodular maximization problem with Knapsack constraints, hence admitting provably near-optimal greedy solutions. In contrast to previous work on formal optimality guarantees, we can prove this result also for self-joins and more general load functions defined as weighted sum of input and output. We further demonstrate through experiments that this theoretical result leads to an effective algorithm for the problem of minimizing running time, even when load is assigned deterministically.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0730-8078
DOI:10.1145/3183713.3183728