AKIN a streaming graph partitioning algorithm for distributed graph storage systems

Many graph-related applications face the challenge of managing excessive and ever-growing graph data in a distributed environment. Therefore, it is necessary to consider a graph partitioning algorithm to distribute graph data onto multiple machines as the data comes in. Balancing data distribution a...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2018 18th IEEE ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) s. 183 - 192
Hlavní autori: Zhang, Wei, Chen, Yong, Dai, Dong
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: Piscataway, NJ, USA IEEE Press 01.05.2018
IEEE
Edícia:ACM Conferences
Predmet:
Computer science
Distributed databases
distributed system
Electronic mail
graph database
graph partitioning
graph storage
Heuristic algorithms
Measurement
Partitioning algorithms
distributed system
graph database
graph partitioning
graph storage
ISBN:1538658151, 9781538658154
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Many graph-related applications face the challenge of managing excessive and ever-growing graph data in a distributed environment. Therefore, it is necessary to consider a graph partitioning algorithm to distribute graph data onto multiple machines as the data comes in. Balancing data distribution and minimizing edge-cut ratio are two basic pursuits of the graph partitioning problem. While achieving balanced partitions for streaming graphs is easy, existing graph partitioning algorithms either fail to work on streaming workloads, or leave edge-cut ratio to be further improved. Our research aims to provide a better solution that fits the need of streaming graph partitioning in a distributed system, which further reduces the edge-cut ratio while maintaining rough balance among all partitions. We exploit the similarity measure on the degree of vertices to gather structural-related vertices in the same partition as much as possible, this reduces the edge-cut ratio even further as compared to the state-of-the-art streaming graph partitioning algorithm - FENNEL. Our evaluation shows that our streaming graph partitioning algorithm is able to achieve better partitioning quality in terms of edge-cut ratio (up to 20% reduction as compared to FENNEL) while maintaining decent balance between all partitions, and such improvement applies to various real-life graphs.
ISBN:1538658151
9781538658154
DOI:10.1109/CCGRID.2018.00033