A meta-graph approach to analyze subgraph-centric distributed programming models

Component-centric distributed graph processing models that use bulk synchronous parallel (BSP) execution have grown popular. These overcome short-comings of Big Data platforms like Hadoop for processing large graphs. However, literature on formal analysis of these component-centric abstractions for...

Full description

Saved in:
Bibliographic Details
Published in:2016 IEEE International Conference on Big Data (Big Data) pp. 37 - 47
Main Authors: Dindokar, Ravikant, Choudhury, Neel, Simmhan, Yogesh
Format: Conference Proceeding
Language:English
Published: IEEE 01.12.2016
Subjects:
Algorithm design and analysis
Analytical models
Big data
Computational modeling
Data models
Partitioning algorithms
Programming
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Component-centric distributed graph processing models that use bulk synchronous parallel (BSP) execution have grown popular. These overcome short-comings of Big Data platforms like Hadoop for processing large graphs. However, literature on formal analysis of these component-centric abstractions for different graphs, graph partitioning, and graph algorithms is lacking. Here, we propose an coarse-grained analytical approach based on a meta-graph sketch to examine the characteristics of component-centric graph programming models. We apply this sketch to subgraph- and block-centric abstractions, and draw a comparison with vertex-centric models like Google's Pregel. We explore the impact of various graph partitioning techniques on the meta-graph, and the impact of the meta-graph on graph algorithms. This decouples large unwieldy graphs and their partitioning artifacts from their algorithmic analysis. We evaluate our approach for five spatial and powerlaw graphs, four different partitioning strategies, and for PageRank and Breadth First Search algorithms. We show that this novel analytical technique is simple, scalable and yet gives a reliable estimate of the number of supersteps, and the communication and computational complexities of the algorithms for various graphs.
DOI:10.1109/BigData.2016.7840587