Fast iterative graph computation a path centric approach

Large scale graph processing represents an interesting challenge due to the lack of locality. This paper presents PathGraph for improving iterative graph computation on graphs with billions of edges. Our system design has three unique features: First, we model a large graph using a collection of tre...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis pp. 401 - 412
Main Authors: Yuan, Pingpeng, Zhang, Wenya, Xie, Changfeng, Jin, Hai, Liu, Ling, Lee, Kisung
Format: Conference Proceeding
Language:English
Published: Piscataway, NJ, USA IEEE Press 16.11.2014
IEEE
Series:ACM Conferences
Subjects:
Analytical models
Computational efficiency
Computational modeling
Computer systems organization > Dependable and fault-tolerant systems and networks
Computing model
Convergence
General and reference > Cross-computing tools and techniques > Performance
Graph
Iterative algorithms
Iterative computation
Mathematics of computing > Discrete mathematics > Graph theory > Graph algorithms
Mathematics of computing > Discrete mathematics > Graph theory > Paths and connectivity problems
Media
Networks > Network performance evaluation
Partitioning algorithms
Path
Programming
Storage
Storage management
System analysis and design
path
computing model
iterative computation
storage
graph
ISBN:1479955000, 9781479955008
ISSN:2167-4329
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Large scale graph processing represents an interesting challenge due to the lack of locality. This paper presents PathGraph for improving iterative graph computation on graphs with billions of edges. Our system design has three unique features: First, we model a large graph using a collection of tree-based partitions and use an path-centric computation rather than vertex-centric or edge-centric computation. Our parallel computation model significantly improves the memory and disk locality for performing iterative computation algorithms. Second, we design a compact storage that further maximize sequential access and minimize random access on storage media. Third, we implement the path-centric computation model by using a scatter/gather programming model, which parallels the iterative computation at partition tree level and performs sequential updates for vertices in each partition tree. The experimental results show that the path-centric approach outperforms vertex-centric and edge-centric systems on a number of graph algorithms for both in-memory and out-of-core graphs.
ISBN:1479955000
9781479955008
ISSN:2167-4329
DOI:10.1109/SC.2014.38