H2Pregel : A partition-based hybrid hierarchical graph computation approach

A partition-based hybrid hierarchical graph computation approach, called H2Pregel is proposed to address the redundant supersteps and inefficient computation problems due to low access locality. The H2Pregel preprocesses the input graph through a distributed recode algorithm to ensure the continuity...

Full description

Saved in:
Bibliographic Details
Published in:Future generation computer systems Vol. 104; pp. 15 - 31
Main Authors: Liu, Qiang, Dong, XiaoShe, Chen, Heng, Zhang, Xingjun
Format: Journal Article
Language:English
Published: Elsevier B.V 01.03.2020
Subjects:
BSP model
Graph computing
Hierarchical
Hybrid computation
Hierarchical
Hybrid computation
Graph computing
BSP model
ISSN:0167-739X, 1872-7115
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A partition-based hybrid hierarchical graph computation approach, called H2Pregel is proposed to address the redundant supersteps and inefficient computation problems due to low access locality. The H2Pregel preprocesses the input graph through a distributed recode algorithm to ensure the continuity and sequence of vertex ids, then employs a hybrid approach to combine the advantages of both synchronous and asynchronous models, and hierarchically computes the high proportion of interior messages generated by high quality partition algorithms. Moreover, H2Pregel leverages configurable parallel threads to accelerate local computation by “sub-supersteps”, and employs an exterior messages stealing optimization to avoid extra communication overheads between tasks. We implemented H2Pregel on Giraph, a classic open source system based on Pregel. The evaluation results on large-scale graphs show that, compared with Pregel in three partition algorithms, H2Pregel can achieve average speedups by 1.12–4.52 times and decrease average communication messages by 23.5%-55.5%, and average supersteps by 15.8%-82.0%. •A distributed recode algorithm is proposed for graph preprocessing to ensure continuity and sequence of vertex ids.•A hybrid four-state model for vertex computation is proposed.•A hierarchical parallelization mechanism relies on interior messages of local computation.
ISSN:0167-739X
1872-7115
DOI:10.1016/j.future.2019.09.021