H2Pregel : A partition-based hybrid hierarchical graph computation approach

A partition-based hybrid hierarchical graph computation approach, called H2Pregel is proposed to address the redundant supersteps and inefficient computation problems due to low access locality. The H2Pregel preprocesses the input graph through a distributed recode algorithm to ensure the continuity...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Future generation computer systems Ročník 104; s. 15 - 31
Hlavní autori: Liu, Qiang, Dong, XiaoShe, Chen, Heng, Zhang, Xingjun
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier B.V 01.03.2020
Predmet:
BSP model
Graph computing
Hierarchical
Hybrid computation
Hierarchical
Hybrid computation
Graph computing
BSP model
ISSN:0167-739X, 1872-7115
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:A partition-based hybrid hierarchical graph computation approach, called H2Pregel is proposed to address the redundant supersteps and inefficient computation problems due to low access locality. The H2Pregel preprocesses the input graph through a distributed recode algorithm to ensure the continuity and sequence of vertex ids, then employs a hybrid approach to combine the advantages of both synchronous and asynchronous models, and hierarchically computes the high proportion of interior messages generated by high quality partition algorithms. Moreover, H2Pregel leverages configurable parallel threads to accelerate local computation by “sub-supersteps”, and employs an exterior messages stealing optimization to avoid extra communication overheads between tasks. We implemented H2Pregel on Giraph, a classic open source system based on Pregel. The evaluation results on large-scale graphs show that, compared with Pregel in three partition algorithms, H2Pregel can achieve average speedups by 1.12–4.52 times and decrease average communication messages by 23.5%-55.5%, and average supersteps by 15.8%-82.0%. •A distributed recode algorithm is proposed for graph preprocessing to ensure continuity and sequence of vertex ids.•A hybrid four-state model for vertex computation is proposed.•A hierarchical parallelization mechanism relies on interior messages of local computation.
ISSN:0167-739X
1872-7115
DOI:10.1016/j.future.2019.09.021