SOOP: Efficient Distributed Graph Computation Supporting Second-Order Random Walks

The second-order random walk has recently been shown to effectively improve the accuracy in graph analysis tasks. Existing work mainly focuses on centralized second-order random walk (SOW) algorithms. SOW algorithms rely on edge-to-edge transition probabilities to generate next random steps. However...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of computer science and technology Ročník 36; číslo 5; s. 985 - 1001
Hlavní autoři: Niu, Songjie, Zhou, Dongyan
Médium: Journal Article
Jazyk:angličtina
Vydáno: Singapore Springer Singapore 01.10.2021
Springer
Springer Nature B.V
University of Chinese Academy of Sciences,Beijing 100049,China%Bytedance Technology,Beijing 100086,China
State Key Laboratory of Computer Architecture,Institute of Computing Technology,Chinese Academy of Sciences Beijing 100190,China
Témata:
Accuracy
Algorithms
Analysis
Artificial Intelligence
Communication
Computer Science
Data Structures and Information Theory
Graphs
Information Systems Applications (incl.Internet)
Mathematical optimization
Optimization techniques
Random walk
Regular Paper
Software Engineering
Theory of Computation
Transition probabilities
second-order PageRank
SOOP (second-order random walks with on-demand probability computation)
distributed graph computation
second-order random walk (SOW)
Node2Vec
SOOP(second-order random walks with on-demand probability computation)
ISSN:1000-9000, 1860-4749
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The second-order random walk has recently been shown to effectively improve the accuracy in graph analysis tasks. Existing work mainly focuses on centralized second-order random walk (SOW) algorithms. SOW algorithms rely on edge-to-edge transition probabilities to generate next random steps. However, it is prohibitively costly to store all the probabilities for large-scale graphs, and restricting the number of probabilities to consider can negatively impact the accuracy of graph analysis tasks. In this paper, we propose and study an alternative approach, SOOP (second-order random walks with on-demand probability computation), that avoids the space overhead by computing the edge-to-edge transition probabilities on demand during the random walk. However, the same probabilities may be computed multiple times when the same edge appears multiple times in SOW, incurring extra cost for redundant computation and communication. We propose two optimization techniques that reduce the complexity of computing edge-to-edge transition probabilities to generate next random steps, and reduce the cost of communicating out-neighbors for the probability computation, respectively. Our experiments on real-world and synthetic graphs show that SOOP achieves orders of magnitude better performance than baseline precompute solutions, and it can efficiently computes SOW algorithms on billion-scale graphs.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1000-9000
1860-4749
DOI:10.1007/s11390-021-1234-y