Estimating Clustering Coefficient via Random Walk on MapReduce

Clustering coefficient plays an important role in many real-world applications, such as social network analysis and community mining. However, it is expensive to compute exact clustering coefficient for large networks. In many real-world applications, estimating clustering coefficient via random wal...

Full description

Saved in:
Bibliographic Details
Published in:2017 IEEE 23rd International Conference on Parallel and Distributed Systems (ICPADS) pp. 493 - 500
Main Authors: Liao, Qun, Yang, Yulu
Format: Conference Proceeding
Language:English
Published: IEEE 01.12.2017
Subjects:
Approximation algorithms
Clustering algorithms
clustering coefficient
Control engineering
distributed algorithm
Distributed algorithms
Distributed computing
graph analysis
MapReduce
Merging
random walk
Social network services
ISSN:1521-9097
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Clustering coefficient plays an important role in many real-world applications, such as social network analysis and community mining. However, it is expensive to compute exact clustering coefficient for large networks. In many real-world applications, estimating clustering coefficient via random walk method is preferable because it is efficient and accurate. MapReduce is a popular distributed programming framework for processing large datasets. In this paper, we propose an algorithm on MapReduce framework to estimate clustering coefficient based on random walk method. Experiments on a Hadoop cluster for large real-world graphs demonstrate that the proposed algorithm is accurate and efficient. Comparing to Doubling algorithm, a state-of-the-art distributed algorithm to implement random walk on MapReduce, the proposed algorithm runs much faster. The proposed algorithm also reduces I/O cost efficiently.
ISSN:1521-9097
DOI:10.1109/ICPADS.2017.00071