Distributed Louvain Algorithm for Graph Community Detection

In most real-world networks, the nodes/vertices tend to be organized into tightly-knit modules known as communities or clusters, such that nodes within a community are more likely to be "related" to one another than they are to the rest of the network. The goodness of partitioning into com...

Full description

Saved in:
Bibliographic Details
Published in:2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS) pp. 885 - 895
Main Authors: Ghosh, Sayan, Halappanavar, Mahantesh, Tumeo, Antonino, Kalyanaraman, Ananth, Lu, Hao, Chavarria-Miranda, Daniel, Khan, Arif, Gebremedhin, Assefaw
Format: Conference Proceeding
Language:English
Published: IEEE 01.05.2018
Subjects:
Community detection
Distributed graph clustering
Heuristic algorithms
Image edge detection
Iterative methods
Mathematical model
Optimization
Parallel algorithms
Parallel graph heuristics
Parallel Louvain
Synchronization
ISSN:1530-2075
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract In most real-world networks, the nodes/vertices tend to be organized into tightly-knit modules known as communities or clusters, such that nodes within a community are more likely to be "related" to one another than they are to the rest of the network. The goodness of partitioning into communities is typically measured using a well known measure called modularity. However, modularity optimization is an NP-complete problem. In 2008, Blondel, et al. introduced a multi-phase, iterative heuristic for modularity optimization, called the Louvain method. Owing to its speed and ability to yield high quality communities, the Louvain method continues to be one of the most widely used tools for serial community detection. In this paper, we present the design of a distributed memory implementation of the Louvain algorithm for parallel community detection. Our approach begins with an arbitrarily partitioned distributed graph input, and employs several heuristics to speedup the computation of the different steps of the Louvain algorithm. We evaluate our implementation and its different variants using real-world networks from various application domains (including internet, biology, social networks). Our MPI+OpenMP implementation yields about 7x speedup (on 4K processes) for soc-friendster network (1.8B edges) over a state-of-the-art shared memory multicore implementation (on 64 threads), without compromising output quality. Furthermore, our distributed implementation was able to process a larger graph (uk-2007; 3.3B edges) in 32 seconds on 1K cores (64 nodes) of NERSC Cori, when the state-of-the-art shared memory implementation failed to run due to insufficient memory on a single Cori node containing 128 GB of memory.
AbstractList In most real-world networks, the nodes/vertices tend to be organized into tightly-knit modules known as communities or clusters, such that nodes within a community are more likely to be "related" to one another than they are to the rest of the network. The goodness of partitioning into communities is typically measured using a well known measure called modularity. However, modularity optimization is an NP-complete problem. In 2008, Blondel, et al. introduced a multi-phase, iterative heuristic for modularity optimization, called the Louvain method. Owing to its speed and ability to yield high quality communities, the Louvain method continues to be one of the most widely used tools for serial community detection. In this paper, we present the design of a distributed memory implementation of the Louvain algorithm for parallel community detection. Our approach begins with an arbitrarily partitioned distributed graph input, and employs several heuristics to speedup the computation of the different steps of the Louvain algorithm. We evaluate our implementation and its different variants using real-world networks from various application domains (including internet, biology, social networks). Our MPI+OpenMP implementation yields about 7x speedup (on 4K processes) for soc-friendster network (1.8B edges) over a state-of-the-art shared memory multicore implementation (on 64 threads), without compromising output quality. Furthermore, our distributed implementation was able to process a larger graph (uk-2007; 3.3B edges) in 32 seconds on 1K cores (64 nodes) of NERSC Cori, when the state-of-the-art shared memory implementation failed to run due to insufficient memory on a single Cori node containing 128 GB of memory.
Author Tumeo, Antonino
Halappanavar, Mahantesh
Khan, Arif
Lu, Hao
Kalyanaraman, Ananth
Chavarria-Miranda, Daniel
Gebremedhin, Assefaw
Ghosh, Sayan
Author_xml – sequence: 1
  givenname: Sayan
  surname: Ghosh
  fullname: Ghosh, Sayan
– sequence: 2
  givenname: Mahantesh
  surname: Halappanavar
  fullname: Halappanavar, Mahantesh
– sequence: 3
  givenname: Antonino
  surname: Tumeo
  fullname: Tumeo, Antonino
– sequence: 4
  givenname: Ananth
  surname: Kalyanaraman
  fullname: Kalyanaraman, Ananth
– sequence: 5
  givenname: Hao
  surname: Lu
  fullname: Lu, Hao
– sequence: 6
  givenname: Daniel
  surname: Chavarria-Miranda
  fullname: Chavarria-Miranda, Daniel
– sequence: 7
  givenname: Arif
  surname: Khan
  fullname: Khan, Arif
– sequence: 8
  givenname: Assefaw
  surname: Gebremedhin
  fullname: Gebremedhin, Assefaw
BookMark eNotjEFLwzAYQKMouM3dBS_9A61f8qVpgqfRzjkoOFDPI00TF1nbkabC_r0DPT0ePN6c3PRDbwl5oJBRCuppu6t27xkDKjMAUPKKLFUhaY5ScBRSXJPZRSBlUOR3ZD6O3wAMkKsZea78GINvpmjbpB6mH-37ZHX8GoKPhy5xQ0g2QZ8OSTl03dT7eE4qG62Jfujvya3Tx9Eu_7kgny_rj_I1rd8223JVp54JGdNGCEM1UKMUzx06ZUVuNEomGXLRSIecFgXTl5g5pxqqsHGFgKY1ORpscUEe_77eWrs_Bd_pcN5LznLGGf4C8MVJJw
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/IPDPS.2018.00098
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9781538643686
1538643685
EISSN 1530-2075
EndPage 895
ExternalDocumentID 8425242
Genre orig-research
GroupedDBID 29O
6IE
6IF
6IH
6IK
6IL
6IN
AAJGR
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
OCL
RIE
RIL
ID FETCH-LOGICAL-i268t-b66c1a01c9945f3f9e65ca38282346b8f341772a2682ff9b193bf760bdc53c3d3
IEDL.DBID RIE
ISICitedReferencesCount 97
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000444710900088&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:47:55 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i268t-b66c1a01c9945f3f9e65ca38282346b8f341772a2682ff9b193bf760bdc53c3d3
PageCount 11
ParticipantIDs ieee_primary_8425242
PublicationCentury 2000
PublicationDate 2018-05
PublicationDateYYYYMMDD 2018-05-01
PublicationDate_xml – month: 05
  year: 2018
  text: 2018-05
PublicationDecade 2010
PublicationTitle 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
PublicationTitleAbbrev IPDPS
PublicationYear 2018
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0020349
ssj0002684650
Score 2.0933268
Snippet In most real-world networks, the nodes/vertices tend to be organized into tightly-knit modules known as communities or clusters, such that nodes within a...
SourceID ieee
SourceType Publisher
StartPage 885
SubjectTerms Community detection
Distributed graph clustering
Heuristic algorithms
Image edge detection
Iterative methods
Mathematical model
Optimization
Parallel algorithms
Parallel graph heuristics
Parallel Louvain
Synchronization
Title Distributed Louvain Algorithm for Graph Community Detection
URI https://ieeexplore.ieee.org/document/8425242
WOSCitedRecordID wos000444710900088&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8NAEB3a4sFT1Vb8Zg8ejU26ySaLJ7FWBSkBFXoru5tZLdhEalrw37uTphXBi7ew5JDMksybnTfvAZyHgQ4iEwWexkS7AoX-g4nKPJdcpE4MOkzgV2YT8WiUjMcybcDFZhYGESvyGV7SZdXLzwqzoKOyHrWMXEppQjOOxWpWa3OeQqolgjp0dbFFuivrtqQvew_pIH0iJhdRJ32Z_DJTqXLJsP2_p9iB7s9QHks36WYXGpjvQXvtysDqj7QDVwPSwiUbK8zYY7FYutqfXb-_FvNp-TZjDqSyO1KpZvVsSPnFBlhWjKy8Cy_D2-ebe6-2SPCm7oVLTwthAuUHRsowstxKFJFR3JVRfR4KnViXpBx-Vu7mvrVSO7imbSx8nZmIG57xfWjlRY4HwITRGIXKxA4ChAqFjizykCuTKeybQB9Ch2Ix-VipYEzqMBz9vXwM2xTsFTXwBFrlfIGnsGWW5fRzflZt3TfUXZlb
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8NAEB1qFfRUtRW_3YNHo0n2o1k8ibW2WEvACr2V3c1EC9pITQv-e3fTtCJ48RaWHJJZknmz8-Y9gHMW6IAbHngaI20LFPcfjFTi2eQidWTQYgK_MJto9vvRcCjjClysZmEQsSCf4aW7LHr5SWZm7qjsyrWMbEpZg3XOWOgvprVWJypOt0S4Hl1ZbjnllWVj0pdX3bgVPzkulyNP-jL6ZadSZJN27X_PsQ2Nn7E8Eq8Szg5UcLILtaUvAyk_0zpct5warjOywoT0stncVv_k5u0lm47z13diYSq5dzrVpJwOyb9IC_OCkzVpwHP7bnDb8UqTBG9sXzj3tBAmUH5gpGQ8palEwY2itpAKKRM6Sm2asgha2ZvDNJXaAjadNoWvE8OpoQndg-okm-A-EGE0cqZM04IAplBoniJlVJlEYWgCfQB1F4vRx0IHY1SG4fDv5TPY7Awee6Net_9wBFsu8Aui4DFU8-kMT2DDzPPx5_S02MZv_zqcog
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2018+IEEE+International+Parallel+and+Distributed+Processing+Symposium+%28IPDPS%29&rft.atitle=Distributed+Louvain+Algorithm+for+Graph+Community+Detection&rft.au=Ghosh%2C+Sayan&rft.au=Halappanavar%2C+Mahantesh&rft.au=Tumeo%2C+Antonino&rft.au=Kalyanaraman%2C+Ananth&rft.date=2018-05-01&rft.pub=IEEE&rft.eissn=1530-2075&rft.spage=885&rft.epage=895&rft_id=info:doi/10.1109%2FIPDPS.2018.00098&rft.externalDocID=8425242