Network Information Processing Analysis Based on Big Data Parallel Graph Partitioning Algorithm

This research proposes an efficient parallel graph partitioning algorithm for the big data environment, aiming to solve the bottlenecks of traditional clustering techniques in terms of processing speed and scalability. The algorithm adopts a multi-level graph partitioning framework, decomposing the...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of computing and information technology Jg. 33; H. 3; S. 139 - 155
Hauptverfasser: Guan, Keqing, Kong, Xianli
Format: Journal Article Paper
Sprache:Englisch
Veröffentlicht: Sveuciliste U Zagrebu 01.09.2025
Sveučilište u Zagrebu Fakultet elektrotehnike i računarstva
University of Zagreb Faculty of Electrical Engineering and Computing
Schlagworte:
Algorithms
Big data
Database design
distributed, network split
Electronic data processing
Energy efficiency
Methods
network information processing
parallel graph partitioning algorithm
Social networks
China
ISSN:1330-1136, 1846-3908
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This research proposes an efficient parallel graph partitioning algorithm for the big data environment, aiming to solve the bottlenecks of traditional clustering techniques in terms of processing speed and scalability. The algorithm adopts a multi-level graph partitioning framework, decomposing the network information processing task into multiple levels, gradually simplifying the graph structure and backtracking refinement, thereby significantly reducing the computational complexity while ensuring the partitioning quality. The algorithm focuses on balancing the node cohesion within partitions and the edge cutting cost of inter-partition communication. By constructing a global objective function, it minimizes the number of edges across partitions and the workload differences among various sub-graphs, thereby achieving a more balanced partitioning result. The research results show that this algorithm achieves a resource utilization rate of 0.95. In the Hadoop cluster environment, 95% of the computing resources are effectively used for actual task processing, which is significantly higher than that of the competing algorithms. The energy efficiency ratio reaches 0.98, indicating that the number of tasks completed per unit of energy consumption is close to the optimal level, which is superior to the 0.78 to 0.67 range of existing methods, reflecting the advantages of this algorithm in green computing. The load imbalance rate is only 0.00395, and the point weight imbalance rate is 0.00141, which are much lower values than those of the comparison algorithm. This indicates that the algorithm achieves a high degree of balance in task allocation and node weight distribution, effectively avoiding resource waste and performance bottlenecks.
Bibliographie:336007
ISSN:1330-1136
1846-3908
DOI:10.20532/cit.2025.1005902