Batch Updates of Distributed Streaming Graphs using Linear Algebra

We develop a distributed-memory parallel algorithm for performing batch updates on streaming graphs, where vertices and edges are continuously added or removed. Our algorithm leverages distributed sparse matrices as the core data structures, utilizing equivalent sparse matrix operations to execute g...

Full description

Saved in:

Bibliographic Details
Published in:	SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis pp. 645 - 649
Main Authors:	Hassani, Elaheh, Hussain, Md Taufique, Azad, Ariful
Format:	Conference Proceeding
Language:	English
Published:	IEEE 17.11.2024
Subjects:	batch graph updates Conferences Data structures distributed-memory algorithms High performance computing Linear algebra Parallel algorithms parallel computing parallel graph algorithms scalability in graph processing Sparse matrices streaming graphs Supercomputers
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	We develop a distributed-memory parallel algorithm for performing batch updates on streaming graphs, where vertices and edges are continuously added or removed. Our algorithm leverages distributed sparse matrices as the core data structures, utilizing equivalent sparse matrix operations to execute graph updates. By reducing unnecessary communication among processes and employing shared-memory parallelism, we accelerate updates of distributed graphs. Additionally, we maintain a balanced load in the output matrix by permuting the resultant matrix during the update process. We demonstrate that our streaming update algorithm is at least 25 times faster than alternative linear-algebraic methods and scales linearly up to 4,096 cores (32 nodes) on a Cray EX supercomputer.
DOI:	10.1109/SCW63240.2024.00089