Batch Updates of Distributed Streaming Graphs using Linear Algebra

We develop a distributed-memory parallel algorithm for performing batch updates on streaming graphs, where vertices and edges are continuously added or removed. Our algorithm leverages distributed sparse matrices as the core data structures, utilizing equivalent sparse matrix operations to execute g...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis s. 645 - 649
Hlavní autoři:	Hassani, Elaheh, Hussain, Md Taufique, Azad, Ariful
Médium:	Konferenční příspěvek
Jazyk:	angličtina
Vydáno:	IEEE 17.11.2024
Témata:	batch graph updates Conferences Data structures distributed-memory algorithms High performance computing Linear algebra Parallel algorithms parallel computing parallel graph algorithms scalability in graph processing Sparse matrices streaming graphs Supercomputers
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	We develop a distributed-memory parallel algorithm for performing batch updates on streaming graphs, where vertices and edges are continuously added or removed. Our algorithm leverages distributed sparse matrices as the core data structures, utilizing equivalent sparse matrix operations to execute graph updates. By reducing unnecessary communication among processes and employing shared-memory parallelism, we accelerate updates of distributed graphs. Additionally, we maintain a balanced load in the output matrix by permuting the resultant matrix during the update process. We demonstrate that our streaming update algorithm is at least 25 times faster than alternative linear-algebraic methods and scales linearly up to 4,096 cores (32 nodes) on a Cray EX supercomputer.
DOI:	10.1109/SCW63240.2024.00089