TARIS: Scalable Incremental Processing of Time-Respecting Algorithms on Streaming Graphs

Temporal graphs change with time and have a lifespan associated with each vertex and edge. These graphs are suitable to process time-respecting algorithms where the traversed edges must have monotonic timestamps. Interval-centric Computing Model (ICM) is a distributed programming abstraction to desi...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on parallel and distributed systems Vol. 35; no. 12; pp. 2527 - 2544
Main Authors: Bhoot, Ruchi, Ghanmode, Suved Sanjay, Simmhan, Yogesh
Format: Journal Article
Language:English
Published: IEEE 01.12.2024
Subjects:
Clustering algorithms
Computational modeling
COVID-19
Data structures
Discrete Mathematics
distributed programming
distributed systems
graph algorithms
graph theory
Heuristic algorithms
information storage and retrieval
information technology and systems
mathematics of computing
Network topology
numerical analysis
parallel algorithms
programming techniques
Roads
Social networking (online)
software/software engineering
Space exploration
systems and software
Topology
ISSN:1045-9219, 1558-2183
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Temporal graphs change with time and have a lifespan associated with each vertex and edge. These graphs are suitable to process time-respecting algorithms where the traversed edges must have monotonic timestamps. Interval-centric Computing Model (ICM) is a distributed programming abstraction to design such temporal algorithms. There has been little work on supporting time-respecting algorithms at large scales for streaming graphs, which are updated continuously at high rates (Millions/s), such as in financial and social networks. In this article, we extend the windowed-variant of ICM for incremental computing over streaming graph updates. We formalize the properties of temporal graph algorithms and prove that our model of incremental computing over streaming updates is equivalent to batch execution of ICM. We design TARIS, a novel distributed graph platform that implements these incremental computing features. We use efficient data structures to reduce memory access and enhance locality during graph updates. We also propose scheduling strategies to interleave updates with computing, and streaming strategies to adapt the execution window for incremental computing to the variable input rates. Our detailed and rigorous evaluation of temporal algorithms on large-scale graphs with up to <inline-formula><tex-math notation="LaTeX">2\,\text{B}</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mspace width="0.166667em"/><mml:mtext>B</mml:mtext></mml:mrow></mml:math><inline-graphic xlink:href="bhoot-ieq1-3471574.gif"/> </inline-formula> edges show that TARIS out-performs contemporary baselines, Tink and Gradoop, by 3-4 orders of magnitude, and handles a high input rate of <inline-formula><tex-math notation="LaTeX"> 83k</tex-math> <mml:math><mml:mrow><mml:mn>83</mml:mn><mml:mi>k</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="bhoot-ieq2-3471574.gif"/> </inline-formula>-<inline-formula><tex-math notation="LaTeX"> 587\,\text{M}</tex-math> <mml:math><mml:mrow><mml:mn>587</mml:mn><mml:mspace width="0.166667em"/><mml:mtext>M</mml:mtext></mml:mrow></mml:math><inline-graphic xlink:href="bhoot-ieq3-3471574.gif"/> </inline-formula> Mutations/s with latencies in the order of seconds-minutes.
ISSN:1045-9219
1558-2183
DOI:10.1109/TPDS.2024.3471574