DSParLib: A C++ Template Library for Distributed Stream Parallelism

Stream processing applications deal with millions of data items continuously generated over time. Often, they must be processed in real-time and scale performance, which requires the use of distributed parallel computing resources. In C/C++, the current state-of-the-art for distributed architectures...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:International journal of parallel programming Jg. 50; H. 5-6; S. 454 - 485
Hauptverfasser: Löff, Júnior, Hoffmann, Renato B., Pieper, Ricardo, Griebler, Dalvan, Fernandes, Luiz G.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.12.2022
Springer Nature B.V
Schlagworte:
Big Data
C plus plus
C++ (programming language)
Communication
Computer Science
Data transmission
Distributed processing
Experiments
Handwriting
Interfaces
Java
Message passing
Parallel processing
Parallel programming
Processor Architectures
Programmers
Programming languages
Software Engineering/Programming and Operating Systems
Special Issue on High-Level Parallel Programming and Applications 2021
Theory of Computation
Stream processing
Parallel programming
Distributed systems
Parallel patterns
MPI
ISSN:0885-7458, 1573-7640
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Stream processing applications deal with millions of data items continuously generated over time. Often, they must be processed in real-time and scale performance, which requires the use of distributed parallel computing resources. In C/C++, the current state-of-the-art for distributed architectures and High-Performance Computing is Message Passing Interface (MPI). However, exploiting stream parallelism using MPI is complex and error-prone because it exposes many low-level details to the programmer. In this work, we introduce a new parallel programming abstraction for implementing distributed stream parallelism named DSParLib. Our abstraction of MPI simplifies parallel programming by providing a pattern-based and building block-oriented development to inter-connect, model, and parallelize data streams found in modern applications. Experiments conducted with five different stream processing applications and the representative PARSEC Ferret benchmark revealed that DSParLib is efficient and flexible. Also, DSParLib achieved similar or better performance, required less coding, and provided simpler abstractions to express parallelism with respect to handwritten MPI programs.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-022-00737-2