Local programming language barriers in stream-based systems

Stream-based data processing systems, such as Complex Event Processing or data stream mining systems, may be composed of several components which may be implemented in various programming languages. In distributed scenarios, computer networks typically represent important bottlenecks. However, the p...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2016 IEEE Symposium on Computers and Communication (ISCC) S. 569 - 574
Hauptverfasser: Gad, Ruediger, Kappes, Martin, Medina-Bulo, Inmaculada
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.06.2016
Schlagworte:
Computer networks
Context
Data mining
Data Stream Processing
Java
Programming Languages
Sockets
Throughput
Throughput Performance
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Stream-based data processing systems, such as Complex Event Processing or data stream mining systems, may be composed of several components which may be implemented in various programming languages. In distributed scenarios, computer networks typically represent important bottlenecks. However, the performance of data exchange in local contexts may be as important as the performance of data exchange via computer networks. Local programming language barriers may represent important bottlenecks for components that are located on the same computer system. In distributed scenarios, it may be beneficial to relocate components on a single physical host for exploiting the higher local data throughput. The properties of stream-based systems pose challenges like high throughput requirements but also open up optimization potential such as leveraging batched transfers. We performed an experimental analysis of ways for bridging local programming language barriers using the examples of C, Java, and Python and analyzed the impact of batched forwarding. While local data exchange can be expected to offer a higher throughput than exchange across networks, our results show that batch forwarding can increase the local throughput by factors of up to 47.6 and we measured net throughputs up to 39.5 Gbps.
DOI:10.1109/ISCC.2016.7543798