Estimating the Frequency of Data Items in Massive Distributed Streams

We investigate the problem of estimating on the fly the frequency at which items recur in large scale distributed data streams, which has become the norm in cloud-based application. This paper presents CASE, a combination of tools and probabilistic algorithms from the data streaming model. In this m...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2015 IEEE Fourth Symposium on Network Cloud Computing and Applications (NCCA) S. 59 - 66
Hauptverfasser: Anceaume, Emmanuelle, Busnel, Yann, Rivetti, Nicolo
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.06.2015
Schlagworte:
Approximation algorithms
Approximation methods
Computational modeling
Computer aided software engineering
Data models
Data stream model
Estimation
Frequency estimation
randomized approximation algorithm
ISBN:9781467377416, 1467377414
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We investigate the problem of estimating on the fly the frequency at which items recur in large scale distributed data streams, which has become the norm in cloud-based application. This paper presents CASE, a combination of tools and probabilistic algorithms from the data streaming model. In this model, functions are estimated on a huge sequence of data items, in an online fashion, and with a very small amount of memory with respect to both the size of the input stream and the values domain from which data items are drawn. We derive upper and lower bounds on the quality of CASE, improving upon the Count-Min sketch algorithm which has, so far, been the best algorithm in terms of space and time performance to estimate the frequency of data items. We prove that CASE guarantees an (e, d)-approximation of the frequency of all the items, provided they are not rare, in a space- efficient way and for any input stream. Experiments on both synthetic and real datasets confirm our analysis.
ISBN:9781467377416
1467377414
DOI:10.1109/NCCA.2015.19