Incremental Maintenance of Online Summaries Over Multiple Streams

We propose a novel approach based on predictive quantization (PQ) for online summarization of multiple time-varying data streams. A synopsis over a sliding window of most recent entries is computed in one pass and dynamically updated in constant time. The correlation between consecutive data element...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on knowledge and data engineering Vol. 20; no. 2; pp. 216 - 229
Main Authors: Altiparmak, F., Tuncel, E., Ferhatosmanoglu, H.
Format: Journal Article
Language:English
Published: New York, NY IEEE 01.02.2008
IEEE Computer Society
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Applied sciences
Computer science; control theory; systems
Computer systems and distributed systems. User interface
Data analysis
Data mining
Data processing. List processing. Character string processing
Data security
Exact sciences and technology
Information systems. Data bases
Intrusion detection
Memory organisation. Data processing
Monitoring
multiple streams
On-line systems
Online
online update
Prediction
Quantization
Queries
Query processing
Real time
Sliding
Software
Stock markets
Streams
summarization
Telecommunication network management
Windows (intervals)
multiple streams
summarization
online update
Prediction
quantization
Streaming
Correlation
Database query
data summarization
Abstract
Updating
Maintenance
synopsis
Real time
time-varying data
predictive quantization
Time varying system
Signal quantization
Sliding window
Linguistics
Multiple streams
ISSN:1041-4347, 1558-2191
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a novel approach based on predictive quantization (PQ) for online summarization of multiple time-varying data streams. A synopsis over a sliding window of most recent entries is computed in one pass and dynamically updated in constant time. The correlation between consecutive data elements is effectively taken into account without the need for preprocessing. We extend PQ to multiple streams and propose structures for real-time summarization and querying of a massive number of streams. Queries on any subsequence of a sliding window over multiple streams are processed in real time. We examine each component of the proposed approach, prediction, and quantization separately and investigate the space-accuracy trade-off for synopsis generation. Complementing the theoretical optimality of PQ-based approaches, we show that the proposed technique, even for very short prediction windows, significantly outperforms the current techniques for a wide variety of query types on both synthetic and real data sets.
Bibliography:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
content type line 23
ISSN:1041-4347
1558-2191
DOI:10.1109/TKDE.2007.190693