iDMS: An Index-Based Framework for Tracking Distributed Multidimensional Data Streams

Histograms are a compact and effective way to summarize large datasets, representing data distribution by partitioning it into blocks or buckets for visualization and analysis. They find widespread use in diverse fields, from image processing to statistical analysis and database optimization. This p...

Full description

Saved in:

Bibliographic Details
Published in:	2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE) pp. 1381 - 1388
Main Authors:	Razavi, Zhinoos, Sellis, Timos, Liao, Kewen, Razavi, Shahab
Format:	Conference Proceeding
Language:	English
Published:	IEEE 24.07.2023
Subjects:	Computational modeling Data models Data Summarization Distributed computing Distributed databases Generative adversarial networks Generative models Histograms Index data structures Merging Statistical analysis
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	Histograms are a compact and effective way to summarize large datasets, representing data distribution by partitioning it into blocks or buckets for visualization and analysis. They find widespread use in diverse fields, from image processing to statistical analysis and database optimization. This paper focuses on merging and maintaining a centralized histogram for distributed summaries in multidimensional spaces. The proposed framework employs a dynamic index data structure to efficiently approximate data distribution from continuous distributed data streams. It offers a simple and implementable solution, gener-ating an up-to-date histogram for aggregated multidimensional data summaries while supporting both centralized and local histograms. Experimental results demonstrate its performance in parallel and distributed settings, considering communication cost, error rate, and practicality.
DOI:	10.1109/CSCE60160.2023.00231