Extracting deltas from column oriented NoSQL databases for different incremental applications and diverse data targets

This paper describes the Change Data Capture (CDC) problems in the context of column-oriented NoSQL databases (CoNoSQLDBs). CDC is a term mostly used by ETL tools and data warehousing environments (DW) to depict a data processing of extracting data changes made at the data sources. Based on analyzin...

Full description

Saved in:
Bibliographic Details
Published in:Data & knowledge engineering Vol. 93; pp. 42 - 59
Main Authors: Hu, Yong, Dessloch, Stefan
Format: Journal Article
Language:English
Published: Elsevier B.V 01.09.2014
Subjects:
CDC approaches
CoNoSQLDB
Delta model
Delta representations
Delta model
Delta representations
CoNoSQLDB
CDC approaches
ISSN:0169-023X, 1872-6933
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper describes the Change Data Capture (CDC) problems in the context of column-oriented NoSQL databases (CoNoSQLDBs). CDC is a term mostly used by ETL tools and data warehousing environments (DW) to depict a data processing of extracting data changes made at the data sources. Based on analyzing the impacts and constraints caused by the core features of CoNoSQLDBs, we propose a logical change data (delta) model and the corresponding delta representations which could work with different incremental applications and diverse data targets. Moreover, we present five feasible CDC approaches, i.e. Timestamp-based approach, Audit-column approach, Log-based approach, Trigger-based approach and Snapshot differential approach and indicate the performance winners under different circumstances.
ISSN:0169-023X
1872-6933
DOI:10.1016/j.datak.2014.07.002