Data integration from traditional to big data: main features and comparisons of ETL approaches

Data integration combines information from different sources to provide a comprehensive view for making informed business decisions. The ETL (Extract, Transform, and Load) process is essential in data integration. In the past two decades, modeling the ETL process has become a priority for effectivel...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:The Journal of supercomputing Ročník 80; číslo 19; s. 26687 - 26725
Hlavní autori: Walha, Afef, Ghozzi, Faiza, Gargouri, Faiez
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York Springer US 01.12.2024
Springer Nature B.V
Predmet:
Artificial intelligence
Big Data
Business intelligence
Classification
Compilers
Computer Science
Data collection
Data integration
Data processing
Data warehouses
Decision making
Decision support systems
Distributed processing
Interpreters
Keywords
Literature reviews
Modelling
Processor Architectures
Programming Languages
Research methodology
Social networks
Storage systems
Trends
Decision Support System
Big Data
Extract–Transform–Load
Data integration
ETL modeling
ISSN:0920-8542, 1573-0484
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Data integration combines information from different sources to provide a comprehensive view for making informed business decisions. The ETL (Extract, Transform, and Load) process is essential in data integration. In the past two decades, modeling the ETL process has become a priority for effectively managing information. This paper aims to explore ETL approaches to help researchers and organizational stakeholders overcome challenges, especially in Big Data integration. It offers a comprehensive overview of ETL methods, from traditional to Big Data, and discusses their advantages, limitations, and the primary trends in Big Data integration. The study emphasizes that many technologies have been integrated into ETL steps for data collection, storage, processing, querying, and analysis without proper modeling. Therefore, more generic and customized design modeling of the ETL steps should be carried out to ensure reusability and flexibility. The paper summarizes the exploration of ETL modeling, focusing on Big Data scalability and processing trends. It also identifies critical dilemmas, such as ensuring compatibility across multiple sources and dealing with large volumes of Big Data. Furthermore, it suggests future directions in Big Data integration by leveraging advanced artificial intelligence processing and storage systems to ensure consistency, efficiency, and data integrity.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0920-8542
1573-0484
DOI:10.1007/s11227-024-06413-1