Terramycelium: a reference architecture for adaptive big data systems.
Uloženo v:
| Název: | Terramycelium: a reference architecture for adaptive big data systems. |
|---|---|
| Autoři: | Ataei, Pouya, Atemkeng, Marcellin |
| Zdroj: | Journal of Big Data; 11/24/2025, Vol. 12 Issue 1, p1-74, 74p |
| Témata: | BIG data, SCALABILITY, ADAPTIVE control systems, MODEL-driven software architecture, DATA structures, REAL-time computing, ELECTRONIC data processing, COMPUTER architecture |
| Abstrakt: | Modern organizations generate and process unprecedented volumes of structured, semi-structured, and unstructured data from diverse sources, creating significant architectural and engineering challenges for traditional data processing systems. Industry analyses consistently report failure rates of 60-85% for Big Data projects, with architectural limitations identified as a primary contributing factor. Current reference architectures suffer from monolithic designs, inadequate cross-cutting concerns (security, privacy, metadata), and limited adaptability to evolving data ecosystems. This paper presents Terramycelium, a novel reference architecture for Big Data systems that addresses these limitations through a domain-driven, event-oriented approach. The architecture integrates principles from complex adaptive systems, domain-driven design, distributed systems, and event-driven architectures to enable autonomous domain-specific data ownership while maintaining system-wide coherence through asynchronous event communication. We developed Terramycelium following empirically grounded reference architecture guidelines and evaluated it through two complementary methods: a case-mechanism experiment and expert opinion assessment. The case-mechanism experiments demonstrated the architecture's capability to process 1.693GB of data with 50-100 second latency, handle 771,305 streaming messages with 0.0000148 second ingestion latency, and maintain stable performance with 24% CPU utilization under high-volume scenarios. Expert evaluation (n=3, 10-32 years experience) validated the architecture's innovative integration of domain-driven design with data engineering, while identifying implementation complexity and organizational readiness as adoption challenges. Terramycelium contributes a validated approach for building scalable, maintainable Big Data systems that addresses the limitations of existing monolithic architectures while aligning with modern software engineering practices. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Big Data is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Databáze: | Complementary Index |
Full Text Finder 
Nájsť tento článok vo Web of Science Buďte první, kdo okomentuje tento záznam!