Ensuring real-time data integrity in smart building applications: A systematic end-to-end comprehensive pipeline evaluated in numerous real-life cases

In this study, we propose a comprehensive, end-to-end data healing pipeline, developed and tested in real buildings facing diverse severity problems. This pipeline is designed to ensure the accuracy and reliability of smart building data in a fast and responsive manner, using computationally lightwe...

Full description

Saved in:
Bibliographic Details
Published in:Energy and buildings Vol. 336; p. 115586
Main Authors: Stefanopoulou, Aliki, Michailidis, Iakovos, Karatzinis, Georgios, Lepidas, Georgios, Boutalis, Yiannis S., Kosmatopoulos, Elias B.
Format: Journal Article
Language:English
Published: Elsevier B.V 01.06.2025
Subjects:
Data filtering
Data imputation
Real-time data processing
Smart buildings
Smart buildings
Real-time data processing
Data imputation
Data filtering
ISSN:0378-7788
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this study, we propose a comprehensive, end-to-end data healing pipeline, developed and tested in real buildings facing diverse severity problems. This pipeline is designed to ensure the accuracy and reliability of smart building data in a fast and responsive manner, using computationally lightweight statistical approaches for outlier detection and LightGBM for data imputation. Both methods are optimized for low computational cost, making them ideal for real-world scenarios requiring immediate feedback and capable of handling very large datasets efficiently. Our system is designed to operate automatically, capable of applying real-time data processing and periodic model updates without manual intervention. It was evaluated using Key Performance Indicators over nine weeks across five smart buildings in the EU, revealing discrepancies in performance across different time periods and buildings. These findings highlight the need for tailored data healing strategies for varying dataset sizes, ultimately enhancing data quality for more reliable analyses and informed decision-making. The implementation of this pipeline contributes to more accurate energy usage, improved occupant comfort, and more efficient building operations, supporting the broader goals of sustainability and energy efficiency in smart buildings.
ISSN:0378-7788
DOI:10.1016/j.enbuild.2025.115586