Unsupervised anomaly detection in hourly water demand data using an asymmetric encoder–decoder model

Water demand forecasting (WDF) is essential for the design and optimal management of water distribution systems (WDS). Historical water demand data contribute significantly to WDF. Yet the obtained water demand data contain anomalies on occasions due to failures in WDS or monitoring systems. The con...

Full description

Saved in:
Bibliographic Details
Published in:Journal of hydrology (Amsterdam) Vol. 613; p. 128389
Main Authors: Yan, Jieru, Tao, Tao
Format: Journal Article
Language:English
Published: Elsevier B.V 01.10.2022
Subjects:
Anomaly detection
Asymmetric encoder–decoder
Autoencoder
forests
Hourly water demand
hybrids
hydrology
Time series
water distribution
water pollution
Hourly water demand
Asymmetric encoder–decoder
Autoencoder
Anomaly detection
Time series
ISSN:0022-1694, 1879-2707
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Water demand forecasting (WDF) is essential for the design and optimal management of water distribution systems (WDS). Historical water demand data contribute significantly to WDF. Yet the obtained water demand data contain anomalies on occasions due to failures in WDS or monitoring systems. The contaminated water demand data are invalid to describe the actual water demand, thus degrading the performance of WDF models. However, the importance of anomaly detection in water demand data is underestimated, or at least not explicitly described in many published papers. To fill the gap, we propose an unsupervised anomaly detection method based on an asymmetric encoder–decoder (asyED) model. Different from the symmetric structure of a traditional autoencoder where a signal is reproduced from itself, asyED is asymmetric where a signal is reproduced from the upstream and downstream information of the signal, while the signal itself does not participate in the reconstruction. In light of this feature, asyED is powerful in identifying anomalies. The proposed method is employed to detect anomalies in hourly water demand data which exhibit trend and seasonality. The results show the superiority of the proposed method over the other four commonly used anomaly detection methods: Z-score, isolation forest, local outlier factor, and seasonal hybrid ESD (Extreme Studentized Deviate test). •An anomaly detection method based on an asymmetric encoder-decoder model is proposed.•The method applies to hourly water demand data which exhibit trend and seasonality.•A cluster-based metric is proposed for model evaluation.•The metric considers the continuity mechanism of anomalies found in the data.•Measures to quantify the strengths of different components in the data are proposed.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0022-1694
1879-2707
DOI:10.1016/j.jhydrol.2022.128389