AREP: an adaptive, machine learning-based algorithm for real-time anomaly detection on network telemetry data

Abnormal behaviour detection is an essential task of real-time monitoring to secure the reliable operation of ICT infrastructures. This paper presents AREP, an adaptive, long short-term memory-based machine learning algorithm for real-time anomaly detection on network telemetry data. AREP is an impr...

Full description

Saved in:
Bibliographic Details
Published in:Neural computing & applications Vol. 35; no. 8; pp. 6079 - 6094
Main Author: Farkas, Karoly
Format: Journal Article
Language:English
Published: London Springer London 01.03.2023
Springer Nature B.V
Subjects:
Adaptive algorithms
Algorithms
Anomalies
Artificial Intelligence
Computational Biology/Bioinformatics
Computational Science and Engineering
Computer Science
Data Mining and Knowledge Discovery
Identification methods
Image Processing and Computer Vision
Machine learning
Original Article
Performance evaluation
Probability and Statistics in Computer Science
Real time
Telemetry
Time series
Network telemetry
AREP
LSTM
Anomaly detection
Machine learning
Time series
ISSN:0941-0643, 1433-3058
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Abnormal behaviour detection is an essential task of real-time monitoring to secure the reliable operation of ICT infrastructures. This paper presents AREP, an adaptive, long short-term memory-based machine learning algorithm for real-time anomaly detection on network telemetry data. AREP is an improved version of Alter-Re 2 , the direct predecessor algorithm developed by our research team. AREP introduces automatic tuning of its two key parameters and includes an offset compensation component to increase accuracy. Unfortunately, AREP and its predecessors perform well only on time series showing specific patterns. Thus, we propose also a data type classification method to identify patterns on which AREP performs best. Moreover, we use an extended range of metrics in our performance evaluations, including area under the curve (AUC). AUC computation is based on receiver operating characteristic (ROC) curves. However, generating ROC curves is not straightforward due to the inherent adaptive threshold technique used by AREP and its predecessors, so we had to develop a novel ROC curve generation approach for these algorithms. We show through rigorous experiments that on network time series following specific data patterns AREP overperforms its predecessors and produces similar or even better performance than other state-of-the-art algorithms.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0941-0643
1433-3058
DOI:10.1007/s00521-022-08000-y