Self-Taught Anomaly Detection With Hybrid Unsupervised/Supervised Machine Learning in Optical Networks

This paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM ena...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of lightwave technology Jg. 37; H. 7; S. 1742 - 1749
Hauptverfasser: Chen, Xiaoliang, Li, Baojia, Proietti, Roberto, Zhu, Zuqing, Yoo, S. J. Ben
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York IEEE 01.04.2019
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Algorithms
Anomalies
Anomaly detection
Artificial intelligence
Artificial neural networks
Clustering
Clustering algorithms
Cognition
data clustering module
Data clustering module (DCM)
data regression and classification module
data regression and classification module (DRCM)
DCM
DRCM
ENGINEERING
hybrid unsupervised and supervised machine learning
Knowledge engineering
Knowledge management
Machine learning
Modules
Monitoring
Optical communication
Optical fiber networks
Regression analysis
self-taught anomaly detection
ISSN:0733-8724, 1558-2213
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM enables a self-learning capability that eliminates the requirement of prior knowledge of abnormal network behaviors and therefore can potentially detect unforeseen anomalies. Second, we introduce a self-taught mechanism that transfers the patterns learned by the DCM to a supervised data regression and classification module (DRCM). The DRCM, whose complexity is mainly related to the scale of the applied supervised learning model, can potentially facilitate more scalable and time-efficient online anomaly detection by avoiding excessively traversing the original dataset. We designed the DCM and DRCM based on the density-based clustering algorithm and the deep neural network structure, respectively. Evaluations with experimental data from two use cases (i.e., single-point detection and end-to-end detection) demonstrate that up to <inline-formula><tex-math notation="LaTeX">99\%</tex-math></inline-formula> anomaly detection accuracy can be achieved with a false positive rate below <inline-formula><tex-math notation="LaTeX">1\%</tex-math></inline-formula> .
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
USDOE Office of Science (SC)
National Science Foundation (NSF)
SC0016700; ICE-T:RC 1836921
ISSN:0733-8724
1558-2213
DOI:10.1109/JLT.2019.2902487