Self-Taught Anomaly Detection With Hybrid Unsupervised/Supervised Machine Learning in Optical Networks

This paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM ena...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of lightwave technology Ročník 37; číslo 7; s. 1742 - 1749
Hlavní autoři: Chen, Xiaoliang, Li, Baojia, Proietti, Roberto, Zhu, Zuqing, Yoo, S. J. Ben
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.04.2019
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithms
Anomalies
Anomaly detection
Artificial intelligence
Artificial neural networks
Clustering
Clustering algorithms
Cognition
data clustering module
Data clustering module (DCM)
data regression and classification module
data regression and classification module (DRCM)
DCM
DRCM
ENGINEERING
hybrid unsupervised and supervised machine learning
Knowledge engineering
Knowledge management
Machine learning
Modules
Monitoring
Optical communication
Optical fiber networks
Regression analysis
self-taught anomaly detection
ISSN:0733-8724, 1558-2213
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM enables a self-learning capability that eliminates the requirement of prior knowledge of abnormal network behaviors and therefore can potentially detect unforeseen anomalies. Second, we introduce a self-taught mechanism that transfers the patterns learned by the DCM to a supervised data regression and classification module (DRCM). The DRCM, whose complexity is mainly related to the scale of the applied supervised learning model, can potentially facilitate more scalable and time-efficient online anomaly detection by avoiding excessively traversing the original dataset. We designed the DCM and DRCM based on the density-based clustering algorithm and the deep neural network structure, respectively. Evaluations with experimental data from two use cases (i.e., single-point detection and end-to-end detection) demonstrate that up to <inline-formula><tex-math notation="LaTeX">99\%</tex-math></inline-formula> anomaly detection accuracy can be achieved with a false positive rate below <inline-formula><tex-math notation="LaTeX">1\%</tex-math></inline-formula> .
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
USDOE Office of Science (SC)
National Science Foundation (NSF)
SC0016700; ICE-T:RC 1836921
ISSN:0733-8724
1558-2213
DOI:10.1109/JLT.2019.2902487