AcLog: An Approach to Detecting Anomalies from System Logs with Active Learning

Log-based anomaly detection is an essential aspect of maintaining software reliability, particularly in the context of microservice systems. However, existing log-based anomaly detection approaches rely on historical anomalous labeled data or require huge labeling efforts. This makes existing log-ba...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings (IEEE International Conference on Web Services. Online) s. 436 - 443
Hlavní autoři: Duan, Chiming, Jia, Tong, Li, Ying, Huang, Gang
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.07.2023
Témata:
Active learning
Anomaly detection
Data models
Deep Learning
Human Feedback
Labeling
Log analysis
Microservice architectures
Software reliability
Training
Training data
Web services
ISSN:2836-3868
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Log-based anomaly detection is an essential aspect of maintaining software reliability, particularly in the context of microservice systems. However, existing log-based anomaly detection approaches rely on historical anomalous labeled data or require huge labeling efforts. This makes existing log-based anomaly detection approaches inefficient. In this paper, we propose AcLog, a novel anomaly detection approach that incorporates human knowledge to enhance model ability based on the framework of deep active learning. It incorporates an unsupervised model to learn from normal log data rather than historical anomalous labeled data and leverages active learning to incorporate human knowledge as a golden signal to augment the quality of training log data. Experiment results on three open log datasets and one log dataset collected from a real-world microservice system show that our approach improves over 7% F1-score with 5% labeled training data on average.
ISSN:2836-3868
DOI:10.1109/ICWS60048.2023.00062