Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark

Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to proc...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA) s. 38 - 44
Hlavní autori: Lavin, Alexander, Ahmad, Subutai
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 01.12.2015
Predmet:
Algorithm design and analysis
anomaly detection
Benchmark testing
benchmarks
Detection algorithms
Detectors
Measurement
Real-time systems
streaming data
time-series data
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data.
DOI:10.1109/ICMLA.2015.141