Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark
Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to proc...
Saved in:
| Published in: | 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA) pp. 38 - 44 |
|---|---|
| Main Authors: | , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
IEEE
01.12.2015
|
| Subjects: | |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data. |
|---|---|
| AbstractList | Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data. |
| Author | Lavin, Alexander Ahmad, Subutai |
| Author_xml | – sequence: 1 givenname: Alexander surname: Lavin fullname: Lavin, Alexander email: alavin@numenta.com organization: Numenta, Inc., Redwood City, CA, USA – sequence: 2 givenname: Subutai surname: Ahmad fullname: Ahmad, Subutai email: sahmad@numenta.com organization: Numenta, Inc., Redwood City, CA, USA |
| BookMark | eNo9jU9LwzAchiPowU2vXrzkC6Tmb5Mea51zUBVGPY80-XUNtql0mbBv70Dx9MLDw_Mu0GWcIiB0x2jGGC0eNtVrXWacMpUxyS7QgilaUMqNZtdou_q2w9GmEPd4C3YgTRgBl3Ea7XDCT5DApTBFXA77aQ6pHw-YENz0gN-OI8Rk_91HiK4f7fx5g646Oxzg9m-X6ON51VQvpH5fb6qyJoFrmQho3fkcjKFeOeU0ayXk1jBlzrgzxntmCwEeFEgPHW3B5Vy20nohvZOdWKL7324AgN3XHM7np52WXHIjxA9tjUyU |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ICMLA.2015.141 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 1509002871 9781509002870 |
| EndPage | 44 |
| ExternalDocumentID | 7424283 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL CBEJK RIE RIL |
| ID | FETCH-LOGICAL-i274t-e77fd6e880d5c5c71b4e6a8158fd6f88dd1a93ede5e4def0bec624b4ad34dc4f3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 277 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000380483600007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Thu Jun 29 18:36:31 EDT 2023 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i274t-e77fd6e880d5c5c71b4e6a8158fd6f88dd1a93ede5e4def0bec624b4ad34dc4f3 |
| PageCount | 7 |
| ParticipantIDs | ieee_primary_7424283 |
| PublicationCentury | 2000 |
| PublicationDate | 20151201 |
| PublicationDateYYYYMMDD | 2015-12-01 |
| PublicationDate_xml | – month: 12 year: 2015 text: 20151201 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA) |
| PublicationTitleAbbrev | ICMLA |
| PublicationYear | 2015 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| Score | 2.1853328 |
| Snippet | Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 38 |
| SubjectTerms | Algorithm design and analysis anomaly detection Benchmark testing benchmarks Detection algorithms Detectors Measurement Real-time systems streaming data time-series data |
| Title | Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark |
| URI | https://ieeexplore.ieee.org/document/7424283 |
| WOSCitedRecordID | wos000380483600007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NTwIxEG2QePCkBozf6cGjlS3bbXePiBBNkBCihhvpbqdAhMXAYuK_d1o2GBMv3pqmzSQznUxfO2-GkBuZcjw3acyMtZoJEQUMo6JkUmkuwMQ68L0I3nqq349Ho2RQIbc7LgwA-OQzuHND_5dvltnGPZU1EMa5-mB7ZE8pueVqlXUYeZA0ntrPvZZL1orQ_fmvbik-WHQP_yfmiNR_WHd0sIsnx6QCeY0MO2U57nxCh3ipY46zQRG0L_T8iz5A4VOpctqaT5aI86eLNWWMovFp34vRu7X3KGC60Kv3Onntdl7aj6zsg8BmiBkLBkpZIwE9zURZlCmeCpA65lGM0zaOjeE6CcFABMKADdAssilSoU0oTCZseEKq-TKHU0ITCQhRUhk2Aytwo3ZspkzLRIFVIJIzUnP6GH9sS12MS1Wc_z19QQ6curfZHZekWqw2cEX2s89itl5de_t8A1xtlLs |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fT8IwEG4QTfRJDRh_2wcfrWys67ZHRAjEsRCChjfSrVcgwjAwTPzvvY4FY-KLb03T5pK7ttevve-OkHsR27huYp8prSXj3LUYekXBhCdtDsqXVl6L4C30osgfjYJ-iTzsuDAAkAefwaNp5n_5aplszFNZDWGcyQ-2R_ZdzuvWlq1VZGK0raDWbfbChgnXcvEAsH_VS8ndRfv4f4JOSPWHd0f7O49ySkqQVsigVSTkTid0gNc6ZlgbFGH7Qs6_6DNkeTBVShvzyRKR_nSxpoxRND-NcjFyN_YJBUwXcvVeJa_t1rDZYUUlBDZD1Jgx8DytBOBeU27iJp4dcxDSt10fu7XvK2XLwAEFLnAF2kLDiDqPuVQOVwnXzhkpp8sUzgkNBCBIiYVTtzTHidLwmRIpAg-0Bzy4IBWjj_HHNtnFuFDF5d_dd-SwM-yF47AbvVyRI6P6bazHNSlnqw3ckIPkM5utV7e5rb4BRkeYAg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2015+IEEE+14th+International+Conference+on+Machine+Learning+and+Applications+%28ICMLA%29&rft.atitle=Evaluating+Real-Time+Anomaly+Detection+Algorithms+--+The+Numenta+Anomaly+Benchmark&rft.au=Lavin%2C+Alexander&rft.au=Ahmad%2C+Subutai&rft.date=2015-12-01&rft.pub=IEEE&rft.spage=38&rft.epage=44&rft_id=info:doi/10.1109%2FICMLA.2015.141&rft.externalDocID=7424283 |