Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark
Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to proc...
Uloženo v:
| Vydáno v: | 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA) s. 38 - 44 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.12.2015
|
| Témata: | |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data. |
|---|---|
| AbstractList | Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data. |
| Author | Lavin, Alexander Ahmad, Subutai |
| Author_xml | – sequence: 1 givenname: Alexander surname: Lavin fullname: Lavin, Alexander email: alavin@numenta.com organization: Numenta, Inc., Redwood City, CA, USA – sequence: 2 givenname: Subutai surname: Ahmad fullname: Ahmad, Subutai email: sahmad@numenta.com organization: Numenta, Inc., Redwood City, CA, USA |
| BookMark | eNo9jU9LwzAchiPowU2vXrzkC6Tmb5Mea51zUBVGPY80-XUNtql0mbBv70Dx9MLDw_Mu0GWcIiB0x2jGGC0eNtVrXWacMpUxyS7QgilaUMqNZtdou_q2w9GmEPd4C3YgTRgBl3Ea7XDCT5DApTBFXA77aQ6pHw-YENz0gN-OI8Rk_91HiK4f7fx5g646Oxzg9m-X6ON51VQvpH5fb6qyJoFrmQho3fkcjKFeOeU0ayXk1jBlzrgzxntmCwEeFEgPHW3B5Vy20nohvZOdWKL7324AgN3XHM7np52WXHIjxA9tjUyU |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ICMLA.2015.141 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 1509002871 9781509002870 |
| EndPage | 44 |
| ExternalDocumentID | 7424283 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL CBEJK RIE RIL |
| ID | FETCH-LOGICAL-i274t-e77fd6e880d5c5c71b4e6a8158fd6f88dd1a93ede5e4def0bec624b4ad34dc4f3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 277 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000380483600007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Thu Jun 29 18:36:31 EDT 2023 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i274t-e77fd6e880d5c5c71b4e6a8158fd6f88dd1a93ede5e4def0bec624b4ad34dc4f3 |
| PageCount | 7 |
| ParticipantIDs | ieee_primary_7424283 |
| PublicationCentury | 2000 |
| PublicationDate | 20151201 |
| PublicationDateYYYYMMDD | 2015-12-01 |
| PublicationDate_xml | – month: 12 year: 2015 text: 20151201 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA) |
| PublicationTitleAbbrev | ICMLA |
| PublicationYear | 2015 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| Score | 2.1854353 |
| Snippet | Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 38 |
| SubjectTerms | Algorithm design and analysis anomaly detection Benchmark testing benchmarks Detection algorithms Detectors Measurement Real-time systems streaming data time-series data |
| Title | Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark |
| URI | https://ieeexplore.ieee.org/document/7424283 |
| WOSCitedRecordID | wos000380483600007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NTwIxEG2QePCkBozf6cGjFXa3226PiBBNkBCihhtpt7NAhMXAYuK_d1o2GBMv3ppJ20lm0nSmfW-GkBuuAONeLpkVmWRcmJQpR84RGOvrUAfGelTlW0_2-8lopAYVcrvjwgCAB5_BnRv6v3y7TDfuqayBaZyrD7ZH9qQUW65WWYcxaKrGU_u513JgrRiPf_CrW4q_LLqH_1NzROo_rDs62N0nx6QCeY0MO2U57nxChxjUMcfZoJi0L_T8iz5A4aFUOW3NJ0vM86eLNWWMovNp36vRu7n3qGC60Kv3Onntdl7aj6zsg8BmmDMWDKTMrAA8aTZO41QGhoPQSRAnKM6SxNpAqwgsxMAtZE10iwi54dpG3KY8i05INV_mcEqochGBypJQA-eRkQa3NlpojgtkGMVnpObsMf7YlroYl6Y4_1t8QQ6cubfojktSLVYbuCL76WcxW6-uvX--AfxXlDY |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fT8IwEG4QTfRJDRh_2wcfrbCta7dHRAjEsRCChjfSrTcgwjAwTPzvvY4FY-KLb03T9pK7tL1rv--OkHvuA_q9XDItEsm4iGLmG3KOQF9f2cqKdI6qfAtkGHqjkd8vkYcdFwYAcvAZPJpm_pevl_HGPJXVMIwz-cH2yL6pnFWwtYpMjFbdr3WbvaBh4FouHgDWr3op-XXRPv6foBNS_eHd0f7uRjklJUgrZNAqEnKnEzpAt44Z1gbFsH2h5l_0GbIcTJXSxnyyxEh_ulhTxiian4a5GLUb-4QCpgu1eq-S13Zr2OywohICm2HUmDGQMtECcK9pN3ZjaUUchPIs18PuxPO0tpTvgAYXuIakjoYRNo-40g7XMU-cM1JOlymcE-obn8BPPFsB504kI1w6UkJxnCBtx70gFaOP8cc22cW4UMXl39135LAz7AXjoBu-XJEjo_ot1uOalLPVBm7IQfyZzdar29xW3-Mil38 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2015+IEEE+14th+International+Conference+on+Machine+Learning+and+Applications+%28ICMLA%29&rft.atitle=Evaluating+Real-Time+Anomaly+Detection+Algorithms+--+The+Numenta+Anomaly+Benchmark&rft.au=Lavin%2C+Alexander&rft.au=Ahmad%2C+Subutai&rft.date=2015-12-01&rft.pub=IEEE&rft.spage=38&rft.epage=44&rft_id=info:doi/10.1109%2FICMLA.2015.141&rft.externalDocID=7424283 |