Anomaly Detection Algorithms on IBM InfoSphere Streams: Anomaly Detection for Data in Motion

This paper presents and shares excerpts from our implementation of near real-time anomaly detection algorithms on the IBM InfoSphere Streams platform. The purpose of this article is to: 1) Describe how to design and implement known anomaly detection algorithms on IBM InfoSphere Streams. 2) Present s...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications S. 301 - 308
Hauptverfasser: Yulevich, Y., Pyasik, A., Gorelik, L.
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.07.2012
Schlagworte:
Algorithm design and analysis
Detection algorithms
Dictionaries
Kernel
Measurement
Principal component analysis
Vectors
ISBN:1467316318, 9781467316316
ISSN:2158-9178
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract This paper presents and shares excerpts from our implementation of near real-time anomaly detection algorithms on the IBM InfoSphere Streams platform. The purpose of this article is to: 1) Describe how to design and implement known anomaly detection algorithms on IBM InfoSphere Streams. 2) Present some performance optimization capabilities of IBM InfoSphere Streams platform and propose a method to use them in anomaly detection applications. 3) Present some IBM InfoSphere Streams best practices and describe how their adoption in the context of anomaly detection application. The document describes the architecture and design of anomaly detection algorithms developed on IBM InfoSphere Streams. Although the solution was designed to be used for cyber security, the implemented algorithms are agnostic regarding the data type that they monitor and therefore can detect anomalies in data from various industries such as healthcare, finance and retail. The document describes the implementation of two anomaly detection algorithms: KOAD and PCA. The KOAD algorithm performs online anomaly detection with incremental learning and the PCA algorithm in performs offline anomaly detection. The solution was designed to provide near real-time insight into low latency on large data volume observation.
AbstractList This paper presents and shares excerpts from our implementation of near real-time anomaly detection algorithms on the IBM InfoSphere Streams platform. The purpose of this article is to: 1) Describe how to design and implement known anomaly detection algorithms on IBM InfoSphere Streams. 2) Present some performance optimization capabilities of IBM InfoSphere Streams platform and propose a method to use them in anomaly detection applications. 3) Present some IBM InfoSphere Streams best practices and describe how their adoption in the context of anomaly detection application. The document describes the architecture and design of anomaly detection algorithms developed on IBM InfoSphere Streams. Although the solution was designed to be used for cyber security, the implemented algorithms are agnostic regarding the data type that they monitor and therefore can detect anomalies in data from various industries such as healthcare, finance and retail. The document describes the implementation of two anomaly detection algorithms: KOAD and PCA. The KOAD algorithm performs online anomaly detection with incremental learning and the PCA algorithm in performs offline anomaly detection. The solution was designed to provide near real-time insight into low latency on large data volume observation.
Author Yulevich, Y.
Pyasik, A.
Gorelik, L.
Author_xml – sequence: 1
  givenname: Y.
  surname: Yulevich
  fullname: Yulevich, Y.
  email: yifat-yuleich@il.ibm.com
  organization: Software Lab., Software Solutions Dept., IBM Israel Software Lab., Rehovot, Israel
– sequence: 2
  givenname: A.
  surname: Pyasik
  fullname: Pyasik, A.
  email: apyasic@il.ibm.com
  organization: Software Lab., Software Solutions Dept., IBM Israel Software Lab., Rehovot, Israel
– sequence: 3
  givenname: L.
  surname: Gorelik
  fullname: Gorelik, L.
  email: leonid.gorelik@il.ibm.com
  organization: Software Lab., Software Solutions Dept., IBM Israel Software Lab., Rehovot, Israel
BookMark eNpljDtPwzAURo0oEm3JyMTiP5Bwr-3YMVtoeURqBVI6IlV5XNOgJK6SLP33FMHGdPQd6TsLNut9T4zdIkSIYO-z_D2NBKCIUMUXLLAmAaNtrAxgfMkWqLSRqCUmMzYXGCehRZNcs2AcvwDg7BNrYc4-0t53RXvia5qomhrf87T99EMzHbqRn1f2uOVZ73x-PNBAPJ8GKrrxgf__OT_wdTEVvOn51v-oG3blinak4I9Ltnt-2q1ew83bS7ZKN2FjYQpRYSmUVRJAgxGqrIWoyaDQsdWudkKSUWArI0DETlWVdKYqDRmnpJMEcsnufrMNEe2PQ9MVw2mvRQIStPwGGklVhA
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/ISPA.2012.145
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEL
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9780769547015
076954701X
EndPage 308
ExternalDocumentID 6280306
Genre orig-research
GroupedDBID 6IE
6IF
6IH
6IK
6IL
6IN
AAJGR
AAWTH
ABLEC
ACGFS
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
M43
OCL
RIE
RIL
ID FETCH-LOGICAL-i90t-141b249430060724bd22de7126596fdf23e7409c72025f4cc3f7cb7e7f43f3e03
IEDL.DBID RIE
ISBN 1467316318
9781467316316
ISSN 2158-9178
IngestDate Wed Aug 27 04:58:40 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i90t-141b249430060724bd22de7126596fdf23e7409c72025f4cc3f7cb7e7f43f3e03
PageCount 8
ParticipantIDs ieee_primary_6280306
PublicationCentury 2000
PublicationDate 2012-July
PublicationDateYYYYMMDD 2012-07-01
PublicationDate_xml – month: 07
  year: 2012
  text: 2012-July
PublicationDecade 2010
PublicationTitle 2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications
PublicationTitleAbbrev ispa
PublicationYear 2012
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003188990
ssj0002672344
Score 1.4918693
Snippet This paper presents and shares excerpts from our implementation of near real-time anomaly detection algorithms on the IBM InfoSphere Streams platform. The...
SourceID ieee
SourceType Publisher
StartPage 301
SubjectTerms Algorithm design and analysis
Detection algorithms
Dictionaries
Kernel
Measurement
Principal component analysis
Vectors
Title Anomaly Detection Algorithms on IBM InfoSphere Streams: Anomaly Detection for Data in Motion
URI https://ieeexplore.ieee.org/document/6280306
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fT8IwEG6Q-OATKhjFH-mDj05Y27WbbygSSYSQQAwPJqTtOl0Cw8Aw8b-3VwbE6ItvuzVbll5v1-vdfR9C10pzzSj3rSEx7TFpQk8a6rpAEqGtmpVrj355Fv1-OB5HgxK62fbCGGNc8Zm5hUuXy4_negVHZQ0OVEqAr70nBF_3am3PUwgXhBauCmS7Vm0oAUcs1qmBTYvQ9XVxoGqygxu4p0LmO_zNRnc4aEHRF7G_keAH64pzOp3K_z73ENV23Xt4sPVLR6hksmNU2dA34MKaq-jVhv4zOf3CbZO7gqwMt6Zv80Wav8-W2Erd-x6GrNQQoAcMhgS2nC3v8O_n7M4Xt2UucZrhnuMFqqFR53H08OQVZAteGjVzz2e-spEYowDQIghTMSGxET7hQcSTOCHUCBsKakHsJilhWoMqlTAiYTShpklPUDmbZ-YUYV8IbV8XyVA3WRIQZSOuSMkQkGBUGERnqApzNflYw2lMimmq_337HB2AKtYVsheonC9W5hLt6888XS6u3Br4BrfCqO8
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fT8IwEG4ImugTKhh_2wcfnWxd126-oUggAiGBGB5MSFs6XQLDwDDxv7dXBsToi2-7NVuWXm_X6919H0I3UjFFfeYZQ6LKoUKHjtC-7QKJuTJqlrY9-qXNu91wOIx6BXS76YXRWtviM30HlzaXP56pJRyVVRlQKQG-9k5AKXFX3VqbExXCOPFzZwWyWa0mmIBDFuPWwKp5aDu7GJA1mcE14FMusy0CZ7XV79Wg7IuYH0nwg3fFup1G6X8ffIAq2_493Nt4pkNU0OkRKq0JHHBuz2X0aoL_qZh84brObElWimuTt9k8yd6nC2yk1kMHQ16qD-ADGkMKW0wX9_j3c2bvi-siEzhJcccyA1XQoPE0eGw6Od2Ck0Ru5njUkyYWoz5AtHBC5ZiQseYeYUHE4nFMfM1NMKg4MdukmCoFypRc85j6sa9d_xgV01mqTxD2OFfmdZEIlUvjgEgTc0VShIAFI8MgOkVlmKvRxwpQY5RP09nft6_RXnPQaY_are7zOdoHtazqZS9QMZsv9SXaVZ9Zsphf2fXwDbi5rDY
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2012+IEEE+10th+International+Symposium+on+Parallel+and+Distributed+Processing+with+Applications&rft.atitle=Anomaly+Detection+Algorithms+on+IBM+InfoSphere+Streams%3A+Anomaly+Detection+for+Data+in+Motion&rft.au=Yulevich%2C+Y.&rft.au=Pyasik%2C+A.&rft.au=Gorelik%2C+L.&rft.date=2012-07-01&rft.pub=IEEE&rft.isbn=1467316318&rft.issn=2158-9178&rft.spage=301&rft.epage=308&rft_id=info:doi/10.1109%2FISPA.2012.145&rft.externalDocID=6280306
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2158-9178&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2158-9178&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2158-9178&client=summon