Anomaly Detection in Networking Logs Using Unsupervised Autoencoder Learning
Modern cloud-based infrastructures frequently operate across multilayered, multi-OS environments supported by numerous vendors. Diagnosing anomalies in such complex systems is a time-consuming task that often results in inaccurate root cause attribution, harming the credibility of otherwise reliable...
Uložené v:
| Vydané v: | ICFAI journal of computer sciences Ročník 19; číslo 3; s. 47 - 60 |
|---|---|
| Hlavný autor: | |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Hyderabad
IUP Publications
10.07.2025
|
| Predmet: | |
| ISSN: | 0973-9904 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Modern cloud-based infrastructures frequently operate across multilayered, multi-OS environments supported by numerous vendors. Diagnosing anomalies in such complex systems is a time-consuming task that often results in inaccurate root cause attribution, harming the credibility of otherwise reliable service components. The paper addresses the critical challenge of anomaly detection in networking logs and subsequent root cause analysis through hardware status data in such virtualized infrastructures. It identifies the limitations of traditional anomaly detection methods, including clustering-based (LOF SOF), statistical (Gaussian-based), rule-based and supervised approaches, which often fail under noisy, high-dimensional, or sparsely labeled settings. To overcome these limitations, the paper proposes a two-stage architecture: (1) an autoencoder-based anomaly detector trained on textual networking logs; and (2) a self-supervised long short-term memory (LSTM) autoencoder trained on hardware metrics augmented by log-derived anomaly flags. This hybrid approach captures temporal dependencies, reduces false positives, and improves root cause traceability. Evaluated on a proprietary dataset comprising over 200K entries, the proposed method outperformed traditional baselines, achieving an F1 score of 0.87, surpassing others by a margin of at least 12%. This solution offers a scalable and automated diagnostic tool for distributed systems with minimal human intervention. |
|---|---|
| AbstractList | Modern cloud-based infrastructures frequently operate across multilayered, multi-OS environments supported by numerous vendors. Diagnosing anomalies in such complex systems is a time-consuming task that often results in inaccurate root cause attribution, harming the credibility of otherwise reliable service components. The paper addresses the critical challenge of anomaly detection in networking logs and subsequent root cause analysis through hardware status data in such virtualized infrastructures. It identifies the limitations of traditional anomaly detection methods, including clustering-based (LOF SOF), statistical (Gaussian-based), rule-based and supervised approaches, which often fail under noisy, high-dimensional, or sparsely labeled settings. To overcome these limitations, the paper proposes a two-stage architecture: (1) an autoencoder-based anomaly detector trained on textual networking logs; and (2) a self-supervised long short-term memory (LSTM) autoencoder trained on hardware metrics augmented by log-derived anomaly flags. This hybrid approach captures temporal dependencies, reduces false positives, and improves root cause traceability. Evaluated on a proprietary dataset comprising over 200K entries, the proposed method outperformed traditional baselines, achieving an F1 score of 0.87, surpassing others by a margin of at least 12%. This solution offers a scalable and automated diagnostic tool for distributed systems with minimal human intervention. |
| Author | Bar, Kaushik |
| Author_xml | – sequence: 1 givenname: Kaushik surname: Bar fullname: Bar, Kaushik |
| BookMark | eNotjctOwzAURL0oEqX0G7DEOqlf8U2WUXkVRYBEs67s-KZKKXaJExB_TxDM5sziaOaCzHzwSMgVZylwKYrVpn55XL-uBBNZyotUpgoSzWZkzgqQSVEwdU6WMR7YlFxkisOcVKUP7-b4TW9wwGbogqedp084fIX-rfN7WoV9pHX8rbWP4wn7zy6io-U4BPRNcNjTCk3vJ-OSnLXmGHH5zwXZ3t1u1w9J9Xy_WZdV0oDOE7SucIDOOS3BWTCgWtOgy7AF4MoJIxBBtFY3kyqYbpjNrGxB20yBMXJBrv9mT334GDEOu0MYez897qSQSgrO81z-ADG4U-M |
| ContentType | Journal Article |
| Copyright | Copyright IUP Publications 2025 |
| Copyright_xml | – notice: Copyright IUP Publications 2025 |
| DBID | 04Q 04S 04W 3V. 7XB 8AL 8FE 8FG 8FK ABUWG AFKRA ARAPS AZQEC BENPR BGLVJ CCPQU DWQXO GNUQQ HCIFZ JQ2 K7- M0N P5Z P62 PHGZM PHGZT PKEHL PQEST PQGLB PQQKQ PQUKI Q9U |
| DOI | 10.71329/IUPJCS/2025.19.3.47-60 |
| DatabaseName | India Database (ProQuest) India Database: Business India Database: Science & Technology ProQuest Central (Corporate) ProQuest Central (purchase pre-March 2016) Computing Database (Alumni Edition) ProQuest SciTech Collection ProQuest Technology Collection ProQuest Central (Alumni) (purchase pre-March 2016) ProQuest Central (Alumni) ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Essentials - QC ProQuest Central Technology Collection ProQuest One ProQuest Central Korea ProQuest Central Student SciTech Premium Collection ProQuest Computer Science Collection Computer Science Database Computing Database Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic (New) ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central Basic |
| DatabaseTitle | Computer Science Database ProQuest Central Student Technology Collection ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Central ProQuest One Applied & Life Sciences ProQuest Central Korea ProQuest Central (New) ProQuest Indian Journals Advanced Technologies & Aerospace Collection ProQuest Computing ProQuest Central Basic ProQuest Computing (Alumni Edition) ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection Advanced Technologies & Aerospace Database Indian Journals: Business ProQuest One Academic UKI Edition Indian Journals: Science & Technology ProQuest One Academic ProQuest Central (Alumni) ProQuest One Academic (New) |
| DatabaseTitleList | Computer Science Database |
| Database_xml | – sequence: 1 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EndPage | 60 |
| GroupedDBID | .4S .DC 04Q 04S 04W 3V. 7XB 8AL 8FE 8FG 8FK ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS ARAPS ARCSS AZQEC BENPR BGLVJ CCPQU DWQXO EDO EOJEC GNUQQ HCIFZ JQ2 K7- M0N OBODZ P62 PHGZM PHGZT PKEHL PQEST PQGLB PQQKQ PQUKI Q9U TUS |
| ID | FETCH-LOGICAL-c768-ebd9d7eddd637db7a74faced5ef7714d2a2ee72fb6cebd206c0b5b3f76b547aa3 |
| IEDL.DBID | 04Q |
| ISSN | 0973-9904 |
| IngestDate | Tue Jul 29 19:24:45 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Issue | 3 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c768-ebd9d7eddd637db7a74faced5ef7714d2a2ee72fb6cebd206c0b5b3f76b547aa3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| PQID | 3234321188 |
| PQPubID | 2029993 |
| PageCount | 14 |
| ParticipantIDs | proquest_journals_3234321188 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-07-10 |
| PublicationDateYYYYMMDD | 2025-07-10 |
| PublicationDate_xml | – month: 07 year: 2025 text: 2025-07-10 day: 10 |
| PublicationDecade | 2020 |
| PublicationPlace | Hyderabad |
| PublicationPlace_xml | – name: Hyderabad |
| PublicationTitle | ICFAI journal of computer sciences |
| PublicationYear | 2025 |
| Publisher | IUP Publications |
| Publisher_xml | – name: IUP Publications |
| SSID | ssj0000825417 |
| Score | 2.2970753 |
| Snippet | Modern cloud-based infrastructures frequently operate across multilayered, multi-OS environments supported by numerous vendors. Diagnosing anomalies in such... |
| SourceID | proquest |
| SourceType | Aggregation Database |
| StartPage | 47 |
| SubjectTerms | Algorithms Anomalies Automation Cloud computing Clustering Complex systems Deep learning Hardware Neural networks Root cause analysis Time series |
| Title | Anomaly Detection in Networking Logs Using Unsupervised Autoencoder Learning |
| URI | https://www.proquest.com/docview/3234321188 |
| Volume | 19 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwELagZWChPMWjIA-sbp04jpMJldKKRxUFaKWKpXJsp6oESWlSJP49duoCEwtLpChWZNm-u-8uX-4D4FJIwgQOFBKpI5GG1BglLEiQo1JMuUtCGopKbIJFUTAeh7EtuBWWVrn2iZWjlrkwNfI2cc0vkBoOB1fzd2RUo8zXVSuhsQnqJk4Zw8Te43eNpUp_KtFd05QG6Yl4K4oXM_Lq7btRfN99Nuk_bTlhi7Q8hmyryt9OuYo0_cZ_57gLdizGhJ3VodgDGyrbB421fgO05nwABjr3f-Ovn_BGlRUjK4OzDEYrYrgOaXCQTwtYkQrgKCuWc-NXCiVhZ1nmpgGm1G-zDVqnh2DY7w27t8iqKyChUwykEhlKpqSUPmEyYZx5KRdKUpUy5njS5a5SzE0TX-ihLvYFTmhCUuYn1GOckyNQy_JMHQOoUZ8IHOJLSoQXyCBIPZdzrEiq4RXF4gQ018s1sRZSTH7W6vTvx2dg2-yQqac6uAlq5WKpzsGW-ChnxeKi2vALUL_uRfGTvntgSF9j-vIFCkm4YQ |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V07T8MwED6VhwQL5SneeIDRbWInsTMgVLWglpaqEkXqVjm2UyFBAqQF8aP4j9hpAkxsDMyOLDl3vvt8_nwfwKlUlEmHayxjV2EDqR0cMR5hV8eOLwgN_VDmYhOs3-ejUTiowEf5FsbSKsuYmAdqlUpbI69TYp9AGjjML56esVWNsrerpYTG3C26-v3NHNmy807L2PeMkKvLYbONC1UBLA20xjpSoWJaKRVQpiImmBcLqZWvY8ZcTxFBtGYkjgJpPiVOIJ3Ij2jMgsj3mBDUTLsASx7lgd1QXYa_Sjr5aSvX-LU9cLBZtzdnlDGr5l7v3A2um7e22uDX3LBGax7DRWfMnzkgT2xX1X_2S9ZhrUDQqDF3-Q2o6GQTqqU6BSqC1Rb0Gkn6KB7eUUtPc75Zgu4T1J_T3k3CRr10kqGcMoHukmz2ZKNmphVqzKapbe-pzGxF-9nJNgz_Ykk7sJikid4FZDCt5C4NlE-lxxXnsUeEcDSNDXj0HbkHh6V1xsX-z8bfptn_ffgEVtrDm9641-l3D2DVOoetHLvOISxOX2b6CJbl6_Q-eznOfQ3B-I8N-QnMixaO |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3JTsMwEB1BQYgLZRVLAR_g6Daxkzg5IFQoFYUqqlgkbpXjpUKCBEgL4tP4O-w0AU7cOHBOZCme8cybyfM8gAMhKRNOqLDQrsQGUjs4YWGCXaUdnxMa-ZEoxCZYHId3d9FgBj6quzCWVlnFxCJQy0zYHnmLEnsF0sDhsKVLWsSg0z1-esZWQcr-aa3kNKYucqne30z5lh_1OsbWh4R0z25Oz3GpMICFgdlYJTKSTEkpA8pkwjjzNBdK-koz5nqScKIUIzoJhHmVOIFwEj-hmgWJ7zHOqVl2FuYYNTVPDeZOzuLB1VeDp6i9CsVfOxEHm13wpvwyZrXdW73bwcXpte09-E03atKmx3A5J_NnRijSXLf-jzdoGZZKbI3a08OwAjMqXYV6pVuByjC2Bv12mj3yh3fUUeOCiZai-xTFU0K8SeWon41yVJAp0G2aT55sPM2VRO3JOLODP6VZrRxMO1qHm7_4pA2opVmqNgEZtCtClwbSp8ILZRhqj3DuKKoNrPQdsQWNylLDMjLkw28zbf_-eB8WjP2G_V58uQOL1k9sS9l1GlAbv0zULsyL1_F9_rJXOh6C4R9b8hMoLCDg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Anomaly+Detection+in+Networking+Logs+Using+Unsupervised+Autoencoder+Learning&rft.jtitle=ICFAI+journal+of+computer+sciences&rft.au=Bar%2C+Kaushik&rft.date=2025-07-10&rft.pub=IUP+Publications&rft.issn=0973-9904&rft.volume=19&rft.issue=3&rft.spage=47&rft.epage=60&rft_id=info:doi/10.71329%2FIUPJCS%2F2025.19.3.47-60&rft.externalDBID=HAS_PDF_LINK |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0973-9904&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0973-9904&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0973-9904&client=summon |