Hunting attacks in the dark: clustering and correlation analysis for unsupervised anomaly detection
Summary Network anomalies and attacks represent a serious challenge to ISPs, who need to cope with an increasing number of unknown events that put their networks' integrity at risk. Most of the network anomaly detection systems proposed so far employ a supervised strategy to accomplish their ta...
Gespeichert in:
| Veröffentlicht in: | International Journal of Network Management Jg. 25; H. 5; S. 283 - 305 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Chichester
Blackwell Publishing Ltd
01.09.2015
Wiley Wiley Subscription Services, Inc |
| Schriftenreihe: | Measure, Detect and Mitigate ‐ Challenges and Trends in Network Security |
| Schlagworte: | |
| ISSN: | 1055-7148, 1099-1190 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Zusammenfassung: | Summary
Network anomalies and attacks represent a serious challenge to ISPs, who need to cope with an increasing number of unknown events that put their networks' integrity at risk. Most of the network anomaly detection systems proposed so far employ a supervised strategy to accomplish their task, using either signature‐based detection methods or supervised‐learning techniques. The former fails to detect unknown anomalies, exposing the network to severe consequences; the latter requires labeled traffic, which is difficult and expensive to produce. In this paper, we introduce a powerful unsupervised approach to detect and characterize network anomalies in the dark, that is, without relying on signatures or labeled traffic. Unsupervised detection is accomplished by means of robust clustering techniques, combining subspace clustering with correlation analysis to blindly identify anomalies. To alleviate network operator's post‐processing tasks and to speed up the deployment of effective countermeasures, anomaly ranking and characterization are automatically performed on the detected events. The system is extensively tested with real traffic from the Widely Integrated Distributed Environment backbone network, spanning 6years of flows captured from a trans‐Pacific link between Japan and the USA, using the MAWILab framework for ground‐truth generation. We additionally evaluate the proposed approach with synthetic data, consisting of traffic from an operational network with synthetic attacks. Finally, we compare the performance of the unsupervised detection against different previously used unsupervised detection techniques, as well as against multiple anomaly detectors used in MAWILab. Copyright © 2015 John Wiley & Sons, Ltd.
This article presents an unsupervised approach to detect and characterize network attacks without relying on signatures, training, or labelled traffic. It uses robust unsupervised machine‐learning techniques to unveil anomalous patterns in traffic flows, reducing the intervention of a human network operator. Through extensive evaluation, we show that it not only outperforms previous unsupervised detectors but also achieves high detection accuracy, comparable with that of standard supervised approaches. Our results show that unsupervised detection and characterization of attacks is feasible, opening the door to a new generation of autonomous security algorithms. |
|---|---|
| Bibliographie: | ArticleID:NEM1903 ark:/67375/WNG-R4SH98W4-T istex:071D030CDA99205A2F1C73B7BF3C61B0E4C74896 ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23 |
| ISSN: | 1055-7148 1099-1190 |
| DOI: | 10.1002/nem.1903 |