Scalable Detection of Anomalous Patterns With Connectivity Constraints

We present GraphScan, a novel method for detecting arbitrarily shaped connected clusters in graph or network data. Given a graph structure, data observed at each node, and a score function defining the anomalousness of a set of nodes, GraphScan can efficiently and exactly identify the most anomalous...

Full description

Saved in:
Bibliographic Details
Published in:Journal of computational and graphical statistics Vol. 24; no. 4; pp. 1014 - 1033
Main Authors: Speakman, Skyler, McFowland, Edward, Neill, Daniel B.
Format: Journal Article
Language:English
Published: Alexandria Taylor & Francis 02.10.2015
American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America
Taylor & Francis Ltd
Subjects:
Biosurveillance
Cluster analysis
Clustering and Pattern Detection
Epidemics
Event detection
Graph mining
Graph theory
Public health
Scan statistics
Sensors
Simulation
Spatial scan statistic
Studies
ISSN:1061-8600, 1537-2715
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present GraphScan, a novel method for detecting arbitrarily shaped connected clusters in graph or network data. Given a graph structure, data observed at each node, and a score function defining the anomalousness of a set of nodes, GraphScan can efficiently and exactly identify the most anomalous (highest-scoring) connected subgraph. Kulldorff's spatial scan, which searches over circles consisting of a center location and its k − 1 nearest neighbors, has been extended to include connectivity constraints by FlexScan. However, FlexScan performs an exhaustive search over connected subsets and is computationally infeasible for k > 30. Alternatively, the upper level set (ULS) scan scales well to large graphs but is not guaranteed to find the highest-scoring subset. We demonstrate that GraphScan is able to scale to graphs an order of magnitude larger than FlexScan, while guaranteeing that the highest-scoring subgraph will be identified. We evaluate GraphScan, Kulldorff's spatial scan (searching over circles) and ULS in two different settings of public health surveillance. The first examines detection power using simulated disease outbreaks injected into real-world Emergency Department data. GraphScan improved detection power by identifying connected, irregularly shaped spatial clusters while requiring less than 4.3 sec of computation time per day of data. The second scenario uses contaminant plumes spreading through a water distribution system to evaluate the spatial accuracy of the methods. GraphScan improved spatial accuracy using data generated from noisy, binary sensors in the network while requiring less than 0.22 sec of computation time per hour of data.
Bibliography:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ISSN:1061-8600
1537-2715
DOI:10.1080/10618600.2014.960926