Detecting spatiotemporal irregularities in videos via a 3D convolutional autoencoder

Spatiotemporal irregularities (i.e., the uncommon appearance and motion patterns) in videos are difficult to detect, as they are usually not well defined and appear rarely in videos. We tackle this problem by learning normal patterns from regular videos, while treating irregularities as deviations f...

Full description

Saved in:
Bibliographic Details
Published in:Journal of visual communication and image representation Vol. 67; p. 102747
Main Authors: Yan, Mengjia, Meng, Jingjing, Zhou, Chunluan, Tu, Zhigang, Tan, Yap-Peng, Yuan, Junsong
Format: Journal Article
Language:English
Published: Elsevier Inc 01.02.2020
Subjects:
3D convolution
Anomaly detection
Autoencoder
Real-time
Spatiotemporal irregularity detection
Unsupervised learning
Spatiotemporal irregularity detection
3D convolution
Autoencoder
Real-time
Anomaly detection
Unsupervised learning
ISSN:1047-3203, 1095-9076
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Spatiotemporal irregularities (i.e., the uncommon appearance and motion patterns) in videos are difficult to detect, as they are usually not well defined and appear rarely in videos. We tackle this problem by learning normal patterns from regular videos, while treating irregularities as deviations from normal patterns. To this end, we introduce a 3D fully convolutional autoencoder (3D-FCAE) that is trainable in an end-to-end manner to detect both temporal and spatiotemporal irregularities in videos using limited training data. Subsequently, temporal irregularities can be detected as frames with high reconstruction errors, and irregular spatiotemporal patterns can be detected as blurry regions that are not well reconstructed. Our approach can accurately locate temporal and spatiotemporal irregularities thanks to the 3D fully convolutional autoencoder and the explored effective architecture. We evaluate the proposed autoencoder for detecting irregular patterns on benchmark video datasets with weak supervision. Comparisons with state-of-the-art approaches demonstrate the effectiveness of our approach. Moreover, the learned autoencoder shows good generalizability across multiple datasets.
ISSN:1047-3203
1095-9076
DOI:10.1016/j.jvcir.2019.102747