A semantically driven self-supervised algorithm for detecting anomalies in image sets

Humans can readily detect when an image does not belong in a set by comparing semantic information between images to derive meaning and relationships from the colors, shapes, sizes, locations, and textures within them. Current self-supervised anomaly detection algorithms in computer vision do not po...

Full description

Saved in:
Bibliographic Details
Published in:Computer vision and image understanding Vol. 213; p. 103279
Main Authors: Wheeler, Bradley J., Karimi, Hassan A.
Format: Journal Article
Language:English
Published: Elsevier Inc 01.12.2021
Subjects:
Anomaly detection
Computer vision
Dimensionality reduction
Feature extraction
Land use classification
Multivariate statistics
Pretext learning
Remote sensing
Representation learning
Self-supervised learning
Representation learning
Dimensionality reduction
Computer vision
Pretext learning
Multivariate statistics
Land use classification
Self-supervised learning
Feature extraction
Anomaly detection
Remote sensing
ISSN:1077-3142
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Humans can readily detect when an image does not belong in a set by comparing semantic information between images to derive meaning and relationships from the colors, shapes, sizes, locations, and textures within them. Current self-supervised anomaly detection algorithms in computer vision do not possess this ability. Most algorithms learn to detect anomalies through a training objective that only optimizes for machine-level features and do not evaluate semantic features. To fill this gap, we propose a novel self-supervised algorithm that detects anomalies by learning and modeling semantic information within a set of images. This is accomplished by first training our algorithm to be sensitive or invariant to targeted semantic information, and then modeling the semantic relationships learned so that we can detect anomalies by measuring how far images deviate from the model. Experimenting with our algorithm, we show that different datasets can have different semantic sensitivities, those sensitivities can fluctuate between and within sets depending on different aspects of the images, and directly targeting those sensitivities can improve anomaly detection performance. Using our algorithm, we were able to achieve a AUROC of 0.7146 on the CIFAR-10 dataset, which is an improvement of 0.0741 over another current leading self-supervised anomaly detection algorithm. •Self-supervised algorithm to learn and model semantic representations of image sets.•Method to allow anomaly detection models to learn targeted invariances to image sets.•Self-supervised anomaly detection algorithm requiring only one network to operate.
ISSN:1077-3142
DOI:10.1016/j.cviu.2021.103279