Visual recognition and inference using dynamic overcomplete sparse learning

We present a hierarchical architecture and learning algorithm for visual recognition and other visual inference tasks such as imagination, reconstruction of occluded images, and expectation-driven segmentation. Using properties of biological vision for guidance, we posit a stochastic generative worl...

Full description

Saved in:
Bibliographic Details
Published in:Neural computation Vol. 19; no. 9; p. 2301
Main Authors: Murray, Joseph F, Kreutz-Delgado, Kenneth
Format: Journal Article
Language:English
Published: United States 01.09.2007
Subjects:
Algorithms
Artificial Intelligence
Humans
Learning - physiology
Models, Neurological
Nonlinear Dynamics
Pattern Recognition, Visual - physiology
Photic Stimulation
ISSN:0899-7667
Online Access:Get more information
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a hierarchical architecture and learning algorithm for visual recognition and other visual inference tasks such as imagination, reconstruction of occluded images, and expectation-driven segmentation. Using properties of biological vision for guidance, we posit a stochastic generative world model and from it develop a simplified world model (SWM) based on a tractable variational approximation that is designed to enforce sparse coding. Recent developments in computational methods for learning overcomplete representations (Lewicki & Sejnowski, 2000; Teh, Welling, Osindero, & Hinton, 2003) suggest that overcompleteness can be useful for visual tasks, and we use an overcomplete dictionary learning algorithm (Kreutz-Delgado, et al., 2003) as a preprocessing stage to produce accurate, sparse codings of images. Inference is performed by constructing a dynamic multilayer network with feedforward, feedback, and lateral connections, which is trained to approximate the SWM. Learning is done with a variant of the back-propagation-through-time algorithm, which encourages convergence to desired states within a fixed number of iterations. Vision tasks require large networks, and to make learning efficient, we take advantage of the sparsity of each layer to update only a small subset of elements in a large weight matrix at each iteration. Experiments on a set of rotated objects demonstrate various types of visual inference and show that increasing the degree of overcompleteness improves recognition performance in difficult scenes with occluded objects in clutter.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0899-7667
DOI:10.1162/neco.2007.19.9.2301