Classification via Minimum Incremental Coding Length

We present a simple new criterion for classification, based on principles from lossy data compression. The criterion assigns a test sample to the class that uses the minimum number of additional bits to code the test sample, subject to an allowable distortion. We demonstrate the asymptotic optimalit...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:SIAM journal on imaging sciences Jg. 2; H. 2; S. 367 - 395
Hauptverfasser: Wright, John, Ma, Yi, Tao, Yangyu, Lin, Zhouchen, Shum, Heung-Yeung
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Philadelphia Society for Industrial and Applied Mathematics 01.01.2009
Schlagworte:
Asymptotic properties
Classification
Classifiers
Coding
Criteria
Matlab
Support vector machines
ISSN:1936-4954, 1936-4954
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We present a simple new criterion for classification, based on principles from lossy data compression. The criterion assigns a test sample to the class that uses the minimum number of additional bits to code the test sample, subject to an allowable distortion. We demonstrate the asymptotic optimality of this criterion for Gaussian distributions and analyze its relationships to classical classifiers. The theoretical results clarify the connections between our approach and popular classifiers such as maximum a posteriori (MAP), regularized discriminant analysis (RDA), $k$-nearest neighbor ($k$-NN), and support vector machine (SVM), as well as unsupervised methods based on lossy coding. Our formulation induces several good effects on the resulting classifier. First, minimizing the lossy coding length induces a regularization effect which stabilizes the (implicit) density estimate in a small sample setting. Second, compression provides a uniform means of handling classes of varying dimension. The new criterion and its kernel and local versions perform competitively on synthetic examples, as well as on real imagery data such as handwritten digits and face images. On these problems, the performance of our simple classifier approaches the best reported results, without using domain-specific information. All MATLAB code and classification results are publicly available for peer evaluation at http://perception.csl.uiuc.edu/coding/home.htm.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
content type line 14
ObjectType-Article-2
ObjectType-Feature-1
content type line 23
ISSN:1936-4954
1936-4954
DOI:10.1137/070707312