Bayesian k-Means as a "maximization-expectation" algorithm

We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering,...

Full description

Saved in:
Bibliographic Details
Published in:Neural computation Vol. 21; no. 4; p. 1145
Main Authors: Kurihara, Kenichi, Welling, Max
Format: Journal Article
Language:English
Published: United States 01.04.2009
Subjects:
Algorithms
Artificial Intelligence
Bayes Theorem
Models, Neurological
ISSN:0899-7667
Online Access:Get more information
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0899-7667
DOI:10.1162/neco.2008.12-06-421