Bayesian k-Means as a "maximization-expectation" algorithm

We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering,...

Full description

Saved in:

Bibliographic Details
Published in:	Neural computation Vol. 21; no. 4; p. 1145
Main Authors:	Kurihara, Kenichi, Welling, Max
Format:	Journal Article
Language:	English
Published:	United States 01.04.2009
Subjects:	Algorithms Artificial Intelligence Bayes Theorem Models, Neurological
ISSN:	0899-7667
Online Access:	Get more information
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.
AbstractList	We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature. We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.
Author	Welling, Max Kurihara, Kenichi
Author_xml	– sequence: 1 givenname: Kenichi surname: Kurihara fullname: Kurihara, Kenichi email: kurihara@mi.cs.titech.ac.jp organization: Department of Computer Science, Tokyo Institute of Technology, Meguro-ku, Tokyo 152-8552, Japan. kurihara@mi.cs.titech.ac.jp – sequence: 2 givenname: Max surname: Welling fullname: Welling, Max
BackLink	https://www.ncbi.nlm.nih.gov/pubmed/19199394$$D View this record in MEDLINE/PubMed
BookMark	eNo1j01LxDAYhHNYcT_0FwhS9uAt65ukTRpvuvgFK170XN6kqUbbtDYt7PrrXXSFGWYODwMzJ5PQBkfIGYMVY5JfBmfbFQfIV4xTkDTlbEJmkGtNlZRqSuYxfgCAZJAdkynTTGuh0xm5usGdix5D8kmfHIaY4F7JssGtb_w3Dr4N1G07Z4ffvkywfmt7P7w3J-Sowjq600MuyOvd7cv6gW6e7x_X1xtqU50NlHELoswQpVZWWOkqAMOERmGUrqwxytk0A2MQQAmmU1XmGlhlUijF3nxBLv52u779Gl0cisZH6-oag2vHWEjFQHGR78HzAziaxpVF1_sG-13x_5b_ADLfVwc
CitedBy_id	crossref_primary_10_3390_a11100151 crossref_primary_10_1016_j_patcog_2014_03_006 crossref_primary_10_1016_j_jspi_2020_05_009 crossref_primary_10_1002_sim_9924 crossref_primary_10_1007_s00500_017_2565_z crossref_primary_10_1016_j_patcog_2020_107783 crossref_primary_10_1007_s41870_024_02279_x crossref_primary_10_1111_eea_13123 crossref_primary_10_1162_NECO_a_00534 crossref_primary_10_1016_j_ijar_2017_11_001 crossref_primary_10_1109_ACCESS_2019_2927593 crossref_primary_10_1007_s10994_023_06491_x crossref_primary_10_1016_j_ijepes_2014_11_029 crossref_primary_10_1038_s41467_020_20753_5 crossref_primary_10_1109_TMI_2019_2897112 crossref_primary_10_1155_2013_251905
ContentType	Journal Article
DBID	CGR CUY CVF ECM EIF NPM 7X8
DOI	10.1162/neco.2008.12-06-421
DatabaseName	Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic
DatabaseTitle	MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic
DatabaseTitleList	MEDLINE MEDLINE - Academic
Database_xml	– sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database
DeliveryMethod	no_fulltext_linktorsrc
Discipline	Computer Science
ExternalDocumentID	19199394
Genre	Research Support, U.S. Gov't, Non-P.H.S Journal Article
GroupedDBID	--- -~X .4S .DC 0R~ 123 36B 4.4 41~ 53G 6IK AAFWJ AAJGR AALMD ABAZT ABDBF ABDNZ ABEFU ABIVO ABJNI ABVLG ACGFO ACUHS ACYGS ADIYS ADMLS AEGXH AEILP AENEX AIAGR ALMA_UNASSIGNED_HOLDINGS AMVHM ARCSS AVWKF AZFZN BEFXN BFFAM BGNUA BKEBE BPEOZ CAG CGR COF CS3 CUY CVF DU5 EAP EAS EBC EBD EBS ECM ECS EDO EIF EJD EMB EMK EMOBN EPL EPS EST ESX F5P FEDTE FNEHJ HVGLF HZ~ H~9 I-F IPLJI JAVBF MCG MINIK MKJ NPM O9- OCL P2P PK0 PQQKQ RMI SV3 TUS WG8 WH7 XJE ZWS 7X8
ID	FETCH-LOGICAL-c495t-12c03d5aa697c3c6ef00b139a3b79fcbb7ec450bba00731947d8901fb40d340d2
IEDL.DBID	7X8
ISICitedReferencesCount	24
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000264896200010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0899-7667
IngestDate	Thu Sep 04 16:07:46 EDT 2025 Mon Jul 21 06:04:34 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	4
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c495t-12c03d5aa697c3c6ef00b139a3b79fcbb7ec450bba00731947d8901fb40d340d2
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23
PMID	19199394
PQID	67107238
PQPubID	23479
ParticipantIDs	proquest_miscellaneous_67107238 pubmed_primary_19199394
PublicationCentury	2000
PublicationDate	2009-04-01
PublicationDateYYYYMMDD	2009-04-01
PublicationDate_xml	– month: 04 year: 2009 text: 2009-04-01 day: 01
PublicationDecade	2000
PublicationPlace	United States
PublicationPlace_xml	– name: United States
PublicationTitle	Neural computation
PublicationTitleAlternate	Neural Comput
PublicationYear	2009
SSID	ssj0006105
Score	2.099636
Snippet	We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This...
SourceID	proquest pubmed
SourceType	Aggregation Database Index Database
StartPage	1145
SubjectTerms	Algorithms Artificial Intelligence Bayes Theorem Models, Neurological
Title	Bayesian k-Means as a "maximization-expectation" algorithm
URI	https://www.ncbi.nlm.nih.gov/pubmed/19199394 https://www.proquest.com/docview/67107238
Volume	21
WOSCitedRecordID	wos000264896200010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText
inHoldings	1
isFullTextHit
isPrint
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LSwMxEB6q9eDF-rY-l-I1uLvJJhsRRMXipaUHhd5Kkk20aLfVVtF_72QfeBIPwmZvy4ZhMo_MN_MBnJqUM01ZQmRsYsJi5UgqWUa4MJFLhRIi0wXZhOj30-FQDhpwUffCeFhlbRMLQ51Njb8jP-PoCj1B1uXslXjOKF9brQg0lqBJMZDxgC4x_JkVzisAI2YURHAuqplDEY_PckztShylByZw3GP0e4RZeJpu6397XIe1KsIMrkqV2ICGzTehVbM3BNVh3oLza_VlfQtl8Ex6Fj1WoPAJOhP1OZ5U3ZnEEwCYslrfCdTLI_5v8TTZhofu7f3NHamYFIjBBGhBotiENEuU4lIYarh1YahRZIpqIZ3RWljDklBr5St3kWQiSzFQcJqFGcUV78ByPs3tHgQUTWSYaOGo1cyoSEsdo8E0LrIpph-qDSe1bEaoqb78oHI7fZ-Paum0YbcU72hWDtQYYc6IYZJk-39-ewCrdTknjA6h6fCM2iNYMR-L8fztuFAAfPcHvW_F5bmT
linkProvider	ProQuest
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Bayesian+k-Means+as+a+%22maximization-expectation%22+algorithm&rft.jtitle=Neural+computation&rft.au=Kurihara%2C+Kenichi&rft.au=Welling%2C+Max&rft.date=2009-04-01&rft.issn=0899-7667&rft.volume=21&rft.issue=4&rft.spage=1145&rft_id=info:doi/10.1162%2Fneco.2008.12-06-421&rft_id=info%3Apmid%2F19199394&rft_id=info%3Apmid%2F19199394&rft.externalDocID=19199394
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0899-7667&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0899-7667&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0899-7667&client=summon