Bayesian k-Means as a "maximization-expectation" algorithm

We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering,...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Neural computation Ročník 21; číslo 4; s. 1145
Hlavní autoři: Kurihara, Kenichi, Welling, Max
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States 01.04.2009
Témata:
Algorithms
Artificial Intelligence
Bayes Theorem
Models, Neurological
ISSN:0899-7667
On-line přístup:Zjistit podrobnosti o přístupu
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.
AbstractList We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.
We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.
Author Welling, Max
Kurihara, Kenichi
Author_xml – sequence: 1
  givenname: Kenichi
  surname: Kurihara
  fullname: Kurihara, Kenichi
  email: kurihara@mi.cs.titech.ac.jp
  organization: Department of Computer Science, Tokyo Institute of Technology, Meguro-ku, Tokyo 152-8552, Japan. kurihara@mi.cs.titech.ac.jp
– sequence: 2
  givenname: Max
  surname: Welling
  fullname: Welling, Max
BackLink https://www.ncbi.nlm.nih.gov/pubmed/19199394$$D View this record in MEDLINE/PubMed
BookMark eNo1j01LxDAYhHNYcT_0FwhS9uAt65ukTRpvuvgFK170XN6kqUbbtDYt7PrrXXSFGWYODwMzJ5PQBkfIGYMVY5JfBmfbFQfIV4xTkDTlbEJmkGtNlZRqSuYxfgCAZJAdkynTTGuh0xm5usGdix5D8kmfHIaY4F7JssGtb_w3Dr4N1G07Z4ffvkywfmt7P7w3J-Sowjq600MuyOvd7cv6gW6e7x_X1xtqU50NlHELoswQpVZWWOkqAMOERmGUrqwxytk0A2MQQAmmU1XmGlhlUijF3nxBLv52u779Gl0cisZH6-oag2vHWEjFQHGR78HzAziaxpVF1_sG-13x_5b_ADLfVwc
CitedBy_id crossref_primary_10_3390_a11100151
crossref_primary_10_1016_j_patcog_2014_03_006
crossref_primary_10_1016_j_jspi_2020_05_009
crossref_primary_10_1002_sim_9924
crossref_primary_10_1007_s00500_017_2565_z
crossref_primary_10_1016_j_patcog_2020_107783
crossref_primary_10_1007_s41870_024_02279_x
crossref_primary_10_1111_eea_13123
crossref_primary_10_1162_NECO_a_00534
crossref_primary_10_1016_j_ijar_2017_11_001
crossref_primary_10_1109_ACCESS_2019_2927593
crossref_primary_10_1007_s10994_023_06491_x
crossref_primary_10_1016_j_ijepes_2014_11_029
crossref_primary_10_1038_s41467_020_20753_5
crossref_primary_10_1109_TMI_2019_2897112
crossref_primary_10_1155_2013_251905
ContentType Journal Article
DBID CGR
CUY
CVF
ECM
EIF
NPM
7X8
DOI 10.1162/neco.2008.12-06-421
DatabaseName Medline
MEDLINE
MEDLINE (Ovid)
MEDLINE
MEDLINE
PubMed
MEDLINE - Academic
DatabaseTitle MEDLINE
Medline Complete
MEDLINE with Full Text
PubMed
MEDLINE (Ovid)
MEDLINE - Academic
DatabaseTitleList MEDLINE
MEDLINE - Academic
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: 7X8
  name: MEDLINE - Academic
  url: https://search.proquest.com/medline
  sourceTypes: Aggregation Database
DeliveryMethod no_fulltext_linktorsrc
Discipline Computer Science
ExternalDocumentID 19199394
Genre Research Support, U.S. Gov't, Non-P.H.S
Journal Article
GroupedDBID ---
-~X
.4S
.DC
0R~
123
36B
4.4
41~
53G
6IK
AAFWJ
AAJGR
AALMD
ABAZT
ABDBF
ABDNZ
ABEFU
ABIVO
ABJNI
ABVLG
ACGFO
ACUHS
ACYGS
ADIYS
ADMLS
AEGXH
AEILP
AENEX
AIAGR
ALMA_UNASSIGNED_HOLDINGS
AMVHM
ARCSS
AVWKF
AZFZN
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CAG
CGR
COF
CS3
CUY
CVF
DU5
EAP
EAS
EBC
EBD
EBS
ECM
ECS
EDO
EIF
EJD
EMB
EMK
EMOBN
EPL
EPS
EST
ESX
F5P
FEDTE
FNEHJ
HVGLF
HZ~
H~9
I-F
IPLJI
JAVBF
MCG
MINIK
MKJ
NPM
O9-
OCL
P2P
PK0
PQQKQ
RMI
SV3
TUS
WG8
WH7
XJE
ZWS
7X8
ID FETCH-LOGICAL-c495t-12c03d5aa697c3c6ef00b139a3b79fcbb7ec450bba00731947d8901fb40d340d2
IEDL.DBID 7X8
ISICitedReferencesCount 24
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000264896200010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0899-7667
IngestDate Thu Sep 04 16:07:46 EDT 2025
Mon Jul 21 06:04:34 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 4
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c495t-12c03d5aa697c3c6ef00b139a3b79fcbb7ec450bba00731947d8901fb40d340d2
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
PMID 19199394
PQID 67107238
PQPubID 23479
ParticipantIDs proquest_miscellaneous_67107238
pubmed_primary_19199394
PublicationCentury 2000
PublicationDate 2009-04-01
PublicationDateYYYYMMDD 2009-04-01
PublicationDate_xml – month: 04
  year: 2009
  text: 2009-04-01
  day: 01
PublicationDecade 2000
PublicationPlace United States
PublicationPlace_xml – name: United States
PublicationTitle Neural computation
PublicationTitleAlternate Neural Comput
PublicationYear 2009
SSID ssj0006105
Score 2.0997398
Snippet We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This...
SourceID proquest
pubmed
SourceType Aggregation Database
Index Database
StartPage 1145
SubjectTerms Algorithms
Artificial Intelligence
Bayes Theorem
Models, Neurological
Title Bayesian k-Means as a "maximization-expectation" algorithm
URI https://www.ncbi.nlm.nih.gov/pubmed/19199394
https://www.proquest.com/docview/67107238
Volume 21
WOSCitedRecordID wos000264896200010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8NAEB6q9eDF-rY-Q_G6NJuku4kIomLxYEsPCr2V3cmuFm1abRX9987mgSfxICS55cEwsztf5pv5AE65IsRFeQPTeZnRMcQ0os8CpKhEEwdW5SPz72S_Hw-HyaAG51UvjKNVVmtivlCnU3T_yNuCtkInkHUxe2VOM8rVVksBjSWoh5TIOEKXHP7MChclgZEQBZNCyHLmEBdBOyNoV_AoHTFBsCjgv2eY-U7TbfzvG9dhrcwwvcvCJTagZrJNaFTqDV4ZzFtwdqW-jGuh9J5Zz9CO5Sk6vNZEfY4nZXcmcwIAWFTrW556eaT3LZ4m2_DQvbm_vmWlkgJDAkALxgP0w7SjlEgkhiiM9X1NJlOhlolFraXBqONrrVzljieRTGNKFKyO_DSkM9iB5WyamT3wbBQit1FqUBOq1jZJ0lSLkFuhJT01bsJJZZsReaorP6jMTN_no8o6TdgtzDuaFQM1RoQZyUmSaP_Pew9gtSrn-PwQ6pZi1BzBCn4sxvO349wB6Nof9L4Bd4e58Q
linkProvider ProQuest
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Bayesian+k-Means+as+a+%22maximization-expectation%22+algorithm&rft.jtitle=Neural+computation&rft.au=Kurihara%2C+Kenichi&rft.au=Welling%2C+Max&rft.date=2009-04-01&rft.issn=0899-7667&rft.volume=21&rft.issue=4&rft.spage=1145&rft_id=info:doi/10.1162%2Fneco.2008.12-06-421&rft_id=info%3Apmid%2F19199394&rft_id=info%3Apmid%2F19199394&rft.externalDocID=19199394
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0899-7667&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0899-7667&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0899-7667&client=summon