A Comparison of New and Old Algorithms for a Mixture Estimation Problem
We investigate the problem of estimating the proportion vector which maximizes the likelihood of a given sample for a mixture of given densities. We adapt a framework developed for supervised learning and give simple derivations for many of the standard iterative algorithms like gradient projection...
Uložené v:
| Vydané v: | Machine learning Ročník 27; číslo 1; s. 97 - 119 |
|---|---|
| Hlavní autori: | , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Dordrecht
Springer Nature B.V
1997
|
| Predmet: | |
| ISSN: | 0885-6125, 1573-0565 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Shrnutí: | We investigate the problem of estimating the proportion vector which maximizes the likelihood of a given sample for a mixture of given densities. We adapt a framework developed for supervised learning and give simple derivations for many of the standard iterative algorithms like gradient projection and EM. In this framework, the distance between the new and old proportion vectors is used as a penalty term. The square distance leads to the gradient projection update, and the relative entropy to a new update which we call the exponentiated gradient update (EG^sub ^). Curiously, when a second order Taylor expansion of the relative entropy is used, we arrive at an update EM^sub ^ which, for =1, gives the usual EM update. Experimentally, both the EM^sub ^-update and the EG^sub ^-update for > 1 outperform the EM algorithm and its variants. We also prove a polynomial bound on the rate of convergence of the EG^sub ^ algorithm.[PUBLICATION ABSTRACT] |
|---|---|
| Bibliografia: | SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-2 content type line 23 |
| ISSN: | 0885-6125 1573-0565 |
| DOI: | 10.1023/A:1007301011561 |