Accelerating distributed Expectation–Maximization algorithms with frequent updates

Expectation–Maximization (EM) is a popular approach for parameter estimation in many applications, such as image understanding, document classification, and genome data analysis. Despite the popularity of EM algorithms, it is challenging to efficiently implement these algorithms in a distributed env...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of parallel and distributed computing Ročník 111; s. 65 - 75
Hlavní autoři: Yin, Jiangtao, Zhang, Yanfeng, Gao, Lixin
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier Inc 01.01.2018
Témata:
Clustering
Concurrent updates
Distributed framework
Expectation–Maximization
Frequent updates
Topic modeling
Topic modeling
Distributed framework
Clustering
Frequent updates
Concurrent updates
Expectation–Maximization
ISSN:0743-7315, 1096-0848
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Expectation–Maximization (EM) is a popular approach for parameter estimation in many applications, such as image understanding, document classification, and genome data analysis. Despite the popularity of EM algorithms, it is challenging to efficiently implement these algorithms in a distributed environment for handling massive data sets. In particular, many EM algorithms that frequently update the parameters have been shown to be much more efficient than their concurrent counterparts. Accordingly, we propose two approaches to parallelize such EM algorithms in a distributed environment so as to scale to massive data sets. We prove that both approaches maintain the convergence properties of the EM algorithms. Based on the approaches, we design and implement a distributed framework, FreEM, to support the implementation of frequent updates for the EM algorithms. We show its efficiency through two categories of EM applications, clustering and topic modeling. These applications include k-means clustering, fuzzy c-means clustering, parameter estimation for the Gaussian Mixture Model, and variational inference for Latent Dirichlet Allocation. We extensively evaluate our framework on both a cluster of local machines and the Amazon EC2 cloud. Our evaluation shows that the EM algorithms with frequent updates implemented on FreEM can converge much faster than those implementations with traditional concurrent updates. •Propose two approaches to parallelize EM algorithms with frequent updates in a distributed environment to scale to massive data sets.•Prove that both approaches maintain the convergence properties of the EM algorithms.•Design and implement a distributed framework to support the implementation of frequent updates for the EM algorithms.•Extensively evaluate our framework on both a cluster of local machines and the Amazon EC2 cloud with several popular EM algorithms.•The EM algorithms with frequent updates implemented on our framework can converge much faster than traditional implementations.
ISSN:0743-7315
1096-0848
DOI:10.1016/j.jpdc.2017.07.005