An Asynchronous Distributed Expectation Maximization Algorithm for Massive Data: The DEM Algorithm

The family of expectation--maximization (EM) algorithms provides a general approach to fitting flexible models for large and complex data. The expectation (E) step of EM-type algorithms is time-consuming in massive data applications because it requires multiple passes through the full data. We addre...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of computational and graphical statistics Ročník 28; číslo 2; s. 233 - 243
Hlavní autoři: Srivastava, Sanvesh, DePalma, Glen, Liu, Chuanhai
Médium: Journal Article
Jazyk:angličtina
Vydáno: Alexandria Taylor & Francis 03.04.2019
American Statistical Association, the Institute of Mathematical Statistics, and the Interface Foundation of North America
Taylor & Francis Ltd
Témata:
Algorithms
Computational grids
Computer simulation
Convergence
Digital Elevation Models
Divide-and-conquer
EM-type algorithm
Iterative computations
Large and complex data
Linear mixed-effects model
Maximization
Message passing interface (MPI)
Optimization
Parameter estimation
Ratings
Workers
ISSN:1061-8600, 1537-2715
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The family of expectation--maximization (EM) algorithms provides a general approach to fitting flexible models for large and complex data. The expectation (E) step of EM-type algorithms is time-consuming in massive data applications because it requires multiple passes through the full data. We address this problem by proposing an asynchronous and distributed generalization of the EM called the distributed EM (DEM). Using DEM, existing EM-type algorithms are easily extended to massive data settings by exploiting the divide-and-conquer technique and widely available computing power, such as grid computing. The DEM algorithm reserves two groups of computing processes called workers and managers for performing the E step and the maximization step (M step), respectively. The samples are randomly partitioned into a large number of disjoint subsets and are stored on the worker processes. The E step of DEM algorithm is performed in parallel on all the workers, and every worker communicates its results to the managers at the end of local E step. The managers perform the M step after they have received results from a γ-fraction of the workers, where γ is a fixed constant in (0, 1]. The sequence of parameter estimates generated by the DEM algorithm retains the attractive properties of EM: convergence of the sequence of parameter estimates to a local mode and linear global rate of convergence. Across diverse simulations focused on linear mixed-effects models, the DEM algorithm is significantly faster than competing EM-type algorithms while having a similar accuracy. The DEM algorithm maintains its superior empirical performance on a movie ratings database consisting of 10 million ratings. Supplementary material for this article is available online.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1061-8600
1537-2715
DOI:10.1080/10618600.2018.1497512