The stochastic proximal distance algorithm

Stochastic versions of proximal methods have gained much attention in statistics and machine learning. These algorithms tend to admit simple, scalable forms, and enjoy numerical stability via implicit updates. In this work, we propose and analyze a stochastic version of the recently proposed proxima...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Statistics and computing Ročník 34; číslo 6
Hlavní autoři: Jiang, Haoyu, Xu, Jason
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.12.2024
Springer Nature B.V
Témata:
Algorithms
Artificial Intelligence
Cognitive tasks
Computer Science
Convergence
Error analysis
Machine learning
Numerical stability
Original Paper
Parameter estimation
Probability and Statistics in Computer Science
Statistical methods
Statistical Theory and Methods
Statistics and Computing/Statistics Programs
Majorization-minimization
Constrained estimation
Stochastic optimization
Proximal methods
Distance penalization
ISSN:0960-3174, 1573-1375
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Stochastic versions of proximal methods have gained much attention in statistics and machine learning. These algorithms tend to admit simple, scalable forms, and enjoy numerical stability via implicit updates. In this work, we propose and analyze a stochastic version of the recently proposed proximal distance algorithm, a class of iterative optimization methods that recover a desired constrained estimation problem as a penalty parameter ρ → ∞ . By uncovering connections to related stochastic proximal methods and interpreting the penalty parameter as the learning rate, we justify heuristics used in practical manifestations of the proximal distance method, establishing their convergence guarantees for the first time. Moreover, we extend recent theoretical devices to establish finite error bounds and a complete characterization of convergence rates regimes. We validate our analysis via a thorough empirical study, also showing that unsurprisingly, the proposed method outpaces batch versions on popular learning tasks.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0960-3174
1573-1375
DOI:10.1007/s11222-024-10524-3