Distributed accelerated gradient methods with restart under quadratic growth condition

We consider solving convex problems satisfying quadratic growth condition (QGC) over a distributed setting with no central server. Such problems are popular in distributed machine learning applications. When QGC growth parameter c is known, we propose distributed accelerated gradient methods with re...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of global optimization Jg. 90; H. 1; S. 153 - 215
Hauptverfasser: Sharma, Chhavi, Narayanan, Vishnu, Balamurugan, P.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.09.2024
Springer
Springer Nature B.V
Schlagworte:
Algorithms
Analysis
Approximation
Communication
Computer Science
Constraints
Design parameters
Machine learning
Mathematics
Mathematics and Statistics
Methods
Operations Research/Decision Theory
Optimization
Parameter estimation
Real Functions
Topology
Distributed optimization
Nesterov accelerated gradient method
Unconstrained and constrained convex optimization
Quadratic growth condition
ISSN:0925-5001, 1573-2916
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We consider solving convex problems satisfying quadratic growth condition (QGC) over a distributed setting with no central server. Such problems are popular in distributed machine learning applications. When QGC growth parameter c is known, we propose distributed accelerated gradient methods with restarts, named PDACA and DACA respectively for constrained and unconstrained settings. In practical problems when c is unavailable, we design mPDACA and mDACA methods respectively for constrained and unconstrained settings, where novel distributed mechanisms are proposed to update the estimates of growth parameter c using only local quantities depending on local proximal operators or local gradients. We further derive theoretical guarantees and gradient computation and communication complexities for all four proposed algorithms. Extensive numerical experiments on logistic regression on different communication topologies showcase the utility of our algorithms in comparison with baseline methods.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0925-5001
1573-2916
DOI:10.1007/s10898-024-01395-z