Asynchronous Parallel Nonconvex Optimization Under the Polyak-Łojasiewicz Condition

Communication delays and synchronization are major bottlenecks for parallel computing, and tolerating asynchrony is therefore crucial for accelerating parallel computation. Motivated by optimization problems that do not satisfy convexity assumptions, we present an asynchronous block coordinate desce...

Full description

Saved in:
Bibliographic Details
Published in:IEEE control systems letters Vol. 6; pp. 524 - 529
Main Authors: Yazdani, Kasra, Hale, Matthew
Format: Journal Article
Language:English
Published: IEEE 2022
Subjects:
asynchronous optimization algorithms
Convergence
Delays
Linear programming
Machine learning algorithms
multi-agent systems
nonconvex optimization
Optimization
Parallel computation
Program processors
Signal processing algorithms
ISSN:2475-1456, 2475-1456
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Communication delays and synchronization are major bottlenecks for parallel computing, and tolerating asynchrony is therefore crucial for accelerating parallel computation. Motivated by optimization problems that do not satisfy convexity assumptions, we present an asynchronous block coordinate descent algorithm for nonconvex optimization problems whose objective functions satisfy the Polyak-Łojasiewicz condition. This condition is a generalization of strong convexity to nonconvex problems and requires neither convexity nor uniqueness of minimizers. Under only assumptions of mild smoothness of objective functions and bounded delays, we prove that a linear convergence rate is obtained. Numerical experiments for logistic regression problems are presented to illustrate the impact of asynchrony upon convergence.
ISSN:2475-1456
2475-1456
DOI:10.1109/LCSYS.2021.3082800