Nonlinear Perturbation-Based Non-Convex Optimization Over Time-Varying Networks

Decentralized optimization strategies are helpful for various applications, from networked estimation to distributed machine learning. This paper studies finite-sum minimization problems described over a network of nodes and proposes a computationally efficient algorithm that solves distributed conv...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on network science and engineering Vol. 11; no. 6; pp. 6461 - 6469
Main Authors: Doostmohammadian, Mohammadreza, Gabidullina, Zulfiya R., Rabiee, Hamid R.
Format: Journal Article
Language:English
Published: Piscataway IEEE 01.11.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithms
Convergence
Convexity
Cost function
Costs
Data exchange
Data transmission
Distributed algorithm
Linear programming
locally non-convex optimization
Machine learning
Network analysis
network science
Network topology
Nonlinear dynamics
Nonlinearity
Optimization
Packet switching
Perturbation
perturbation theory
Quantization (signal)
Switches
ISSN:2327-4697, 2334-329X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Decentralized optimization strategies are helpful for various applications, from networked estimation to distributed machine learning. This paper studies finite-sum minimization problems described over a network of nodes and proposes a computationally efficient algorithm that solves distributed convex problems and optimally finds the solution to locally non-convex objective functions. In contrast to batch gradient optimization in some literature, our algorithm is on a single-time scale with no extra inner consensus loop. It evaluates one gradient entry per node per time. Further, the algorithm addresses link-level nonlinearity representing, for example, logarithmic quantization of the exchanged data or clipping of the exchanged data bits. Leveraging perturbation-based theory and algebraic Laplacian network analysis proves optimal convergence and dynamics stability over time-varying and switching networks. The time-varying network setup might be due to packet drops or link failures. Despite the nonlinear nature of the dynamics, we prove exact convergence in the face of odd sign-preserving sector-bound nonlinear data transmission over the links. Illustrative numerical simulations further highlight our contributions.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2327-4697
2334-329X
DOI:10.1109/TNSE.2024.3439744