Decentralized Dual Proximal Gradient Algorithms for Non-Smooth Constrained Composite Optimization Problems

Decentralized dual methods play significant roles in large-scale optimization, which effectively resolve many constrained optimization problems in machine learning and power systems. In this article, we focus on studying a class of totally non-smooth constrained composite optimization problems over...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on parallel and distributed systems Vol. 32; no. 10; pp. 2594 - 2605
Main Authors: Li, Huaqing, Hu, Jinhui, Ran, Liang, Wang, Zheng, Lu, Qingguo, Du, Zhenyuan, Huang, Tingwen
Format: Journal Article
Language:English
Published: New York IEEE 01.10.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithms
Convex optimization
decentralized machine learning
DERC problems
DSLR problems
Energy sources
Iterative methods
Linear programming
Machine learning
Machine learning algorithms
Multi-agent systems
Multiagent systems
non-smooth constrained composite optimization problems
Optimization
Power systems
Signal processing algorithms
synchronous and asynchronous decentralized algorithms
ISSN:1045-9219, 1558-2183
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Decentralized dual methods play significant roles in large-scale optimization, which effectively resolve many constrained optimization problems in machine learning and power systems. In this article, we focus on studying a class of totally non-smooth constrained composite optimization problems over multi-agent systems, where the mutual goal of agents in the system is to optimize a sum of two separable non-smooth functions consisting of a strongly-convex function and another convex (not necessarily strongly-convex) function. Agents in the system conduct parallel local computation and communication in the overall process without leaking their private information. In order to resolve the totally non-smooth constrained composite optimization problem in a fully decentralized manner, we devise a synchronous decentralized dual proximal (SynDe-DuPro) gradient algorithm and its asynchronous version (AsynDe-DuPro) based on the randomized block-coordinate method. Both SynDe-DuPro and AsynDe-DuPro algorithms are theoretically proved to achieve the globally optimal solution to the totally non-smooth constrained composite optimization problem relied on the quasi-Fejér monotone theorem. As a main result, AsynDe-DuPro algorithm attains the globally optimal solution without requiring all agents to be activated at each iteration and thus is more robust than most existing synchronous algorithms. The practicability of the proposed algorithms and correctness of the theoretical findings are demonstrated by the experiments on a constrained Decentralized Sparse Logistic Regression (DSLR) problem in machine learning and a Decentralized Energy Resources Coordination (DERC) problem in power systems.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1045-9219
1558-2183
DOI:10.1109/TPDS.2021.3072373