Multi-consensus decentralized primal-dual fixed point algorithm for distributed learning

Decentralized distributed learning has recently attracted significant attention in many applications in machine learning and signal processing. To solve a decentralized optimization with regularization, we propose a Multi-consensus Decentralized Primal-Dual Fixed Point (MD-PDFP) algorithm. We apply...

Full description

Saved in:
Bibliographic Details
Published in:Machine learning Vol. 113; no. 7; pp. 4315 - 4357
Main Authors: Tang, Kejie, Liu, Weidong, Mao, Xiaojun
Format: Journal Article
Language:English
Published: New York Springer US 01.07.2024
Springer Nature B.V
Subjects:
Algorithms
Artificial Intelligence
Computer Science
Control
Convex analysis
Convexity
Machine Learning
Mechatronics
Natural Language Processing (NLP)
Optimization
Regularization
Robotics
Simulation and Modeling
Gradient tracking
Proximal gradient
Decentralized optimization
Fixed point
ISSN:0885-6125, 1573-0565
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Decentralized distributed learning has recently attracted significant attention in many applications in machine learning and signal processing. To solve a decentralized optimization with regularization, we propose a Multi-consensus Decentralized Primal-Dual Fixed Point (MD-PDFP) algorithm. We apply multiple consensus steps with the gradient tracking technique to extend the primal-dual fixed point method over a network. The communication complexities of our procedure are given under certain conditions. Moreover, we show that our algorithm is consistent under general conditions and enjoys global linear convergence under strong convexity. With some particular choices of regularizations, our algorithm can be applied to decentralized machine learning applications. Finally, several numerical experiments and real data analyses are conducted to demonstrate the effectiveness of the proposed algorithm.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-024-06537-8