2D-THA-ADMM: communication efficient distributed ADMM algorithm framework based on two-dimensional torus hierarchical AllReduce

Model synchronization refers to the communication process involved in large-scale distributed machine learning tasks. As the cluster scales up, the synchronization of model parameters becomes a challenging task that has to be coordinated among thousands of workers. Firstly, this study proposes a h i...

Full description

Bibliographic Details

Published in:	International journal of machine learning and cybernetics Vol. 15; no. 2; pp. 207 - 226
Main Authors:	Wang, Guozheng, Lei, Yongmei, Zhang, Zeyu, Peng, Cunlu
Format:	Journal Article
Language:	English
Published:	Berlin/Heidelberg Springer Berlin Heidelberg 01.02.2024
Subjects:	Artificial Intelligence Complex Systems Computational Intelligence Control Engineering Mechatronics Original Article Pattern Recognition Robotics Systems Biology Hierarchical AllReduce Two-dimensional torus Synchronization algorithm ADMM
ISSN:	1868-8071, 1868-808X
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!