GR-ADMM: A Communication Efficient Algorithm Based on ADMM

In recent work, the decentralized algorithm has received more attention. In the centralized network, the worker nodes need to communicate with the central nodes, which results in the growth of communication traffic with the network expansion. Based on the purpose of reducing the communication costs...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom) s. 220 - 227
Hlavní autoři: Huang, Xin, Wang, Guozheng, Lei, Yongmei
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.09.2021
Témata:
ADMM
Clustering algorithms
Communication effectiveness
Communication-efficient decentralized algorithm
Computer architecture
Costs
GR-ADMM
grouping strategy
Mobile communication
Ring All-Reduce
Scalability
Wireless communication
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In recent work, the decentralized algorithm has received more attention. In the centralized network, the worker nodes need to communicate with the central nodes, which results in the growth of communication traffic with the network expansion. Based on the purpose of reducing the communication costs in the distributed system, we proposed a decentralized algorithm based on ADMM - Grouping Ring All-Reduce ADMM (GR-ADMM) in this paper. First, GR-ADMM adopts decentralized architecture to avoid the problem of communication bottleneck in the central network. Second, to ensure the scalability of the distributed system, GR-ADMM introduces the Ring All-Reduce to the ADMM. Ring All-Reduce architecture has the advantage of its constant communication overhead. However, its performance is bounded by the stragglers (i.e., slow nodes). Third, GR-ADMM adopts the grouping strategy to alleviate the problem of stragglers. Experiments show that our algorithm has better convergence performance than QSGD and GADMM, especially in massive clusters. Compared with GADMM's, the overall communication cost of GR-ADMM is reduced by 72%.
DOI:10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00041