A Dynamic Sliding Window Based Tensor Communication Scheduling Framework for Distributed Deep Learning

Simultaneous tensor communication can effectively improve the scalability of distributed deep learning on large clusters. However, a fixed number of tensor blocks communicated concurrently violates the priority-based scheduling strategy and cannot minimize communication overheads. In this paper, we...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on network science and engineering Ročník 12; číslo 2; s. 1080 - 1095
Hlavní autori:	Gao, Yunqi, Hu, Bing, Mashhadi, Mahdi Boloursaz, Wang, Wei, Tafazolli, Rahim, Debbah, Merouane
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Piscataway IEEE 01.03.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Artificial neural networks Clusters Communication communication scheduling Computational modeling Computer architecture data parallelism Deep learning Distributed deep learning Dynamic scheduling Energy consumption generative pre-trained transformer (GPT) Greedy algorithms Mathematical models Parallel processing Priority scheduling Processor scheduling Sliding tensor partitioning Tensors Training
ISSN:	2327-4697, 2334-329X
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Buďte prvý, kto okomentuje tento záznam!