A Dynamic Sliding Window Based Tensor Communication Scheduling Framework for Distributed Deep Learning

Simultaneous tensor communication can effectively improve the scalability of distributed deep learning on large clusters. However, a fixed number of tensor blocks communicated concurrently violates the priority-based scheduling strategy and cannot minimize communication overheads. In this paper, we...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on network science and engineering Ročník 12; číslo 2; s. 1080 - 1095
Hlavní autoři:	Gao, Yunqi, Hu, Bing, Mashhadi, Mahdi Boloursaz, Wang, Wei, Tafazolli, Rahim, Debbah, Merouane
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Piscataway IEEE 01.03.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Artificial neural networks Clusters Communication communication scheduling Computational modeling Computer architecture data parallelism Deep learning Distributed deep learning Dynamic scheduling Energy consumption generative pre-trained transformer (GPT) Greedy algorithms Mathematical models Parallel processing Priority scheduling Processor scheduling Sliding tensor partitioning Tensors Training
ISSN:	2327-4697, 2334-329X
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Buďte první, kdo okomentuje tento záznam!