Cross-Search With Improved Multi-Dimensional Dichotomy-Based Joint Optimization for Distributed Parallel Training of DNN

Distributed parallel training of large-scale deep neural networks (DNN) has attracted the attentions of both artificial intelligence and high-performance distributed computing. One of efficient approaches is the micro-batch-based pipeline parallelism (MBPP), e.g., GPipe and Terapipe. Based on the MB...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on parallel and distributed systems Ročník 36; číslo 8; s. 1680 - 1694
Hlavní autoři:	Zhou, Guangyao, Fu, Yiqin, Lan, Haocheng, Xie, Yuanlun, Tian, Wenhong, Buyya, Rajkumar, Qian, Jiahong, Su, Teng
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	IEEE 01.08.2025
Témata:	Computational modeling Costs cross-search Data models Distributed parallelism improved multi-dimensional dichotomy large DNN Mathematical models micro-batch pipeline Optimization Parallel processing Partitioning algorithms Pipelines Training transformer Transformers
ISSN:	1045-9219, 1558-2183
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Buďte první, kdo okomentuje tento záznam!