SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN

The increasing size of deep neural networks (DNNs) raises a high demand for distributed training. An expert could find good hybrid parallelism strategies, but designing suitable strategies is time and labor-consuming. Therefore, automating parallelism strategy generation is crucial and desirable for...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	International journal of parallel programming Ročník 51; číslo 2-3; s. 109 - 127
Hlavní autori:	Wang, Haoran, Tachon, Thibaut, Li, Chong, Robert, Sophie, Limet, Sébastien
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York Springer US 01.06.2023 Springer Nature B.V Springer Verlag
Predmet:	Artificial neural networks Communication Computation Computer Science Cost analysis Cost function Homomorphisms Mathematical analysis Parallel processing Portability Processor Architectures Searching Semantics Software Engineering/Programming and Operating Systems Special Issue on High-Level Parallel Programming and Applications (HLPP 2022) Theory of Computation Training Parallel modeling Distributed training Deep network networks Performance analysis Functional transformation Symbolic cost model Distributed training ; Deep network networks; Parallel modeling; Symbolic cost model; Functional transformation; Performance analysis
ISSN:	0885-7458, 1573-7640
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Buďte prvý, kto okomentuje tento záznam!