SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN

The increasing size of deep neural networks (DNNs) raises a high demand for distributed training. An expert could find good hybrid parallelism strategies, but designing suitable strategies is time and labor-consuming. Therefore, automating parallelism strategy generation is crucial and desirable for...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	International journal of parallel programming Ročník 51; číslo 2-3; s. 109 - 127
Hlavní autoři:	Wang, Haoran, Tachon, Thibaut, Li, Chong, Robert, Sophie, Limet, Sébastien
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York Springer US 01.06.2023 Springer Nature B.V Springer Verlag
Témata:	Artificial neural networks Communication Computation Computer Science Cost analysis Cost function Homomorphisms Mathematical analysis Parallel processing Portability Processor Architectures Searching Semantics Software Engineering/Programming and Operating Systems Special Issue on High-Level Parallel Programming and Applications (HLPP 2022) Theory of Computation Training Parallel modeling Distributed training Deep network networks Performance analysis Functional transformation Symbolic cost model Distributed training ; Deep network networks; Parallel modeling; Symbolic cost model; Functional transformation; Performance analysis
ISSN:	0885-7458, 1573-7640
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Buďte první, kdo okomentuje tento záznam!