SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN

The increasing size of deep neural networks (DNNs) raises a high demand for distributed training. An expert could find good hybrid parallelism strategies, but designing suitable strategies is time and labor-consuming. Therefore, automating parallelism strategy generation is crucial and desirable for...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	International journal of parallel programming Jg. 51; H. 2-3; S. 109 - 127
Hauptverfasser:	Wang, Haoran, Tachon, Thibaut, Li, Chong, Robert, Sophie, Limet, Sébastien
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York Springer US 01.06.2023 Springer Nature B.V Springer Verlag
Schlagworte:	Artificial neural networks Communication Computation Computer Science Cost analysis Cost function Homomorphisms Mathematical analysis Parallel processing Portability Processor Architectures Searching Semantics Software Engineering/Programming and Operating Systems Special Issue on High-Level Parallel Programming and Applications (HLPP 2022) Theory of Computation Training Parallel modeling Distributed training Deep network networks Performance analysis Functional transformation Symbolic cost model Distributed training ; Deep network networks; Parallel modeling; Symbolic cost model; Functional transformation; Performance analysis
ISSN:	0885-7458, 1573-7640
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Schreiben Sie den ersten Kommentar!