APapo: An asynchronous parallel optimization method for DNN models

To address the challenges related to segmentation complexity, high memory usage, extended training duration, and low equipment utilization in parallel optimization of large-scale deep neural network (DNN) models, this paper proposes an asynchronous parallel optimization method APapo. Firstly, a mult...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	Future generation computer systems Jg. 152; S. 317 - 330
Hauptverfasser:	Liu, Shuai, Ju, Tao
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Elsevier B.V 01.03.2024
Schlagworte:	Asynchronous pipeline parallelism Augmented antichain Computation–communication overlap DNN model parallelism Model segmentation Model segmentation Augmented antichain Computation–communication overlap DNN model parallelism Asynchronous pipeline parallelism
ISSN:	0167-739X, 1872-7115
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!