MDLoader: A Hybrid Model-Driven Data Loader for Distributed Graph Neural Network Training

Scalable data management is essential for processing large scientific dataset on HPC platforms for distributed deep learning. In-memory distributed storage is preferred for its speed, enabling rapid, random, and frequent data access required by stochastic optimizers. Processes use one-sided or colle...

Celý popis

Podrobná bibliografia

Vydané v:	SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis s. 1046 - 1057
Hlavní autori:	Bae, Jonghyun, Choi, Jong Youl, Pasini, Massimiliano Lupo, Mehta, Kshitij, Zhang, Pei, Ibrahim, Khaled Z.
Médium:	Konferenčný príspevok..
Jazyk:	English
Vydavateľské údaje:	IEEE 17.11.2024
Predmet:	Data models Distributed databases Graph Neural Network Graph neural networks Libraries Load modeling Loading Low latency communication MPI communication Multiprocessor interconnection Performance estimator Stochastic processes Training
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!