MAD-Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems

Training and deploying large-scale machine learning models is time-consuming, requires significant distributed computing infrastructures, and incurs high operational costs. Our analysis, grounded in real-world large model training on datacenter-scale infrastructures, reveals that 14~32% of all GPU h...

Celý popis

Podrobná bibliografia

Vydané v:	2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) s. 818 - 833
Hlavní autori:	Hsia, Samuel, Golden, Alicia, Acun, Bilge, Ardalani, Newsha, DeVito, Zachary, Wei, Gu-Yeon, Brooks, David, Wu, Carole-Jean
Médium:	Konferenčný príspevok..
Jazyk:	English
Vydavateľské údaje:	IEEE 29.06.2024
Predmet:	Analytical models Computational modeling Computer architecture Costs Data Center Distributed Inference Distributed Training GPU Graphics processing units Hardware-Software Co-Design Machine learning Parallelization Performance Model Simulator Training
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!