MAD-Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems

Training and deploying large-scale machine learning models is time-consuming, requires significant distributed computing infrastructures, and incurs high operational costs. Our analysis, grounded in real-world large model training on datacenter-scale infrastructures, reveals that 14~32% of all GPU h...

Celý popis

Podrobná bibliografie

Vydáno v:	2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) s. 818 - 833
Hlavní autoři:	Hsia, Samuel, Golden, Alicia, Acun, Bilge, Ardalani, Newsha, DeVito, Zachary, Wei, Gu-Yeon, Brooks, David, Wu, Carole-Jean
Médium:	Konferenční příspěvek
Jazyk:	angličtina
Vydáno:	IEEE 29.06.2024
Témata:	Analytical models Computational modeling Computer architecture Costs Data Center Distributed Inference Distributed Training GPU Graphics processing units Hardware-Software Co-Design Machine learning Parallelization Performance Model Simulator Training
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!