A Stochastic Approach for Scheduling AI Training Jobs in GPU-based Systems

In this work, we optimize the scheduling of Deep Learning (DL) training jobs from the perspective of a Cloud Service Provider running a data center, which efficiently selects resources for the execution of each job to minimize the average energy consumption while satisfying time constraints. To mode...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on cloud computing Jg. 12; H. 1; S. 1 - 17
Hauptverfasser: Filippini, Federica, Anselmi, Jonatha, Ardagna, Danilo, Gaujal, Bruno
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Piscataway IEEE 01.01.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Average energy consumption minimization
Cloud computing
Computer Science
Constraint modelling
Costs
Deep Learning
Energy consumption
Energy costs
GPU cluster
GPU sharing
Graphics processing units
Job Tardiness
Linear programming
Mixed integer
Nonlinear programming
Optimal scheduling
Optimization
Scheduling
Stochastic processes
Training
Job Tardiness
consumption minimization
GPU sharing
Scheduling
GPU cluster
Deep Learning
Average energy
ISSN:2168-7161, 2372-0018
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this work, we optimize the scheduling of Deep Learning (DL) training jobs from the perspective of a Cloud Service Provider running a data center, which efficiently selects resources for the execution of each job to minimize the average energy consumption while satisfying time constraints. To model the problem, we first develop a Mixed-Integer Non-Linear Programming formulation. Unfortunately, the computation of an optimal solution is prohibitively expensive, and to overcome this difficulty, we design a heuristic STochastic Scheduler (STS). Exploiting the probability distribution of early termination, STS determines how to adapt the resource assignment during the execution of the jobs to minimize the expected energy cost while meeting the job due dates. The results of an extensive experimental evaluation show that STS guarantees significantly better results than other methods in the literature, effectively avoiding due date violations and yielding a percentage total cost reduction between 32% and 80% on average. We also prove the applicability of our method in real-world scenarios, as obtaining optimal schedules for systems of up to 100 nodes and 400 concurrent jobs requires less than 5 seconds. Finally, we evaluated the effectiveness of GPU sharing, i.e., running multiple jobs in a single GPU. The obtained results demonstrate that depending on the workload and GPU memory, this further reduces the energy cost by 17-29% on average.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2168-7161
2372-0018
DOI:10.1109/TCC.2023.3336540