A Stochastic Approach for Scheduling AI Training Jobs in GPU-based Systems

In this work, we optimize the scheduling of Deep Learning (DL) training jobs from the perspective of a Cloud Service Provider running a data center, which efficiently selects resources for the execution of each job to minimize the average energy consumption while satisfying time constraints. To mode...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on cloud computing Ročník 12; číslo 1; s. 1 - 17
Hlavní autoři: Filippini, Federica, Anselmi, Jonatha, Ardagna, Danilo, Gaujal, Bruno
Médium: Journal Article
Jazyk:angličtina
Vydáno: Piscataway IEEE 01.01.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Average energy consumption minimization
Cloud computing
Computer Science
Constraint modelling
Costs
Deep Learning
Energy consumption
Energy costs
GPU cluster
GPU sharing
Graphics processing units
Job Tardiness
Linear programming
Mixed integer
Nonlinear programming
Optimal scheduling
Optimization
Scheduling
Stochastic processes
Training
Job Tardiness
consumption minimization
GPU sharing
Scheduling
GPU cluster
Deep Learning
Average energy
ISSN:2168-7161, 2372-0018
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this work, we optimize the scheduling of Deep Learning (DL) training jobs from the perspective of a Cloud Service Provider running a data center, which efficiently selects resources for the execution of each job to minimize the average energy consumption while satisfying time constraints. To model the problem, we first develop a Mixed-Integer Non-Linear Programming formulation. Unfortunately, the computation of an optimal solution is prohibitively expensive, and to overcome this difficulty, we design a heuristic STochastic Scheduler (STS). Exploiting the probability distribution of early termination, STS determines how to adapt the resource assignment during the execution of the jobs to minimize the expected energy cost while meeting the job due dates. The results of an extensive experimental evaluation show that STS guarantees significantly better results than other methods in the literature, effectively avoiding due date violations and yielding a percentage total cost reduction between 32% and 80% on average. We also prove the applicability of our method in real-world scenarios, as obtaining optimal schedules for systems of up to 100 nodes and 400 concurrent jobs requires less than 5 seconds. Finally, we evaluated the effectiveness of GPU sharing, i.e., running multiple jobs in a single GPU. The obtained results demonstrate that depending on the workload and GPU memory, this further reduces the energy cost by 17-29% on average.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2168-7161
2372-0018
DOI:10.1109/TCC.2023.3336540