Reinforcement Learning initialization by evolutionary formulation: Application for workflow autoscaling in the Cloud

Scientific workflow execution is usually fulfilled through Cloud Computing, but correct autoscaling techniques are needed for proper performance. Reinforcement Learning (RL) has been used for autoscaling, but presents low performance in early stages. Poor initial performance accumulates over episode...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Engineering applications of artificial intelligence Ročník 162; s. 112663
Hlavní autoři: Robino, Luciano, Garí, Yisel, Pacini, Elina, Mateos, Cristian, Yannibelli, Virginia, Monge, David A.
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier Ltd 24.12.2025
Témata:
Autoscaling
Cloud computing
Improved Decomposition-Based Evolutionary Algorithm
Non-dominated Sorting Genetic Algorithm III
Q-Learning
Workflow
Cloud computing
Workflow
Q-Learning
Improved Decomposition-Based Evolutionary Algorithm
Non-dominated Sorting Genetic Algorithm III
Autoscaling
ISSN:0952-1976
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Scientific workflow execution is usually fulfilled through Cloud Computing, but correct autoscaling techniques are needed for proper performance. Reinforcement Learning (RL) has been used for autoscaling, but presents low performance in early stages. Poor initial performance accumulates over episodes, making the learning process more expensive, which is critical in the context of Cloud autoscaling. Solutions to this problem are sparse and difficult to generalize. Here, we present Reinforcement Learning Initialization by Evolutionary Formulation (ReLIEF), which uses evolutionary algorithm to generate an initial pre-optimized RL policy, that is later refined via RL. Proposed initilization aims to reduce the accumulated losses in monetary cost and execution time (i.e. makespan) during learning. In this article two prominent evolutionary algorithm are used: Non-dominated Sorting Genetic Algorithm III (NSGA-III) and Improved Decomposition-Based Evolutionary Algorithm (I-DBEA). On the other hand, for Reinforcement Learning only Q-Learning in tabular form is used. Four benchmark workflows were used to validate savings produced by the proposal. In 3 out of 4 workflows analyzed, ReLIEF outperformed baseline RL agents. In the remaining workflow, competitive performance was obtained. [Display omitted] •ReLIEF, an evolutionary algorithm to build preoptimized RL policies for autoscaling.•RL-based autoscaler performance is improved during online learning process.•ReLIEF is tested on 4 workflows, comparing RL metrics and EA variants for insights.
ISSN:0952-1976
DOI:10.1016/j.engappai.2025.112663