Reinforcement Learning initialization by evolutionary formulation: Application for workflow autoscaling in the Cloud

Scientific workflow execution is usually fulfilled through Cloud Computing, but correct autoscaling techniques are needed for proper performance. Reinforcement Learning (RL) has been used for autoscaling, but presents low performance in early stages. Poor initial performance accumulates over episode...

Full description

Saved in:
Bibliographic Details
Published in:Engineering applications of artificial intelligence Vol. 162; p. 112663
Main Authors: Robino, Luciano, Garí, Yisel, Pacini, Elina, Mateos, Cristian, Yannibelli, Virginia, Monge, David A.
Format: Journal Article
Language:English
Published: Elsevier Ltd 24.12.2025
Subjects:
Autoscaling
Cloud computing
Improved Decomposition-Based Evolutionary Algorithm
Non-dominated Sorting Genetic Algorithm III
Q-Learning
Workflow
Cloud computing
Workflow
Q-Learning
Improved Decomposition-Based Evolutionary Algorithm
Non-dominated Sorting Genetic Algorithm III
Autoscaling
ISSN:0952-1976
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Scientific workflow execution is usually fulfilled through Cloud Computing, but correct autoscaling techniques are needed for proper performance. Reinforcement Learning (RL) has been used for autoscaling, but presents low performance in early stages. Poor initial performance accumulates over episodes, making the learning process more expensive, which is critical in the context of Cloud autoscaling. Solutions to this problem are sparse and difficult to generalize. Here, we present Reinforcement Learning Initialization by Evolutionary Formulation (ReLIEF), which uses evolutionary algorithm to generate an initial pre-optimized RL policy, that is later refined via RL. Proposed initilization aims to reduce the accumulated losses in monetary cost and execution time (i.e. makespan) during learning. In this article two prominent evolutionary algorithm are used: Non-dominated Sorting Genetic Algorithm III (NSGA-III) and Improved Decomposition-Based Evolutionary Algorithm (I-DBEA). On the other hand, for Reinforcement Learning only Q-Learning in tabular form is used. Four benchmark workflows were used to validate savings produced by the proposal. In 3 out of 4 workflows analyzed, ReLIEF outperformed baseline RL agents. In the remaining workflow, competitive performance was obtained. [Display omitted] •ReLIEF, an evolutionary algorithm to build preoptimized RL policies for autoscaling.•RL-based autoscaler performance is improved during online learning process.•ReLIEF is tested on 4 workflows, comparing RL metrics and EA variants for insights.
ISSN:0952-1976
DOI:10.1016/j.engappai.2025.112663