Reinforcement Learning initialization by evolutionary formulation: Application for workflow autoscaling in the Cloud
Scientific workflow execution is usually fulfilled through Cloud Computing, but correct autoscaling techniques are needed for proper performance. Reinforcement Learning (RL) has been used for autoscaling, but presents low performance in early stages. Poor initial performance accumulates over episode...
Saved in:
| Published in: | Engineering applications of artificial intelligence Vol. 162; p. 112663 |
|---|---|
| Main Authors: | , , , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Elsevier Ltd
24.12.2025
|
| Subjects: | |
| ISSN: | 0952-1976 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Scientific workflow execution is usually fulfilled through Cloud Computing, but correct autoscaling techniques are needed for proper performance. Reinforcement Learning (RL) has been used for autoscaling, but presents low performance in early stages. Poor initial performance accumulates over episodes, making the learning process more expensive, which is critical in the context of Cloud autoscaling. Solutions to this problem are sparse and difficult to generalize. Here, we present Reinforcement Learning Initialization by Evolutionary Formulation (ReLIEF), which uses evolutionary algorithm to generate an initial pre-optimized RL policy, that is later refined via RL. Proposed initilization aims to reduce the accumulated losses in monetary cost and execution time (i.e. makespan) during learning. In this article two prominent evolutionary algorithm are used: Non-dominated Sorting Genetic Algorithm III (NSGA-III) and Improved Decomposition-Based Evolutionary Algorithm (I-DBEA). On the other hand, for Reinforcement Learning only Q-Learning in tabular form is used. Four benchmark workflows were used to validate savings produced by the proposal. In 3 out of 4 workflows analyzed, ReLIEF outperformed baseline RL agents. In the remaining workflow, competitive performance was obtained.
[Display omitted]
•ReLIEF, an evolutionary algorithm to build preoptimized RL policies for autoscaling.•RL-based autoscaler performance is improved during online learning process.•ReLIEF is tested on 4 workflows, comparing RL metrics and EA variants for insights. |
|---|---|
| AbstractList | Scientific workflow execution is usually fulfilled through Cloud Computing, but correct autoscaling techniques are needed for proper performance. Reinforcement Learning (RL) has been used for autoscaling, but presents low performance in early stages. Poor initial performance accumulates over episodes, making the learning process more expensive, which is critical in the context of Cloud autoscaling. Solutions to this problem are sparse and difficult to generalize. Here, we present Reinforcement Learning Initialization by Evolutionary Formulation (ReLIEF), which uses evolutionary algorithm to generate an initial pre-optimized RL policy, that is later refined via RL. Proposed initilization aims to reduce the accumulated losses in monetary cost and execution time (i.e. makespan) during learning. In this article two prominent evolutionary algorithm are used: Non-dominated Sorting Genetic Algorithm III (NSGA-III) and Improved Decomposition-Based Evolutionary Algorithm (I-DBEA). On the other hand, for Reinforcement Learning only Q-Learning in tabular form is used. Four benchmark workflows were used to validate savings produced by the proposal. In 3 out of 4 workflows analyzed, ReLIEF outperformed baseline RL agents. In the remaining workflow, competitive performance was obtained.
[Display omitted]
•ReLIEF, an evolutionary algorithm to build preoptimized RL policies for autoscaling.•RL-based autoscaler performance is improved during online learning process.•ReLIEF is tested on 4 workflows, comparing RL metrics and EA variants for insights. |
| ArticleNumber | 112663 |
| Author | Yannibelli, Virginia Robino, Luciano Mateos, Cristian Monge, David A. Pacini, Elina Garí, Yisel |
| Author_xml | – sequence: 1 givenname: Luciano orcidid: 0009-0002-2110-6026 surname: Robino fullname: Robino, Luciano email: lrobino@conicet.gov.ar organization: ISISTAN-UNICEN-CONICET, Tandil, Buenos Aires, Argentina – sequence: 2 givenname: Yisel orcidid: 0000-0003-4353-7257 surname: Garí fullname: Garí, Yisel email: ygari@uncu.edu.ar organization: Laboratorio de Sistemas Inteligentes (LABSIN) - Facultad de Ingeniería, UNCuyo, Mendoza, Argentina – sequence: 3 givenname: Elina orcidid: 0000-0003-2882-766X surname: Pacini fullname: Pacini, Elina email: epacini@uncu.edu.ar organization: Laboratorio de Sistemas Inteligentes (LABSIN) - Facultad de Ingeniería, UNCuyo, Mendoza, Argentina – sequence: 4 givenname: Cristian orcidid: 0000-0001-5761-1898 surname: Mateos fullname: Mateos, Cristian email: cristian.mateos@isistan.unicen.edu.ar organization: ISISTAN-UNICEN-CONICET, Tandil, Buenos Aires, Argentina – sequence: 5 givenname: Virginia orcidid: 0000-0001-7854-7610 surname: Yannibelli fullname: Yannibelli, Virginia email: virginia.yannibelli@isistan.unicen.edu.ar organization: ISISTAN-UNICEN-CONICET, Tandil, Buenos Aires, Argentina – sequence: 6 givenname: David A. orcidid: 0000-0001-6444-4610 surname: Monge fullname: Monge, David A. email: dmonge@uncu.edu.ar organization: Universidad Nacional de Cuyo, Mendoza, Argentina |
| BookMark | eNqFkF1LwzAUhnMxwW36FyR_oDVJ23T1yjH8goEgeh3S9GRmZklJ04356-1WvfbqcHjP-3B4ZmjivAOEbihJKaH8dpuC28i2lSZlhBUppYzzbIKmpCpYQquSX6JZ120JIdki51MU38A47YOCHbiI1yCDM26DjTPRSGu-ZTTe4fqIYe9tf1pkOOKhsevtObvDy7a1Ro2HQ4APPnxp6w9Y9tF3aqCcgTh-Al5Z3zdX6EJL28H175yjj8eH99Vzsn59elkt14linMaklFxr1QBv2EKVFVFZyRZMMSg4lZDTRjMGtOKgmKzrmmdUlXVBqkZXOfCszuaIj1wVfNcF0KINZjf8LygRJ19iK_58iZMvMfoaivdjEYbv9gaC6JQBp6AxAVQUjTf_IX4AP8Z_nQ |
| Cites_doi | 10.1016/j.engappai.2021.104288 10.35833/MPCE.2019.000055 10.1016/j.swevo.2018.03.011 10.23919/INM.2017.7987304 10.1504/IJGUC.2019.099667 10.1109/ACCESS.2022.3149955 10.1371/journal.pone.0279438 10.1016/j.ascom.2018.04.002 10.1016/j.nucengdes.2023.112423 10.1016/j.future.2021.09.007 10.1016/j.swevo.2024.101517 10.1002/pmic.201800489 10.1080/17538947.2015.1130087 10.1016/j.future.2024.04.014 10.1007/s11227-020-03364-1 10.1109/TNSM.2022.3210211 10.1109/TEVC.2014.2339823 10.1145/3603532 10.14778/1920841.1920902 10.1109/TEVC.2013.2281535 10.1016/j.jnca.2019.102464 |
| ContentType | Journal Article |
| Copyright | 2025 Elsevier Ltd |
| Copyright_xml | – notice: 2025 Elsevier Ltd |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.engappai.2025.112663 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences Computer Science |
| ExternalDocumentID | 10_1016_j_engappai_2025_112663 S0952197625026946 |
| GroupedDBID | --K --M .DC .~1 0R~ 1B1 1~. 1~5 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN AABNK AAEDT AAEDW AAIKJ AAKOC AALRI AAOAW AAQFI AATTM AAXKI AAXUO AAYFN AAYWO ABBOA ABJNI ABMAC ACDAQ ACGFS ACLOT ACRLP ACVFH ACZNC ADBBV ADCNI ADEZE ADTZH AEBSH AECPX AEIPS AEKER AENEX AEUPX AFJKZ AFPUW AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIGII AIIUN AIKHN AITUG AKBMS AKRWK AKYEP ALMA_UNASSIGNED_HOLDINGS AMRAJ ANKPU AOUOD APXCP AXJTR BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EFKBS EFLBG EO8 EO9 EP2 EP3 F5P FDB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ IHE J1W JJJVA KOM LG9 LY7 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 ROL RPZ SDF SDG SDP SES SEW SPC SPCBC SST SSV SSZ T5K TN5 ~G- ~HD 29G 9DU AAQXK AAYXX ABWVN ABXDB ACNNM ACRPL ADJOM ADMUD ADNMO AGQPQ ASPBG AVWKF AZFZN CITATION EJD FEDTE FGOYB G-2 HLZ HVGLF HZ~ R2- SBC SET UHS WUQ ZMT |
| ID | FETCH-LOGICAL-c261t-7a6ffcde6d28c790c37282c2e561ae41df22e196ec2abbb631c7b509df94e63b3 |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001597416300012&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0952-1976 |
| IngestDate | Sat Nov 29 06:50:35 EST 2025 Sat Nov 29 17:09:13 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Cloud computing Workflow Q-Learning Improved Decomposition-Based Evolutionary Algorithm Non-dominated Sorting Genetic Algorithm III Autoscaling |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c261t-7a6ffcde6d28c790c37282c2e561ae41df22e196ec2abbb631c7b509df94e63b3 |
| ORCID | 0000-0003-2882-766X 0000-0001-5761-1898 0000-0001-7854-7610 0009-0002-2110-6026 0000-0001-6444-4610 0000-0003-4353-7257 |
| ParticipantIDs | crossref_primary_10_1016_j_engappai_2025_112663 elsevier_sciencedirect_doi_10_1016_j_engappai_2025_112663 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-12-24 |
| PublicationDateYYYYMMDD | 2025-12-24 |
| PublicationDate_xml | – month: 12 year: 2025 text: 2025-12-24 day: 24 |
| PublicationDecade | 2020 |
| PublicationTitle | Engineering applications of artificial intelligence |
| PublicationYear | 2025 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Li, Wang, Fan, Geng (b17) 2022; 17 Asafuddoula, Ray, Sarker (b2) 2015; 19 Tournaire, Castel-Taleb, Hyon (b30) 2023; 8 Garí, Monge, Mateos, García Garino (b10) 2019 Drugan (b5) 2019; 44 Yannibelli, Pacini, Monge, Mateos, Rodriguez (b34) 2020 Ali, Keerio, Laghari (b1) 2021; 9 Meade, Fluke (b20) 2018; 23 Song, Wu, Guo, Yan, Suganthan, Zhang, Pedrycz, Das, Mallipeddi, Ajani, Feng (b28) 2024; 86 Monge, Pacini, Mateos, Alba, García Garino (b21) 2020; 149 Asghari, Sohrabi, Yaghmaee (b3) 2021; 77 Yannibelli, Pacini, Monge, Mateos, Rodriguez, Millán, Santos (b35) 2023 Ericson, Mohammadian, Santana (b8) 2017 MacGlashan (b19) 2016 Schad, Dittrich, Quiané-Ruiz (b25) 2010; 3 Shen, Chen, Ma, Zhang (b26) 2025 Saltelli, Ratto, Andres, Campolongo, Cariboni, Gatelli, Saisana, Tarantola (b24) 2008 Hadka (b14) 2024 Silva Filho, M.C., Oliveira, R.L., Monteiro, C.C., Inácio, P.R.M., Freire, M.M., 2017. Cloudsim plus: A cloud computing simulation framework pursuing software engineering principles for improved modularity, extensibility and correctness. In: 2017 IFIP/IEEE Symposium on Integrated Network and Service Management. IM, pp. 400–406. Liu, Feld, Xue, Garcke, Soddemann, Pan (b18) 2016; 9 Radaideh, Du, Seurin, Seyler, Gu, Wang, Shirvan (b23) 2023; 412 Wei, Kudenko, Liu, Pan, Wu, Meng (b32) 2019 Khadka, Tumer (b15) 2018 Garí, Monge, Mateos (b9) 2022; 127 Xu, Song, Ilager, Gill, Zhao, Ye, Xu (b33) 2022; 19 Netto, Calheiros, Rodrigues, Cunha, Buyya (b22) 2018; 51 Dutreilh, X., Kirgizov, S., 2011. Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In: 7th International Conference on Autonomic and Autonomous Systems. pp. 67–74. Garí, Pacini, Robino, Mateos, Monge (b12) 2024; 157 Guyon, Orgerie, Morin, Agarwal (b13) 2019; 10 Vandenbrouck, Christiany, Combes, Loux, Brun (b31) 2019; 19 Garí, Monge, Pacini, Mateos, García Garino (b11) 2021; 102 Sutton, Barto (b29) 2018 Kruekaew, Kimpan (b16) 2022; 10 Deb, Jain (b4) 2014; 18 Eiben, Smith (b7) 2015 Guyon (10.1016/j.engappai.2025.112663_b13) 2019; 10 Garí (10.1016/j.engappai.2025.112663_b9) 2022; 127 Khadka (10.1016/j.engappai.2025.112663_b15) 2018 Ali (10.1016/j.engappai.2025.112663_b1) 2021; 9 Xu (10.1016/j.engappai.2025.112663_b33) 2022; 19 Liu (10.1016/j.engappai.2025.112663_b18) 2016; 9 Yannibelli (10.1016/j.engappai.2025.112663_b35) 2023 Garí (10.1016/j.engappai.2025.112663_b10) 2019 Garí (10.1016/j.engappai.2025.112663_b12) 2024; 157 Saltelli (10.1016/j.engappai.2025.112663_b24) 2008 10.1016/j.engappai.2025.112663_b27 Li (10.1016/j.engappai.2025.112663_b17) 2022; 17 Deb (10.1016/j.engappai.2025.112663_b4) 2014; 18 Drugan (10.1016/j.engappai.2025.112663_b5) 2019; 44 MacGlashan (10.1016/j.engappai.2025.112663_b19) 2016 Schad (10.1016/j.engappai.2025.112663_b25) 2010; 3 10.1016/j.engappai.2025.112663_b6 Eiben (10.1016/j.engappai.2025.112663_b7) 2015 Garí (10.1016/j.engappai.2025.112663_b11) 2021; 102 Meade (10.1016/j.engappai.2025.112663_b20) 2018; 23 Sutton (10.1016/j.engappai.2025.112663_b29) 2018 Wei (10.1016/j.engappai.2025.112663_b32) 2019 Ericson (10.1016/j.engappai.2025.112663_b8) 2017 Vandenbrouck (10.1016/j.engappai.2025.112663_b31) 2019; 19 Asafuddoula (10.1016/j.engappai.2025.112663_b2) 2015; 19 Kruekaew (10.1016/j.engappai.2025.112663_b16) 2022; 10 Monge (10.1016/j.engappai.2025.112663_b21) 2020; 149 Shen (10.1016/j.engappai.2025.112663_b26) 2025 Tournaire (10.1016/j.engappai.2025.112663_b30) 2023; 8 Song (10.1016/j.engappai.2025.112663_b28) 2024; 86 Radaideh (10.1016/j.engappai.2025.112663_b23) 2023; 412 Hadka (10.1016/j.engappai.2025.112663_b14) 2024 Asghari (10.1016/j.engappai.2025.112663_b3) 2021; 77 Yannibelli (10.1016/j.engappai.2025.112663_b34) 2020 Netto (10.1016/j.engappai.2025.112663_b22) 2018; 51 |
| References_xml | – year: 2024 ident: b14 article-title: Moea framework: A free and open source java framework for multiobjective optimization (version 4.5) – volume: 17 year: 2022 ident: b17 article-title: A novel q-learning algorithm based on improved whale optimization algorithm for path planning publication-title: PLoS ONE – year: 2015 ident: b7 publication-title: Introduction To Evolutionary Computing – volume: 51 year: 2018 ident: b22 article-title: Hpc cloud for scientific and business applications: Taxonomy, vision, and research challenges publication-title: ACM Comput. Surv. – volume: 412 year: 2023 ident: b23 article-title: Neorl: Neuroevolution optimization with reinforcement learning—applications to carbon-free energy systems publication-title: Nucl. Eng. Des. – year: 2008 ident: b24 article-title: Global Sensitivity Analysis: The Primer – volume: 127 start-page: 168 year: 2022 end-page: 180 ident: b9 article-title: A q-learning approach for the autoscaling of scientific workflows in the cloud publication-title: Future Gener. Comput. Syst. – volume: 19 start-page: 3995 year: 2022 end-page: 4009 ident: b33 article-title: Coscal: Multifaceted scaling of microservices with reinforcement learning publication-title: IEEE Trans. Netw. Serv. Manag. – reference: Dutreilh, X., Kirgizov, S., 2011. Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In: 7th International Conference on Autonomic and Autonomous Systems. pp. 67–74. – year: 2019 ident: b10 article-title: Learning budget assignment policies for autoscaling scientific workflows in the cloud publication-title: Clust. Comput. – volume: 19 start-page: 445 year: 2015 end-page: 460 ident: b2 article-title: A decomposition-based evolutionary algorithm for many objective optimization publication-title: IEEE Trans. Evol. Comput. – volume: 23 start-page: 124 year: 2018 end-page: 140 ident: b20 article-title: Evaluating virtual hosted desktops for graphics-intensive astronomy publication-title: Astron. Comput. – volume: 86 year: 2024 ident: b28 article-title: Reinforcement learning-assisted evolutionary algorithm: A survey and research opportunities publication-title: Swarm Evol. Comput. – volume: 44 start-page: 228 year: 2019 end-page: 246 ident: b5 article-title: Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms publication-title: Swarm Evol. Comput. – volume: 149 year: 2020 ident: b21 article-title: CMI: An online multi-objective genetic autoscaler for scientific and engineering workflows in cloud infrastructures with unreliable virtual machines publication-title: J. Netw. Comput. Appl. – volume: 18 start-page: 577 year: 2014 end-page: 601 ident: b4 article-title: An evolutionary many-objective optimization algorithm using reference-point-based non-dominated sorting approach, part i: solving problems with box constraints publication-title: IEEE Trans. Evol. Comput. – year: 2018 ident: b29 article-title: Reinforcement Learning: An Introduction – volume: 157 start-page: 573 year: 2024 end-page: 586 ident: b12 article-title: Online rl-based cloud autoscaling for scientific workflows: Evaluation of q-learning and sarsa publication-title: Future Gener. Comput. Syst. – start-page: 249 year: 2020 end-page: 263 ident: b34 article-title: An nsga-iii-based multi-objective intelligent autoscaler for executing engineering applications in cloud infrastructures publication-title: Advances in Soft Computing – volume: 8 year: 2023 ident: b30 article-title: Efficient computation of optimal thresholds in cloud auto-scaling systems publication-title: ACM Trans. Model. Perform. Eval. Comput. Syst. – volume: 10 start-page: 17803 year: 2022 end-page: 17818 ident: b16 article-title: Multi-objective task scheduling optimization for load balancing in cloud computing environment using hybrid artificial bee colony algorithm with reinforcement learning publication-title: IEEE Access – year: 2018 ident: b15 article-title: Evolution-guided policy gradient in reinforcement learning publication-title: Advances in Neural Information Processing Systems – volume: 19 year: 2019 ident: b31 article-title: Bioinformatics tools and workflow to select blood biomarkers for early cancer diagnosis: an application to pancreatic cancer publication-title: Proteomics – volume: 102 year: 2021 ident: b11 article-title: Reinforcement learning-based application autoscaling in the cloud: A survey publication-title: Eng. Appl. Artif. Intell. – reference: Silva Filho, M.C., Oliveira, R.L., Monteiro, C.C., Inácio, P.R.M., Freire, M.M., 2017. Cloudsim plus: A cloud computing simulation framework pursuing software engineering principles for improved modularity, extensibility and correctness. In: 2017 IFIP/IEEE Symposium on Integrated Network and Service Management. IM, pp. 400–406. – start-page: 3 year: 2025 end-page: 18 ident: b26 article-title: Cost-aware dynamic cloud workflow scheduling using self-attention and evolutionary reinforcement learnin publication-title: Service-Oriented Computing – year: 2016 ident: b19 article-title: Brown-umbc reinforcement learning and planning burlap – start-page: 1 year: 2019 end-page: 11 ident: b32 article-title: A reinforcement learning based auto-scaling approach for saas providers in dynamic cloud environment publication-title: Math. Probl. Eng. – volume: 10 start-page: 272 year: 2019 end-page: 282 ident: b13 article-title: Involving users in energy conservation: A case study in scientific clouds publication-title: Int. J. Grid Util. Comput. – volume: 3 start-page: 460 year: 2010 end-page: 471 ident: b25 article-title: Runtime measurements in the cloud: observing, analyzing, and reducing variance publication-title: Proc. VLDB Endow. – volume: 9 start-page: 748 year: 2016 end-page: 765 ident: b18 article-title: An efficient geosciences workflow on multi-core processors and gpus: a case study for aerosol optical depth retrieval from modis satellite data publication-title: Int. J. Digit. Earth – volume: 9 start-page: 404 year: 2021 end-page: 415 ident: b1 article-title: Optimal site and size of distributed generation allocation in radial distribution network using multi-objective optimization publication-title: J. Mod. Power Syst. Clean Energy – volume: 77 start-page: 2800 year: 2021 end-page: 2828 ident: b3 article-title: Task scheduling, resource provisioning, and load balancing on scientific workflows using parallel SARSA reinforcement learning agents and genetic algorithm publication-title: J. Supercomput. – start-page: 308 year: 2017 end-page: 314 ident: b8 article-title: Analysis of performance variability in public cloud computing publication-title: Proceedings of the 2017 IEEE International Conference on Information Reuse and Integration – year: 2023 ident: b35 article-title: An in-Depth Benchmarking of Evolutionary and Swarm Intelligence Algorithms for Autoscaling Parameter Sweep Applications on Public Clouds – volume: 102 year: 2021 ident: 10.1016/j.engappai.2025.112663_b11 article-title: Reinforcement learning-based application autoscaling in the cloud: A survey publication-title: Eng. Appl. Artif. Intell. doi: 10.1016/j.engappai.2021.104288 – volume: 9 start-page: 404 year: 2021 ident: 10.1016/j.engappai.2025.112663_b1 article-title: Optimal site and size of distributed generation allocation in radial distribution network using multi-objective optimization publication-title: J. Mod. Power Syst. Clean Energy doi: 10.35833/MPCE.2019.000055 – volume: 44 start-page: 228 year: 2019 ident: 10.1016/j.engappai.2025.112663_b5 article-title: Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms publication-title: Swarm Evol. Comput. doi: 10.1016/j.swevo.2018.03.011 – ident: 10.1016/j.engappai.2025.112663_b27 doi: 10.23919/INM.2017.7987304 – volume: 10 start-page: 272 year: 2019 ident: 10.1016/j.engappai.2025.112663_b13 article-title: Involving users in energy conservation: A case study in scientific clouds publication-title: Int. J. Grid Util. Comput. doi: 10.1504/IJGUC.2019.099667 – volume: 10 start-page: 17803 year: 2022 ident: 10.1016/j.engappai.2025.112663_b16 article-title: Multi-objective task scheduling optimization for load balancing in cloud computing environment using hybrid artificial bee colony algorithm with reinforcement learning publication-title: IEEE Access doi: 10.1109/ACCESS.2022.3149955 – volume: 17 year: 2022 ident: 10.1016/j.engappai.2025.112663_b17 article-title: A novel q-learning algorithm based on improved whale optimization algorithm for path planning publication-title: PLoS ONE doi: 10.1371/journal.pone.0279438 – volume: 23 start-page: 124 year: 2018 ident: 10.1016/j.engappai.2025.112663_b20 article-title: Evaluating virtual hosted desktops for graphics-intensive astronomy publication-title: Astron. Comput. doi: 10.1016/j.ascom.2018.04.002 – volume: 412 year: 2023 ident: 10.1016/j.engappai.2025.112663_b23 article-title: Neorl: Neuroevolution optimization with reinforcement learning—applications to carbon-free energy systems publication-title: Nucl. Eng. Des. doi: 10.1016/j.nucengdes.2023.112423 – year: 2008 ident: 10.1016/j.engappai.2025.112663_b24 – volume: 127 start-page: 168 year: 2022 ident: 10.1016/j.engappai.2025.112663_b9 article-title: A q-learning approach for the autoscaling of scientific workflows in the cloud publication-title: Future Gener. Comput. Syst. doi: 10.1016/j.future.2021.09.007 – year: 2018 ident: 10.1016/j.engappai.2025.112663_b15 article-title: Evolution-guided policy gradient in reinforcement learning – year: 2018 ident: 10.1016/j.engappai.2025.112663_b29 – start-page: 249 year: 2020 ident: 10.1016/j.engappai.2025.112663_b34 article-title: An nsga-iii-based multi-objective intelligent autoscaler for executing engineering applications in cloud infrastructures – year: 2015 ident: 10.1016/j.engappai.2025.112663_b7 – year: 2023 ident: 10.1016/j.engappai.2025.112663_b35 – volume: 86 year: 2024 ident: 10.1016/j.engappai.2025.112663_b28 article-title: Reinforcement learning-assisted evolutionary algorithm: A survey and research opportunities publication-title: Swarm Evol. Comput. doi: 10.1016/j.swevo.2024.101517 – volume: 19 year: 2019 ident: 10.1016/j.engappai.2025.112663_b31 article-title: Bioinformatics tools and workflow to select blood biomarkers for early cancer diagnosis: an application to pancreatic cancer publication-title: Proteomics doi: 10.1002/pmic.201800489 – year: 2019 ident: 10.1016/j.engappai.2025.112663_b10 article-title: Learning budget assignment policies for autoscaling scientific workflows in the cloud publication-title: Clust. Comput. – volume: 51 year: 2018 ident: 10.1016/j.engappai.2025.112663_b22 article-title: Hpc cloud for scientific and business applications: Taxonomy, vision, and research challenges publication-title: ACM Comput. Surv. – start-page: 3 year: 2025 ident: 10.1016/j.engappai.2025.112663_b26 article-title: Cost-aware dynamic cloud workflow scheduling using self-attention and evolutionary reinforcement learnin – start-page: 1 year: 2019 ident: 10.1016/j.engappai.2025.112663_b32 article-title: A reinforcement learning based auto-scaling approach for saas providers in dynamic cloud environment publication-title: Math. Probl. Eng. – volume: 9 start-page: 748 year: 2016 ident: 10.1016/j.engappai.2025.112663_b18 article-title: An efficient geosciences workflow on multi-core processors and gpus: a case study for aerosol optical depth retrieval from modis satellite data publication-title: Int. J. Digit. Earth doi: 10.1080/17538947.2015.1130087 – year: 2024 ident: 10.1016/j.engappai.2025.112663_b14 – volume: 157 start-page: 573 year: 2024 ident: 10.1016/j.engappai.2025.112663_b12 article-title: Online rl-based cloud autoscaling for scientific workflows: Evaluation of q-learning and sarsa publication-title: Future Gener. Comput. Syst. doi: 10.1016/j.future.2024.04.014 – volume: 77 start-page: 2800 year: 2021 ident: 10.1016/j.engappai.2025.112663_b3 article-title: Task scheduling, resource provisioning, and load balancing on scientific workflows using parallel SARSA reinforcement learning agents and genetic algorithm publication-title: J. Supercomput. doi: 10.1007/s11227-020-03364-1 – volume: 19 start-page: 3995 year: 2022 ident: 10.1016/j.engappai.2025.112663_b33 article-title: Coscal: Multifaceted scaling of microservices with reinforcement learning publication-title: IEEE Trans. Netw. Serv. Manag. doi: 10.1109/TNSM.2022.3210211 – volume: 19 start-page: 445 year: 2015 ident: 10.1016/j.engappai.2025.112663_b2 article-title: A decomposition-based evolutionary algorithm for many objective optimization publication-title: IEEE Trans. Evol. Comput. doi: 10.1109/TEVC.2014.2339823 – volume: 8 year: 2023 ident: 10.1016/j.engappai.2025.112663_b30 article-title: Efficient computation of optimal thresholds in cloud auto-scaling systems publication-title: ACM Trans. Model. Perform. Eval. Comput. Syst. doi: 10.1145/3603532 – volume: 3 start-page: 460 year: 2010 ident: 10.1016/j.engappai.2025.112663_b25 article-title: Runtime measurements in the cloud: observing, analyzing, and reducing variance publication-title: Proc. VLDB Endow. doi: 10.14778/1920841.1920902 – volume: 18 start-page: 577 year: 2014 ident: 10.1016/j.engappai.2025.112663_b4 article-title: An evolutionary many-objective optimization algorithm using reference-point-based non-dominated sorting approach, part i: solving problems with box constraints publication-title: IEEE Trans. Evol. Comput. doi: 10.1109/TEVC.2013.2281535 – start-page: 308 year: 2017 ident: 10.1016/j.engappai.2025.112663_b8 article-title: Analysis of performance variability in public cloud computing – ident: 10.1016/j.engappai.2025.112663_b6 – year: 2016 ident: 10.1016/j.engappai.2025.112663_b19 – volume: 149 year: 2020 ident: 10.1016/j.engappai.2025.112663_b21 article-title: CMI: An online multi-objective genetic autoscaler for scientific and engineering workflows in cloud infrastructures with unreliable virtual machines publication-title: J. Netw. Comput. Appl. doi: 10.1016/j.jnca.2019.102464 |
| SSID | ssj0003846 |
| Score | 2.444218 |
| Snippet | Scientific workflow execution is usually fulfilled through Cloud Computing, but correct autoscaling techniques are needed for proper performance. Reinforcement... |
| SourceID | crossref elsevier |
| SourceType | Index Database Publisher |
| StartPage | 112663 |
| SubjectTerms | Autoscaling Cloud computing Improved Decomposition-Based Evolutionary Algorithm Non-dominated Sorting Genetic Algorithm III Q-Learning Workflow |
| Title | Reinforcement Learning initialization by evolutionary formulation: Application for workflow autoscaling in the Cloud |
| URI | https://dx.doi.org/10.1016/j.engappai.2025.112663 |
| Volume | 162 |
| WOSCitedRecordID | wos001597416300012&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 issn: 0952-1976 databaseCode: AIEXJ dateStart: 19950201 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.sciencedirect.com omitProxy: false ssIdentifier: ssj0003846 providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LT9tAEF4F6IELhRYEbUF74IacJuvn9oYCCBCgqqJSbtZ6vUZBkY0SJ9Afx39j9mVvWlTgwMWyNvLG9nzeeew3MwjtFwK-IB70vFAw3wtoL_SSKIk8QkCdBTSH80I1m4ivrpLhkP7sdB5tLsx8HJdl8vBA795V1DAGwpaps28QdzMpDMA5CB2OIHY4vkrwv4QqhspV3M_WT5V5KyP4mMcm7VJanWJu7kMS56Ttajp5qVhhu6-tiIiSvlWMq_sDNqurKcjVpMJIs3UwrmYL7T6dGocH7ga54hxMFDlJtQpxqoG2-z7gqavo7cVMxluqhh_E9J7-kdIZskdUu_nFR6ovlaSolY2WuQQjWlMIB2YdcwMcJJRkEZ1X3UQqidenuk1Ms2ibNVwvuzIPSq-T_2gEHZy47YryBp6YjbryL7rtBYsluP9SjQ1h0XLhblM7TyrnSfU8S2iFxCEFvbByeHY8PG9MAT_RmWL2CZwU9efv6HnryLF4rtfRmnFV8KGG2AbqiPIT-mjcFmyUwhSGbGcQO_YZ1QsgxBaEeBGEOPuDXRBiB4Q_sANB-QO2EMQOBGFCDBDECoKb6PfJ8fXg1DP9PTwOfnvtxSwqCp6LKCcJj2mP-zFJCCcCbHomgn5eECJAQwhOWJZlkd_ncQYGbl7QQER-5m-h5bIqxTbCmSAhpwycZREGlLMkDHnEwJXnjOVR4e-g7_a9pne6jEv6f5nuIGpff2qMUW1kpoCsF6798uZ_-4pWW-h_Q8v1ZCZ20Qc-r0fTyZ6B1RPgmrcC |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Reinforcement+Learning+initialization+by+evolutionary+formulation%3A+Application+for+workflow+autoscaling+in+the+Cloud&rft.jtitle=Engineering+applications+of+artificial+intelligence&rft.au=Robino%2C+Luciano&rft.au=Gar%C3%AD%2C+Yisel&rft.au=Pacini%2C+Elina&rft.au=Mateos%2C+Cristian&rft.date=2025-12-24&rft.issn=0952-1976&rft.volume=162&rft.spage=112663&rft_id=info:doi/10.1016%2Fj.engappai.2025.112663&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_engappai_2025_112663 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0952-1976&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0952-1976&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0952-1976&client=summon |