Automated Metaheuristic Algorithm Design With Autoregressive Learning
Automated design of metaheuristic algorithms offers an attractive avenue to reduce human effort and gain enhanced performance beyond human intuition. Current automated methods design algorithms within a fixed structure and operate from scratch. This poses a clear gap toward fully discovering potenti...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on evolutionary computation Jg. 29; H. 5; S. 2004 - 2018 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
01.10.2025
|
| Schlagworte: | |
| ISSN: | 1089-778X, 1941-0026 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Automated design of metaheuristic algorithms offers an attractive avenue to reduce human effort and gain enhanced performance beyond human intuition. Current automated methods design algorithms within a fixed structure and operate from scratch. This poses a clear gap toward fully discovering potentials over the metaheuristic family and fertilizing from prior design experience. To bridge the gap, this article proposes an autoregressive learning-based designer for automated design of metaheuristic algorithms. Our designer formulates metaheuristic algorithm design as a sequence generation task, and harnesses an autoregressive generative network to handle the task. This offers two advances. First, through autoregressive inference, the designer generates algorithms with diverse lengths and structures, enabling to fully discover potentials over the metaheuristic family. Second, prior design knowledge learned and accumulated in neurons of the designer can be retrieved for designing algorithms for future problems, paving the way to continual design of algorithms for open-ended problem solving. Extensive experiments on numeral benchmarks and real-world problems reveal that the proposed designer generates algorithms that outperform all human-created baselines on 24 out of 25 test problems. The generated algorithms display various structures and behaviors, reasonably fitting for different problem-solving contexts. Code is available at https://github.com/auto4opt/ALDes . |
|---|---|
| AbstractList | Automated design of metaheuristic algorithms offers an attractive avenue to reduce human effort and gain enhanced performance beyond human intuition. Current automated methods design algorithms within a fixed structure and operate from scratch. This poses a clear gap toward fully discovering potentials over the metaheuristic family and fertilizing from prior design experience. To bridge the gap, this article proposes an autoregressive learning-based designer for automated design of metaheuristic algorithms. Our designer formulates metaheuristic algorithm design as a sequence generation task, and harnesses an autoregressive generative network to handle the task. This offers two advances. First, through autoregressive inference, the designer generates algorithms with diverse lengths and structures, enabling to fully discover potentials over the metaheuristic family. Second, prior design knowledge learned and accumulated in neurons of the designer can be retrieved for designing algorithms for future problems, paving the way to continual design of algorithms for open-ended problem solving. Extensive experiments on numeral benchmarks and real-world problems reveal that the proposed designer generates algorithms that outperform all human-created baselines on 24 out of 25 test problems. The generated algorithms display various structures and behaviors, reasonably fitting for different problem-solving contexts. Code is available at https://github.com/auto4opt/ALDes . |
| Author | Zhao, Qi Shi, Yuhui Liu, Tengfei Duan, Qiqi Yan, Bai Yang, Jian |
| Author_xml | – sequence: 1 givenname: Qi orcidid: 0000-0003-4800-1136 surname: Zhao fullname: Zhao, Qi email: zhaoq@sustech.edu.cn organization: Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China – sequence: 2 givenname: Tengfei orcidid: 0009-0003-1435-9209 surname: Liu fullname: Liu, Tengfei email: 12332470@mail.sustech.edu.cn organization: Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China – sequence: 3 givenname: Bai orcidid: 0000-0003-3374-093X surname: Yan fullname: Yan, Bai email: yanb@sustech.edu.cn organization: Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, Shenzhen, China – sequence: 4 givenname: Qiqi orcidid: 0000-0002-4518-1701 surname: Duan fullname: Duan, Qiqi email: 11749325@mail.sustech.edu.cn organization: Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China – sequence: 5 givenname: Jian surname: Yang fullname: Yang, Jian email: yangj33@sustech.edu.cn organization: Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China – sequence: 6 givenname: Yuhui orcidid: 0000-0002-8840-723X surname: Shi fullname: Shi, Yuhui email: shiyh@sustech.edu.cn organization: Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China |
| BookMark | eNpNkM1OwzAQhC1UJNrCAyBxyAuk7NpOYh-rUgpSEJfyc4scZ5MatQmyUyTenkTtgb3MjjQzh2_GJm3XEmO3CAtE0Pfb9ftqwYHLhZCpTLPsgk1RS4wBeDoZflA6zjL1ecVmIXwBoExQT9l6eey7g-mpil6oNzs6ehd6Z6Plvum863eH6IGCa9roYzDRmPbUeArB_VCUk_Gta5trdlmbfaCbs87Z2-N6u3qK89fN82qZx5aj6mNpQGgpqoQEotSlBVsJpTCtRZlUda2sAJ4gNxVPLJdlNpwC0KZEQtQk5gxPu9Z3IXiqi2_vDsb_FgjFyKEYORQjh-LMYejcnTqOiP7lUyUVJOIPZa1btw |
| CODEN | ITEVF5 |
| Cites_doi | 10.1126/science.220.4598.671 10.1109/TEVC.2015.2474158 10.1145/2598394.2605681 10.1109/TETCI.2022.3146882 10.1126/sciadv.aap7885 10.48550/ARXIV.1706.03762 10.1145/3319619.3326913 10.1613/jair.1.13676 10.1007/978-3-319-96514-7 10.18653/v1/D16-1244 10.1162/evco_a_00263 10.1016/j.eswa.2022.119461 10.1038/s42256-022-00446-y 10.1016/0041-5553(67)90144-9 10.1109/MCI.2020.2976182 10.1109/TCOMM.2020.3033006 10.1007/3-540-48885-5_2 10.1109/CVPR.2018.00907 10.1016/j.ins.2013.04.015 10.1007/BF00175355 10.1109/ICEC.1998.699146 10.1287/ijoc.1.3.190 10.1016/j.asoc.2022.109725 10.1038/nature14236 10.1109/TEVC.2022.3197298 10.1007/978-3-642-01181-8_10 10.1007/0-306-48056-5_11 10.1109/JSAC.2020.3000813 10.1016/j.cor.2018.12.015 10.1162/evco.1993.1.1.1 10.1613/jair.2861 10.1016/j.eswa.2021.115493 10.1007/978-3-642-25566-3_40 10.1145/3205651.3208304 10.1073/pnas.1611835114 10.1109/TPAMI.2021.3057446 10.1007/978-3-319-91086-4_17 10.1016/j.orp.2016.09.002 10.1016/j.ejor.2021.05.042 10.1145/1543834.1543915 10.1016/S0305-0548(97)00031-2 10.1007/978-3-030-58115-2_10 10.1145/2001576.2001690 10.1057/jors.2013.71 10.1109/TEVC.2022.3159087 10.1007/978-3-642-17310-3_2 10.1109/TIT.2003.821988 10.1145/2464576.2482728 10.1109/CEC.2007.4424455 10.1007/978-3-030-72699-7_2 10.1109/TEVC.2021.3102863 10.3389/fphy.2014.00005 10.1038/s42256-021-00411-1 10.1109/SITIS.2010.22 10.1137/0202009 10.18653/v1/2023.acl-long.205 10.1162/evco_a_00240 10.1016/j.asoc.2019.106027 10.1162/evco_a_00242 10.1145/3319619.3326827 10.1007/978-3-319-30668-1_19 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TEVC.2024.3464677 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library (IEL) (UW System Shared) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1941-0026 |
| EndPage | 2018 |
| ExternalDocumentID | 10_1109_TEVC_2024_3464677 10684805 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Shenzhen Municipal Fundamental Research Program; Shenzhen Fundamental Research Program grantid: JCYJ20200109141235597 funderid: 10.13039/501100017607 – fundername: Program for Guangdong Introducing Innovative and Entrepreneurial Teams grantid: 2017ZT07X386 – fundername: Guangdong Basic and Applied Basic Research Foundation grantid: 2024A1515012241; 2021A1515110024 funderid: 10.13039/501100001321 – fundername: National Science Foundation of China grantid: 72401122; 61761136008 funderid: 10.13039/501100001809 – fundername: Shenzhen Peacock Plan grantid: KQTD2016112514355531 funderid: 10.13039/501100012234 |
| GroupedDBID | -~X .DC 0R~ 29I 4.4 5GY 5VS 6IF 6IK 97E AAJGR AASAJ AAWTH ABAZT ABJNI ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 EBS EJD HZ~ H~9 IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P PQQKQ RIA RIE RNS TN5 VH1 AAYXX CITATION |
| ID | FETCH-LOGICAL-c218t-4a03943d5e31149bc0cd38816f3b5dff8c302512ad25c24b77778009ab1e119e3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001591697300026&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1089-778X |
| IngestDate | Sat Nov 29 07:12:18 EST 2025 Sat Oct 25 03:12:18 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c218t-4a03943d5e31149bc0cd38816f3b5dff8c302512ad25c24b77778009ab1e119e3 |
| ORCID | 0009-0003-1435-9209 0000-0002-4518-1701 0000-0003-4800-1136 0000-0002-8840-723X 0000-0003-3374-093X |
| PageCount | 15 |
| ParticipantIDs | crossref_primary_10_1109_TEVC_2024_3464677 ieee_primary_10684805 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-10-01 |
| PublicationDateYYYYMMDD | 2025-10-01 |
| PublicationDate_xml | – month: 10 year: 2025 text: 2025-10-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE transactions on evolutionary computation |
| PublicationTitleAbbrev | TEVC |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref13 ref57 ref12 ref56 ref15 ref59 ref14 ref58 ref11 ref55 ref10 ref54 ref17 ref16 ref19 ref18 ref50 Glover (ref1) 2006 ref48 ref41 ref44 ref43 Kingma (ref64) 2017 ref49 ref8 ref7 ref4 ref6 ref5 Schulman (ref52) ref40 Loynd (ref51) Goodfellow (ref46) 2016 ref35 ref34 ref37 ref36 ref31 ref30 ref33 ref32 ref2 ref39 Mikolov (ref53) 2013 Zhao (ref3) 2024 ref71 Zoph (ref47) ref70 ref73 ref72 ref24 ref68 ref23 ref67 ref26 ref25 ref69 ref20 ref63 ref22 ref66 Lindauer (ref9) 2022; 23 ref21 ref65 ref28 ref27 Adriaensen (ref38) ref29 ref60 ref62 ref61 Schulman (ref42) 2017 Lin (ref45) 2017 |
| References_xml | – ident: ref32 doi: 10.1126/science.220.4598.671 – ident: ref16 doi: 10.1109/TEVC.2015.2474158 – ident: ref37 doi: 10.1145/2598394.2605681 – ident: ref21 doi: 10.1109/TETCI.2022.3146882 – ident: ref49 doi: 10.1126/sciadv.aap7885 – ident: ref43 doi: 10.48550/ARXIV.1706.03762 – ident: ref72 doi: 10.1145/3319619.3326913 – ident: ref7 doi: 10.1613/jair.1.13676 – ident: ref13 doi: 10.1007/978-3-319-96514-7 – ident: ref44 doi: 10.18653/v1/D16-1244 – ident: ref18 doi: 10.1162/evco_a_00263 – ident: ref40 doi: 10.1016/j.eswa.2022.119461 – year: 2017 ident: ref64 article-title: Adam: A method for stochastic optimization publication-title: arXiv:1412.6980 – ident: ref67 doi: 10.1038/s42256-022-00446-y – ident: ref56 doi: 10.1016/0041-5553(67)90144-9 – ident: ref5 doi: 10.1109/MCI.2020.2976182 – ident: ref68 doi: 10.1109/TCOMM.2020.3033006 – ident: ref29 doi: 10.1007/3-540-48885-5_2 – start-page: 1889 volume-title: Proc. Int. Conf. Mach. Learn. ident: ref52 article-title: Trust region policy optimization – ident: ref48 doi: 10.1109/CVPR.2018.00907 – ident: ref58 doi: 10.1016/j.ins.2013.04.015 – start-page: 554 volume-title: Proc. 25th Int. Joint Conf. Artif. Intell. ident: ref38 article-title: Towards a white box approach to automated algorithm design – ident: ref14 doi: 10.1007/BF00175355 – ident: ref33 doi: 10.1109/ICEC.1998.699146 – start-page: 6404 volume-title: Proc. Int. Conf. Mach. Learn. ident: ref51 article-title: Working memory graphs – ident: ref63 doi: 10.1287/ijoc.1.3.190 – ident: ref69 doi: 10.1016/j.asoc.2022.109725 – ident: ref41 doi: 10.1038/nature14236 – ident: ref20 doi: 10.1109/TEVC.2022.3197298 – ident: ref30 doi: 10.1007/978-3-642-01181-8_10 – ident: ref62 doi: 10.1007/0-306-48056-5_11 – ident: ref71 doi: 10.1109/JSAC.2020.3000813 – ident: ref35 doi: 10.1016/j.cor.2018.12.015 – start-page: 1 year: 2024 ident: ref3 article-title: Automated design of metaheuristic algorithms: A survey publication-title: Trans. Mach. Learn. Res. – ident: ref15 doi: 10.1162/evco.1993.1.1.1 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Represent. ident: ref47 article-title: Neural architecture search with reinforcement learning – ident: ref11 doi: 10.1613/jair.2861 – ident: ref39 doi: 10.1016/j.eswa.2021.115493 – ident: ref8 doi: 10.1007/978-3-642-25566-3_40 – ident: ref23 doi: 10.1145/3205651.3208304 – ident: ref59 doi: 10.1073/pnas.1611835114 – year: 2017 ident: ref42 article-title: Proximal policy optimization algorithms publication-title: arXiv:1707.06347 – ident: ref60 doi: 10.1109/TPAMI.2021.3057446 – volume: 23 start-page: 1 issue: 54 year: 2022 ident: ref9 article-title: SMAC3: A versatile Bayesian optimization package for hyperparameter optimization publication-title: J. Mach. Learn. Res. – ident: ref4 doi: 10.1007/978-3-319-91086-4_17 – ident: ref10 doi: 10.1016/j.orp.2016.09.002 – ident: ref2 doi: 10.1016/j.ejor.2021.05.042 – ident: ref25 doi: 10.1145/1543834.1543915 – year: 2017 ident: ref45 article-title: A structured self-attentive sentence embedding publication-title: arXiv:1703.03130 – volume-title: Deep Learning year: 2016 ident: ref46 – ident: ref65 doi: 10.1016/S0305-0548(97)00031-2 – ident: ref57 doi: 10.1007/978-3-030-58115-2_10 – ident: ref55 doi: 10.1145/2001576.2001690 – volume-title: Handbook of Metaheuristics year: 2006 ident: ref1 – ident: ref12 doi: 10.1057/jors.2013.71 – ident: ref22 doi: 10.1109/TEVC.2022.3159087 – ident: ref28 doi: 10.1007/978-3-642-17310-3_2 – ident: ref70 doi: 10.1109/TIT.2003.821988 – ident: ref27 doi: 10.1145/2464576.2482728 – ident: ref26 doi: 10.1109/CEC.2007.4424455 – ident: ref73 doi: 10.1007/978-3-030-72699-7_2 – ident: ref19 doi: 10.1109/TEVC.2021.3102863 – ident: ref66 doi: 10.3389/fphy.2014.00005 – ident: ref50 doi: 10.1038/s42256-021-00411-1 – ident: ref36 doi: 10.1109/SITIS.2010.22 – ident: ref34 doi: 10.1137/0202009 – ident: ref54 doi: 10.18653/v1/2023.acl-long.205 – year: 2013 ident: ref53 article-title: Efficient estimation of word representations in vector space publication-title: arXiv:1301.3781 – ident: ref17 doi: 10.1162/evco_a_00240 – ident: ref61 doi: 10.1016/j.asoc.2019.106027 – ident: ref6 doi: 10.1162/evco_a_00242 – ident: ref24 doi: 10.1145/3319619.3326827 – ident: ref31 doi: 10.1007/978-3-319-30668-1_19 |
| SSID | ssj0014519 |
| Score | 2.4929075 |
| Snippet | Automated design of metaheuristic algorithms offers an attractive avenue to reduce human effort and gain enhanced performance beyond human intuition. Current... |
| SourceID | crossref ieee |
| SourceType | Index Database Publisher |
| StartPage | 2004 |
| SubjectTerms | Approximation algorithms Automated design evolutionary algorithm learning metaheuristic Metaheuristics Neurons optimization Problem-solving Search problems transformer Transformers Vectors |
| Title | Automated Metaheuristic Algorithm Design With Autoregressive Learning |
| URI | https://ieeexplore.ieee.org/document/10684805 |
| Volume | 29 |
| WOSCitedRecordID | wos001591697300026&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE/IET Electronic Library (IEL) (UW System Shared) customDbUrl: eissn: 1941-0026 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014519 issn: 1089-778X databaseCode: RIE dateStart: 19970101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELWgYoCBQimifMkDE1JKEjv-GKvSioWKoUC3KLYvbSVoUUj4_dhOisrAQKYk8mC9xL473917CN0kQipGExPkmoYBFYYFmQIIVBIrrg1jRudebIJPJmI2k09Ns7rvhQEAX3wGfXfrc_lmrSt3VGZXOBNUOMbSXc5Z3az1kzJwPCl1Nb20LqOYNSnMKJR309HL0IaCMe0TyuzOwH8ZoS1VFW9Uxu1_TucIHTbeIx7Un_sY7cCqg9obZQbcLNQOOtiiGTxBo0FVrq1nCgY_QpktoKrpmfHgbb4uluXiHd_7Qg78ah-wG12Aj8PtVogbBtZ5Fz2PR9PhQ9DIJwTa2u0yoFlIJCUmAWKDHql0qA0RImI5UYnJc6GJCzDizMSJjqni9rLuo8xUBFEkgZyi1mq9gjOEaU5NbiCJNFGUcimcSIlUMXApNVOkh243eKYfNUtG6qOLUKYO_NSBnzbg91DXYbk1sIbx_I_3F2g_dqK7voLuErXKooIrtKe_yuVnce1_gm9VDLAZ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT8IwFG4MmqgHUcSIP3vwZDLc1m5rjwQhGIF4QOW2rO0bkCiYufn323bD4MGDO21LD823te-9vve-D6GbgHER0kA5qaSuQ5kKnUQAOCLwRSRVGCqZWrGJaDxm0yl_qprVbS8MANjiM2ibW5vLVytZmKMyvcJDRplhLN0OKPXdsl3rJ2lgmFLKenqunUY2rZKYnsvvJr2Xrg4GfdomNNR7Q_TLDG3oqliz0q__c0KH6KDyH3Gn_OBHaAuWDVRfazPgaqk20P4G0eAx6nWKfKV9U1B4BHkyh6IkaMadt9kqW-Tzd3xvSznwq37AZnQGNhLXmyGuOFhnTfTc7026A6cSUHCktty5QxOXcEpUAESHPVxIVyrCmBemRAQqTZkkJsTwE-UH0qci0pd2IHkiPPA8DuQE1ZarJZwiTFOqUgWBJ4mgNOLMyJRw4UPEuQwFaaHbNZ7xR8mTEdv4wuWxAT824McV-C3UNFhuDCxhPPvj_TXaHUxGw3j4MH48R3u-keC19XQXqJZnBVyiHfmVLz6zK_tDfAOu4rNg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Automated+Metaheuristic+Algorithm+Design+With+Autoregressive+Learning&rft.jtitle=IEEE+transactions+on+evolutionary+computation&rft.au=Zhao%2C+Qi&rft.au=Liu%2C+Tengfei&rft.au=Yan%2C+Bai&rft.au=Duan%2C+Qiqi&rft.date=2025-10-01&rft.issn=1089-778X&rft.eissn=1941-0026&rft.volume=29&rft.issue=5&rft.spage=2004&rft.epage=2018&rft_id=info:doi/10.1109%2FTEVC.2024.3464677&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TEVC_2024_3464677 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1089-778X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1089-778X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1089-778X&client=summon |