Best practices for comparing optimization algorithms

Comparing, or benchmarking, of optimization algorithms is a complicated task that involves many subtle considerations to yield a fair and unbiased evaluation. In this paper, we systematically review the benchmarking process of optimization algorithms, and discuss the challenges of fair comparison. W...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Optimization and engineering Ročník 18; číslo 4; s. 815 - 848
Hlavní autoři:	Beiranvand, Vahid, Hare, Warren, Lucet, Yves
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York Springer US 01.12.2017 Springer Nature B.V
Témata:	Algorithms Benchmarks Control Engineering Environmental Management Financial Engineering Mathematics Mathematics and Statistics Operations Research/Decision Theory Optimization Optimization algorithms Systems Theory Optimization algorithms Benchmarking Algorithm comparison Software Metric Timing Performance Guidelines Testing
ISSN:	1389-4420, 1573-2924
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	Comparing, or benchmarking, of optimization algorithms is a complicated task that involves many subtle considerations to yield a fair and unbiased evaluation. In this paper, we systematically review the benchmarking process of optimization algorithms, and discuss the challenges of fair comparison. We provide suggestions for each step of the comparison process and highlight the pitfalls to avoid when evaluating the performance of optimization algorithms. We also discuss various methods of reporting the benchmarking results. Finally, some suggestions for future research are presented to improve the current benchmarking process.
AbstractList	Comparing, or benchmarking, of optimization algorithms is a complicated task that involves many subtle considerations to yield a fair and unbiased evaluation. In this paper, we systematically review the benchmarking process of optimization algorithms, and discuss the challenges of fair comparison. We provide suggestions for each step of the comparison process and highlight the pitfalls to avoid when evaluating the performance of optimization algorithms. We also discuss various methods of reporting the benchmarking results. Finally, some suggestions for future research are presented to improve the current benchmarking process.
Author	Lucet, Yves Beiranvand, Vahid Hare, Warren
Author_xml	– sequence: 1 givenname: Vahid surname: Beiranvand fullname: Beiranvand, Vahid organization: Department of Computer Science, University of British Columbia – sequence: 2 givenname: Warren surname: Hare fullname: Hare, Warren organization: Department of Mathematics, University of British Columbia – sequence: 3 givenname: Yves surname: Lucet fullname: Lucet, Yves email: yves.lucet@ubc.ca organization: Department of Computer Science, University of British Columbia
BookMark	eNp9kMtKAzEUhoNUsFYfwN2A6-g5mVwmSy3eoOCm-xDTTE1pJ2OSLvTpnTqCIOjqnMX_nct3SiZd7DwhFwhXCKCuMyI0SAEV1bWUFI_IFIWqKdOMT4a-bjTlnMEJOc15A4BSsGZK-K3PpeqTdSU4n6s2psrFXW9T6NZV7EvYhQ9bQuwqu13HFMrrLp-R49Zusz__rjOyvL9bzh_p4vnhaX6zoI4jK7RlwtcrkHKF2nOLXngvOEOvhADXvGjnvbW1VrzVtgUvWic1OGFhBU619YxcjmP7FN_2w51mE_epGzYaBrIWUiHKIYVjyqWYc_Kt6VPY2fRuEMzBjRndmMGNObgxODDqF-NC-fqyJBu2_5JsJHN_MOTTz01_Q58eQXqS
CitedBy_id	crossref_primary_10_1080_00207543_2021_1910872 crossref_primary_10_1002_adfm_202407999 crossref_primary_10_3390_en15207734 crossref_primary_10_1002_cmtd_202100091 crossref_primary_10_1039_D3RE00476G crossref_primary_10_1007_s00500_019_03826_6 crossref_primary_10_33693_2313_223X_2024_11_2_58_85 crossref_primary_10_3390_app14072786 crossref_primary_10_1016_j_asoc_2019_04_032 crossref_primary_10_1016_j_future_2024_05_021 crossref_primary_10_1364_AO_472556 crossref_primary_10_1109_TSMC_2023_3325810 crossref_primary_10_1007_s00500_020_05016_1 crossref_primary_10_1016_j_icheatmasstransfer_2020_104822 crossref_primary_10_1145_3617590 crossref_primary_10_1016_j_isatra_2022_11_025 crossref_primary_10_1016_j_advengsoft_2022_103375 crossref_primary_10_3390_s23063293 crossref_primary_10_1016_j_enbuild_2019_01_048 crossref_primary_10_1016_j_swevo_2024_101590 crossref_primary_10_1016_j_enbuild_2021_110916 crossref_primary_10_1007_s00500_024_09716_w crossref_primary_10_1016_j_electacta_2025_145673 crossref_primary_10_1007_s10589_020_00256_1 crossref_primary_10_1080_10556788_2020_1808977 crossref_primary_10_1080_15567036_2022_2032879 crossref_primary_10_1029_2017WR022051 crossref_primary_10_1111_mice_12879 crossref_primary_10_1080_10556788_2022_2053970 crossref_primary_10_1007_s11081_024_09952_x crossref_primary_10_3390_electronics11060909 crossref_primary_10_1029_2020JB020545 crossref_primary_10_1016_j_asoc_2020_106315 crossref_primary_10_1016_j_jfoodeng_2021_110888 crossref_primary_10_1371_journal_pone_0323097 crossref_primary_10_1016_j_compchemeng_2022_107909 crossref_primary_10_3390_econometrics5040049 crossref_primary_10_1016_j_compbiomed_2024_108329 crossref_primary_10_1109_ACCESS_2025_3598642 crossref_primary_10_1007_s10898_020_00928_6 crossref_primary_10_1002_aenm_202500247 crossref_primary_10_1016_j_asoc_2022_109478 crossref_primary_10_1145_3310362 crossref_primary_10_1016_j_egyr_2022_10_403 crossref_primary_10_1016_j_engappai_2022_105521 crossref_primary_10_1145_3338517 crossref_primary_10_3390_a13120332 crossref_primary_10_3390_s21237990 crossref_primary_10_1016_j_dajour_2023_100294 crossref_primary_10_1109_TEVC_2021_3139263 crossref_primary_10_1007_s10898_020_00939_3 crossref_primary_10_3390_infrastructures9100175 crossref_primary_10_1007_s11227_024_06547_2 crossref_primary_10_3390_en15041270 crossref_primary_10_1007_s10589_019_00115_8 crossref_primary_10_1080_15325008_2022_2136292 crossref_primary_10_1016_j_jocm_2020_100226 crossref_primary_10_3390_s22186843 crossref_primary_10_1016_j_fluid_2021_113180 crossref_primary_10_1039_D5DD00050E crossref_primary_10_1109_TEVC_2022_3210897 crossref_primary_10_3390_biomimetics8030278 crossref_primary_10_1002_2050_7038_13181 crossref_primary_10_1080_02331934_2024_2390119 crossref_primary_10_3390_app14020623 crossref_primary_10_1007_s00521_022_07788_z crossref_primary_10_1016_j_ejor_2020_11_016 crossref_primary_10_3390_a16020092 crossref_primary_10_1109_TEVC_2022_3199647 crossref_primary_10_1002_mp_15373 crossref_primary_10_1002_jnm_2833 crossref_primary_10_1080_00207543_2023_2226778 crossref_primary_10_1038_s41598_023_45349_z crossref_primary_10_1002_jnm_3016 crossref_primary_10_1515_eng_2017_0050 crossref_primary_10_1109_ACCESS_2024_3463393 crossref_primary_10_1016_j_knosys_2023_110697 crossref_primary_10_1088_1757_899X_734_1_012141 crossref_primary_10_1016_j_enbuild_2025_115562 crossref_primary_10_1080_10556788_2023_2296432 crossref_primary_10_1016_j_ejco_2021_100012 crossref_primary_10_1287_ijoo_2022_0073 crossref_primary_10_1016_j_jss_2024_112183 crossref_primary_10_1109_JSEN_2024_3424307 crossref_primary_10_3390_su12145803 crossref_primary_10_1016_j_infsof_2025_107807 crossref_primary_10_1016_j_ijhydene_2023_06_264 crossref_primary_10_3390_biomimetics10090612 crossref_primary_10_1137_20M1318894 crossref_primary_10_1007_s00158_020_02521_7 crossref_primary_10_1007_s00500_019_04558_3 crossref_primary_10_3390_make7010024 crossref_primary_10_1016_j_automatica_2021_109890 crossref_primary_10_1080_10556788_2018_1504049 crossref_primary_10_1080_0305215X_2019_1691547 crossref_primary_10_1016_j_ijepes_2021_107795 crossref_primary_10_1155_2021_5513860 crossref_primary_10_1007_s10462_020_09906_6 crossref_primary_10_1016_j_solener_2022_08_015 crossref_primary_10_1007_s12065_022_00741_y crossref_primary_10_1007_s11081_020_09483_1 crossref_primary_10_1016_j_stueduc_2024_101406 crossref_primary_10_1016_j_asoc_2024_111776 crossref_primary_10_1007_s41660_022_00306_7 crossref_primary_10_1016_j_ifacol_2019_12_063 crossref_primary_10_1287_ijoc_2022_1261 crossref_primary_10_1103_PhysRevResearch_6_033142 crossref_primary_10_1007_s10898_018_0673_7 crossref_primary_10_1109_ACCESS_2025_3573679 crossref_primary_10_1080_10407782_2019_1630241 crossref_primary_10_1007_s12190_023_01964_7
Cites_doi	10.1007/978-1-4614-8042-6 10.1007/s11081-013-9234-6 10.1007/978-3-642-25566-3_47 10.1007/s10589-014-9687-3 10.1007/0-306-47648-7_6 10.1007/s00500-011-0754-8 10.1016/j.ejor.2011.06.011 10.1145/962437.962439 10.1145/235141.235145 10.1007/978-3-642-00483-4 10.1007/s10898-012-9951-y 10.1007/978-3-642-02538-9_11 10.1145/2950048 10.1007/BF01404569 10.1007/978-1-4684-7074-1_6 10.1016/j.cam.2004.11.029 10.1007/s11750-011-0209-5 10.1115/1.3254783 10.1007/978-3-642-85974-8_3 10.1145/200979.201043 10.1007/978-3-642-46424-9 10.1007/978-1-4613-0241-4_5 10.1145/962437.962444 10.1007/978-1-4757-5362-2_14 10.1016/j.cor.2014.06.019 10.1007/BF00933402 10.1287/opre.33.5.1024 10.1023/A:1011319115230 10.1137/0707011 10.1007/978-3-540-39901-8_16 10.1007/s10898-007-9133-5 10.1023/A:1008306323448 10.1287/ijoc.1060.0182 10.1007/BF01581203 10.1016/j.matcom.2016.05.006 10.1137/S003614450242889 10.1080/10556788.2016.1226305 10.1007/978-3-642-95406-1_3 10.1023/A:1008677427361 10.1145/355934.355936 10.1007/BF01580667 10.1007/978-3-642-95406-1_7 10.1007/s10107-013-0679-3 10.1007/s12532-011-0025-9 10.1007/978-3-642-95406-1_19 10.1145/138351.138378 10.1007/BF02242139 10.1007/BF01588801 10.1287/ijoc.8.1.1 10.1145/355759.355760 10.1007/978-3-642-48320-2 10.1016/j.cnsns.2014.08.026 10.1137/0101002 10.1007/978-1-4615-4677-1 10.1007/BF00926604 10.1007/s101070100263 10.1287/ijoc.5.1.2 10.1023/A:1008632215341 10.1145/355826.355833 10.1023/A:1008395408187 10.1007/BF00933404 10.1007/s10107-003-0418-2 10.1137/090754595 10.1007/978-1-4757-5362-2_15 10.1016/j.advwatres.2008.01.010 10.1007/s11590-013-0641-6 10.1063/1.4952200 10.1007/978-3-642-33206-7_10 10.1137/040620886 10.1007/s10898-006-9099-8 10.1007/s10107-005-0585-4 10.1007/s10898-004-9971-3 10.4310/SII.2017.v10.n1.a6 10.1155/MPE.2005.165 10.1115/1.3254782 10.1007/BF01932400 10.1137/040621132 10.1137/S1064827599365823 10.1080/10556788.2010.547581 10.1007/978-1-4757-3040-1 10.1016/j.orl.2013.08.007 10.1007/BF01584657 10.1007/s12532-014-0067-x 10.1145/356068.356075 10.1093/comjnl/9.1.67 10.1007/s10898-013-0131-5 10.1007/978-3-642-13520-0_23 10.1007/BF01096734 10.1287/mnsc.23.12.1273 10.1109/TAC.1969.1099262 10.1007/s10898-006-9084-2 10.1007/s10898-004-9972-2 10.1080/10556780500065366 10.1007/BF02592099 10.1007/s10107-002-0355-5 10.1137/080724083 10.1016/j.advengsoft.2014.09.014 10.1007/978-3-540-39901-8_17 10.1007/s11081-009-9082-6 10.1007/978-0-387-74759-0_71 10.1137/0801023 10.1103/PhysRevC.82.024313 10.1007/s10898-014-0180-4 10.1007/BF02430363 10.2172/775270 10.1109/CEC.2005.1554761 10.1007/3-540-53032-0 10.1145/318371.318618 10.1016/j.amc.2017.05.014 10.1613/jair.2861 10.2172/79972 10.2172/834714 10.1145/1143997.1144029 10.1007/s10589-014-9708-2 10.1007/978-1-4612-5022-7
ContentType	Journal Article
Copyright	Springer Science+Business Media, LLC 2017 Copyright Springer Science & Business Media 2017
Copyright_xml	– notice: Springer Science+Business Media, LLC 2017 – notice: Copyright Springer Science & Business Media 2017
DBID	AAYXX CITATION 7TB 8FD FR3 KR7
DOI	10.1007/s11081-017-9366-1
DatabaseName	CrossRef Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database Civil Engineering Abstracts
DatabaseTitle	CrossRef Civil Engineering Abstracts Engineering Research Database Technology Research Database Mechanical & Transportation Engineering Abstracts
DatabaseTitleList	Civil Engineering Abstracts
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Mathematics
EISSN	1573-2924
EndPage	848
ExternalDocumentID	10_1007_s11081_017_9366_1
GroupedDBID	-5D -5G -BR -EM -Y2 -~C .86 .DC .VR 06D 0R~ 0VY 123 1N0 1SB 203 29N 2J2 2JN 2JY 2KG 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5VS 67Z 6NX 8TC 8UJ 95- 95. 95~ 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDZT ABECU ABFSI ABFTD ABFTV ABHLI ABHQN ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFO ACGFS ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACREN ACSNA ACZOJ ADHHG ADHIR ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADYOE ADZKW AEBTG AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFGCZ AFLOW AFQWF AFWTZ AFYQB AFZKB AGAYW AGDGC AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMTXH AMXSW AMYLF AMYQR AOCGG ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. BA0 BAPOH BDATZ BGNMA BSONS CAG COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DU5 EBLON EBS EIOEI EJD ESBYG FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I09 IHE IJ- IKXTQ IWAJR IXC IXD IXE IZIGR IZQ I~X I~Z J-C J0Z J9A JBSCW JCJTX JZLTJ KDC KOV LAK LLZTM M4Y MA- N2Q N9A NB0 NPVJJ NQJWS NU0 O9- O93 O9J OAM OVD P2P P9R PF0 PT4 Q2X QOS R89 R9I RNI RNS ROL RPX RSV RZC RZE S16 S1Z S27 S3B SAP SDH SHX SISQX SJYHP SMT SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 TEORI TSG TSK TSV TUC U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WK8 YLTOR Z45 Z7R Z7S Z7X Z7Y Z7Z Z83 Z88 ZMTXR ~A9 8AO AAPKM AAYXX ABBRH ABDBE ABFSG ABJCF ABRTQ ACSTC ADHKG AEZWR AFDZB AFFHD AFHIU AFKRA AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA BENPR BGLVJ CCPQU CITATION HCIFZ M7S PHGZM PHGZT PQGLB PTHSS S0W 7TB 8FD FR3 KR7
ID	FETCH-LOGICAL-c412t-f25e3d066d19e4a1e5ee5421e7550c8b9ceeaa3974f9af0e5fc690c5a0d0c7f3
IEDL.DBID	RSV
ISICitedReferencesCount	148
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000414427400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1389-4420
IngestDate	Thu Sep 25 00:55:26 EDT 2025 Sat Nov 29 01:39:08 EST 2025 Tue Nov 18 21:39:34 EST 2025 Fri Feb 21 02:33:03 EST 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	4
Keywords	Optimization algorithms Benchmarking Algorithm comparison Software Metric Timing Performance Guidelines Testing
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c412t-f25e3d066d19e4a1e5ee5421e7550c8b9ceeaa3974f9af0e5fc690c5a0d0c7f3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
OpenAccessLink	http://hdl.handle.net/2429/71039
PQID	2063567116
PQPubID	326265
PageCount	34
ParticipantIDs	proquest_journals_2063567116 crossref_primary_10_1007_s11081_017_9366_1 crossref_citationtrail_10_1007_s11081_017_9366_1 springer_journals_10_1007_s11081_017_9366_1
PublicationCentury	2000
PublicationDate	2017-12-01
PublicationDateYYYYMMDD	2017-12-01
PublicationDate_xml	– month: 12 year: 2017 text: 2017-12-01 day: 01
PublicationDecade	2010
PublicationPlace	New York
PublicationPlace_xml	– name: New York – name: Dordrecht
PublicationSubtitle	International Multidisciplinary Journal to Promote Optimization Theory & Applications in Engineering Sciences
PublicationTitle	Optimization and engineering
PublicationTitleAbbrev	Optim Eng
PublicationYear	2017
Publisher	Springer US Springer Nature B.V
Publisher_xml	– name: Springer US – name: Springer Nature B.V
References	Khompatraporn, Pinter, Zabinsky (CR83) 2005; 31 Jackson, Boggs, Nash, Powell (CR80) 1990; 49 Hare, Planiden (CR65) 2014; 8 Birattari (CR24) 2009 Eason, Mulvey (CR46) 1982 Audet, Dang, Orban (CR5) 2014; 6 Dembo (CR38) 1978; 26 Hough, Kolda, Torczon (CR75) 2001; 23 CR33 CR32 Koch, Achterberg, Andersen, Bastert, Berthold, Bixby, Danna, Gamrath, Gleixner, Heinz, Lodi, Mittelmann, Ralphs, Salvagnin, Steffy, Wolter (CR85) 2011; 3 Lenard, Minkoff (CR92) 1984; 10 Dixon, Szegö (CR40) 1978 Andrei (CR3) 2008; 10 Mulvey (CR101) 1982 Crowder, Dembo, Mulvey (CR35) 1979; 5 Barr, Hickman (CR13) 1993; 5 Opara, Arabas (CR109) 2011; 4 Nash, Nocedal (CR103) 1991; 1 Kvasov, Sergeyev (CR89) 2015; 80 Hutter, Hoos, Leyton-Brown, Lodi, Milano, Toth (CR78) 2010 CR48 Gould, Scott (CR59) 2016; 43 Nell, Fawcett, Hoos, Leyton-Brown, Coello (CR104) 2011 Vaz, Vicente (CR146) 2007; 39 Ridge, Kudenko, Bartz-Beielstein, Chiarandini, Paquete, Preuss (CR120) 2010 Tufte, Graves-Morris (CR140) 1983 CR43 Schittkowski, Stoer (CR129) 1978; 31 CR42 Famularo, Pugliese, Sergeyev, Dzemyda, Šaltenis, Žilinskas (CR49) 2002 LaMarca, Ladner (CR91) 1996; 1 Hare, Koch, Lucet (CR64) 2011; 215 Golden, Stewart, Lawler, Lenstra, Rinnooy Kan, Shmoys (CR57) 1985 Regis, Shoemaker (CR117) 2007; 19 Hansen, Jaumard, Lu (CR63) 1992; 55 Ng, Li (CR107) 2014; 51 Sergeyev, Kvasov (CR133) 2015; 21 Hutter, Hoos, Leyton-Brown, Stützle (CR79) 2009; 36 Audet, Dang, Orban, Ken, Keita, John, Reiji (CR6) 2010 McGeoch (CR94) 1996; 8 Sandgren, Ragsdell (CR125) 1980; 102 Rios, Sahinidis (CR122) 2013; 56 Gould, Orban, Toint (CR60) 2015; 60 Kvasov, Mukhametzhanov (CR88) 2016; 1738 Strongin, Sergeyev (CR137) 2000 Tedford, Martins (CR139) 2010; 11 Hillstrom (CR69) 1977; 3 Kortelainen, Lesinski, Moré, Nazarewicz, Sarich, Schunck, Stoitsov, Wild (CR87) 2010; 82 Regis, Wild (CR118) 2017; 32 CR55 Gould, Orban, Toint (CR58) 2003; 29 Hoffman, Mannos, Sokolowsky, Wiegmann (CR74) 1953; 1 Mittelmann, Pruessner (CR98) 2006; 21 Gillard, Kvasov (CR56) 2017; 10 Zhigljavsky, Žilinskas (CR149) 2008 Billups, Dirkse, Ferris (CR23) 1997; 7 McGeoch, Pardalos, Romeijn (CR95) 2002 Rijckaert, Martens (CR121) 1978; 26 Beltrami, Beckmann, Künzi (CR19) 1969 Box (CR28) 1966; 9 Hock, Schittkowski (CR72) 1983; 30 Parejo, Ruiz-Cortés, Lozano, Fernandez (CR110) 2012; 16 Asaadi (CR4) 1973; 4 Dembo (CR37) 1976; 10 Vanderbei, Shanno (CR145) 1999; 13 CR68 Kolda, Lewis, Torczon (CR86) 2003; 45 Bard (CR11) 1970; 7 Derigs (CR39) 1985; 33 Floudas, Pardalos (CR50) 1990 Balint, Gall, Kapler, Retz (CR10) 2010; 7 Benson, Shanno, Vanderbei, Di Pillo, Murli (CR20) 2003 Dannenbring (CR36) 1977; 23 Jamil, Yang (CR81) 2013; 4 Hare, Sagastizábal (CR67) 2010; 20 Hock, Schittkowski (CR71) 1981 Schoen (CR131) 1993; 3 Zhang (CR148) 2014; 146 Bussieck, Drud, Meeraus, Pruessner, Bliek, Jermann, Neumaier (CR31) 2003 Rardin, Uzsoy (CR116) 2001; 7 Tukey (CR141) 1977 Barr, Golden, Kelly, Resende, Stewart (CR12) 1995; 1 Bartz-Beielstein, Preuss, Borenstein, Moraglio (CR15) 2014 Gaviano, Kvasov, Lera, Sergeyev (CR54) 2003; 29 Fowler, Reese, Kees, Dennis, Kelley, Miller, Audet, Booker, Couture, Darwin, Farthing, Finkel, Gablonsky, Gray, Kolda (CR53) 2008; 31 Ridge (CR119) 2007 Sergeyev, Kvasov (CR132) 2006; 16 Pintér (CR112) 2007; 38 Bongartz, Conn, Gould, Toint (CR26) 1995; 21 Fourer, Gay, Kernighan (CR52) 2002 Moré, Garbow, Hillstrom (CR99) 1981; 7 Tabak (CR138) 1969; 14 Mittelmann (CR97) 2003; 95 Vanden Berghen, Bersini (CR144) 2005; 181 Grundel, Jeffcoat, Floudas, Pardalos (CR62) 2009 Törn, Žilinskas (CR143) 1989 Sergeyev, Strongin, Lera (CR134) 2013 Ramsin, Wedin (CR115) 1977; 17 Bussieck, Dirkse, Vigerske (CR30) 2014; 59 Benson, Shanno, Vanderbei (CR21) 2004; 99 Schittkowski (CR128) 1980 CR8 CR130 CR9 Hiriart-Urruty, Lemaréchal (CR70) 1993 Ali, Khompatraporn, Zabinsky (CR2) 2005; 31 CR124 CR123 CR82 Pintér, Pardalos, Romeijn (CR114) 2002 Törn, Ali, Viitanen (CR142) 1999; 14 Paulavičius, Sergeyev, Kvasov, Žilinskas (CR111) 2014; 59 Eason, Fenton (CR45) 1974; 96 Elam, Klingman, Mulvey (CR47) 1982 Shcherbina, Neumaier, Sam-Haroud, Vu, Nguyen, Bliek, Jermann, Neumaier (CR136) 2003 Sergeyev, Kvasov, Mukhametzhanov (CR135) 2016; 141 Grishagin, Floudas, Pardalos (CR61) 2009 Buckley (CR29) 1992; 18 Dolan, Moré (CR41) 2002; 91 Houstis, Rice, Christara, Vavalis, Rice (CR76) 1988 CR18 Huang, Levy (CR77) 1970; 6 CR17 CR16 Schichl, Markót (CR127) 2012; 27 CR14 Knuth (CR84) 1994 Domes, Fuchs, Schichl, Neumaier (CR44) 2014; 15 Floudas, Pardalos, Adjiman, Esposito, Gëmës, Harding, Klepeis, Meyer, Schweiger (CR51) 1999 Sandgren, Ragsdell (CR126) 1980; 102 CR96 Pintér, Kampas (CR113) 2013; 21 Liu, Zhang (CR93) 2000; 16 CR90 Yeniay (CR147) 2005; 2005 Audet, Orban (CR7) 2006; 17 Moré, Wild (CR100) 2009; 20 Berthold (CR22) 2013; 41 CR27 Hare, Sagastizábal (CR66) 2006; 2 CR25 Conn, Gould, Toint (CR34) 1996; 73 CR105 CR102 Neumaier, Shcherbina, Huyer, Vinkó (CR106) 2005; 103 Hoffman, Jackson, Mulvey (CR73) 1982 CR108 Addis, Locatelli (CR1) 2007; 38 ED Eason (9366_CR46) 1982 KE Hillstrom (9366_CR69) 1977; 3 9366_CR102 HD Mittelmann (9366_CR97) 2003; 95 U Derigs (9366_CR39) 1985; 33 WL Hare (9366_CR64) 2011; 215 JJ Moré (9366_CR100) 2009; 20 ML Lenard (9366_CR92) 1984; 10 O Yeniay (9366_CR147) 2005; 2005 K Opara (9366_CR109) 2011; 4 DE Kvasov (9366_CR89) 2015; 80 9366_CR9 9366_CR8 W Hare (9366_CR67) 2010; 20 E Sandgren (9366_CR126) 1980; 102 J Asaadi (9366_CR4) 1973; 4 Y Bard (9366_CR11) 1970; 7 RL Rardin (9366_CR116) 2001; 7 F Domes (9366_CR44) 2014; 15 MJ Box (9366_CR28) 1966; 9 M Gaviano (9366_CR54) 2003; 29 9366_CR68 MM Ali (9366_CR2) 2005; 31 D Famularo (9366_CR49) 2002 K Schittkowski (9366_CR128) 1980 RG Strongin (9366_CR137) 2000 A Törn (9366_CR142) 1999; 14 P Hansen (9366_CR63) 1992; 55 R Paulavičius (9366_CR111) 2014; 59 YD Sergeyev (9366_CR133) 2015; 21 N Gould (9366_CR60) 2015; 60 ER Tufte (9366_CR140) 1983 9366_CR55 ED Dolan (9366_CR41) 2002; 91 DG Dannenbring (9366_CR36) 1977; 23 JA Parejo (9366_CR110) 2012; 16 RS Dembo (9366_CR37) 1976; 10 SC Billups (9366_CR23) 1997; 7 9366_CR82 A Neumaier (9366_CR106) 2005; 103 C-K Ng (9366_CR107) 2014; 51 R Fourer (9366_CR52) 2002 CA Floudas (9366_CR50) 1990 N Andrei (9366_CR3) 2008; 10 JJ Elam (9366_CR47) 1982 O Shcherbina (9366_CR136) 2003 MR Bussieck (9366_CR31) 2003 M Kortelainen (9366_CR87) 2010; 82 JD Pintér (9366_CR114) 2002 RS Dembo (9366_CR38) 1978; 26 C Nell (9366_CR104) 2011 BL Golden (9366_CR57) 1985 J-B Hiriart-Urruty (9366_CR70) 1993 CC McGeoch (9366_CR95) 2002 T Berthold (9366_CR22) 2013; 41 W Hare (9366_CR66) 2006; 2 CA Floudas (9366_CR51) 1999 CC McGeoch (9366_CR94) 1996; 8 H Schichl (9366_CR127) 2012; 27 HD Mittelmann (9366_CR98) 2006; 21 I Bongartz (9366_CR26) 1995; 21 KR Fowler (9366_CR53) 2008; 31 T Koch (9366_CR85) 2011; 3 JJ Moré (9366_CR99) 1981; 7 F Hutter (9366_CR78) 2010 F Schoen (9366_CR131) 1993; 3 HY Huang (9366_CR77) 1970; 6 RS Barr (9366_CR13) 1993; 5 T Bartz-Beielstein (9366_CR15) 2014 D Grundel (9366_CR62) 2009 RG Regis (9366_CR117) 2007; 19 9366_CR27 JD Pintér (9366_CR112) 2007; 38 A Törn (9366_CR143) 1989 W Hock (9366_CR71) 1981 C Audet (9366_CR6) 2010 EN Houstis (9366_CR76) 1988 RS Barr (9366_CR12) 1995; 1 YD Sergeyev (9366_CR135) 2016; 141 9366_CR25 MR Bussieck (9366_CR30) 2014; 59 D Tabak (9366_CR138) 1969; 14 P Hough (9366_CR75) 2001; 23 9366_CR90 E Sandgren (9366_CR125) 1980; 102 9366_CR130 DE Kvasov (9366_CR88) 2016; 1738 S Nash (9366_CR103) 1991; 1 YD Sergeyev (9366_CR132) 2006; 16 TG Kolda (9366_CR86) 2003; 45 MJ Rijckaert (9366_CR121) 1978; 26 N Gould (9366_CR59) 2016; 43 E Ridge (9366_CR120) 2010 C Audet (9366_CR7) 2006; 17 9366_CR16 9366_CR18 9366_CR17 KL Hoffman (9366_CR73) 1982 D Liu (9366_CR93) 2000; 16 9366_CR96 B Addis (9366_CR1) 2007; 38 E Ridge (9366_CR119) 2007 HY Benson (9366_CR21) 2004; 99 JD Pintér (9366_CR113) 2013; 21 AIF Vaz (9366_CR146) 2007; 39 JW Gillard (9366_CR56) 2017; 10 9366_CR14 M Birattari (9366_CR24) 2009 A Zhigljavsky (9366_CR149) 2008 9366_CR123 9366_CR124 HY Benson (9366_CR20) 2003 C Audet (9366_CR5) 2014; 6 RHF Jackson (9366_CR80) 1990; 49 A Balint (9366_CR10) 2010; 7 W Hock (9366_CR72) 1983; 30 EJ Beltrami (9366_CR19) 1969 9366_CR48 AR Conn (9366_CR34) 1996; 73 F Hutter (9366_CR79) 2009; 36 A Hoffman (9366_CR74) 1953; 1 M Jamil (9366_CR81) 2013; 4 9366_CR43 9366_CR42 YD Sergeyev (9366_CR134) 2013 RJ Vanderbei (9366_CR145) 1999; 13 K Schittkowski (9366_CR129) 1978; 31 AG Buckley (9366_CR29) 1992; 18 JW Tukey (9366_CR141) 1977 A LaMarca (9366_CR91) 1996; 1 ED Eason (9366_CR45) 1974; 96 NP Tedford (9366_CR139) 2010; 11 LCW Dixon (9366_CR40) 1978 H Ramsin (9366_CR115) 1977; 17 H Crowder (9366_CR35) 1979; 5 F Vanden Berghen (9366_CR144) 2005; 181 N Gould (9366_CR58) 2003; 29 DE Knuth (9366_CR84) 1994 (9366_CR101) 1982 C Khompatraporn (9366_CR83) 2005; 31 9366_CR32 Z Zhang (9366_CR148) 2014; 146 VA Grishagin (9366_CR61) 2009 9366_CR108 RG Regis (9366_CR118) 2017; 32 9366_CR105 9366_CR33 W Hare (9366_CR65) 2014; 8 L Rios (9366_CR122) 2013; 56
References_xml	– year: 2013 ident: CR134 publication-title: Introduction to global optimization exploiting space-filling curves. Springer briefs in optimization doi: 10.1007/978-1-4614-8042-6 – year: 1978 ident: CR40 publication-title: Towards global optimisation 2 – volume: 15 start-page: 443 issue: 2 year: 2014 end-page: 468 ident: CR44 article-title: The optimization test environment publication-title: Optim Eng doi: 10.1007/s11081-013-9234-6 – start-page: 600 year: 2011 end-page: 615 ident: CR104 article-title: HAL: a framework for the automated analysis and design of high-performance algorithms publication-title: Learning and intelligent optimization. Lecture notes in computer science doi: 10.1007/978-3-642-25566-3_47 – ident: CR68 – ident: CR16 – year: 1977 ident: CR141 publication-title: Exploratory data analysis – volume: 60 start-page: 545 issue: 3 year: 2015 end-page: 557 ident: CR60 article-title: CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization publication-title: Comput Optim Appl doi: 10.1007/s10589-014-9687-3 – start-page: 93 year: 2002 end-page: 109 ident: CR49 article-title: Test problems for Lipschitz univariate global optimization with multiextremal constraints publication-title: Stochastic and global optimization. Nonconvex optimization and its applications doi: 10.1007/0-306-47648-7_6 – volume: 16 start-page: 527 issue: 3 year: 2012 end-page: 561 ident: CR110 article-title: Metaheuristic optimization frameworks: a survey and benchmarking publication-title: Soft Comput doi: 10.1007/s00500-011-0754-8 – volume: 215 start-page: 470 issue: 2 year: 2011 end-page: 480 ident: CR64 article-title: Models and algorithms to improve earthwork operations in road design using mixed integer linear programming publication-title: Eur J Oper Res doi: 10.1016/j.ejor.2011.06.011 – volume: 29 start-page: 373 issue: 4 year: 2003 end-page: 394 ident: CR58 article-title: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited publication-title: ACM Trans Math Softw doi: 10.1145/962437.962439 – volume: 1 start-page: 4 year: 1996 ident: CR91 article-title: The influence of caches on the performance of heaps publication-title: J Exp Algorithmics doi: 10.1145/235141.235145 – year: 2009 ident: CR24 publication-title: Tuning metaheuristics: a machine learning perspective doi: 10.1007/978-3-642-00483-4 – year: 2002 ident: CR52 publication-title: AMPL: a modeling language for mathematical programming – volume: 56 start-page: 1247 issue: 3 year: 2013 end-page: 1293 ident: CR122 article-title: Derivative-free optimization: a review of algorithms and comparison of software implementations publication-title: J Glob Optim doi: 10.1007/s10898-012-9951-y – ident: CR8 – start-page: 265 year: 2010 end-page: 286 ident: CR120 article-title: Tuning an algorithm using design of experiments publication-title: Experimental methods for the analysis of optimization algorithms doi: 10.1007/978-3-642-02538-9_11 – volume: 43 start-page: 15:1 issue: 2 year: 2016 end-page: 15:5 ident: CR59 article-title: A note on performance profiles for benchmarking software publication-title: ACM Trans Math Softw doi: 10.1145/2950048 – volume: 31 start-page: 431 issue: 4 year: 1978 end-page: 463 ident: CR129 article-title: A factorization method for the solution of constrained linear least squares problems allowing subsequent data changes publication-title: Numer Math doi: 10.1007/BF01404569 – ident: CR25 – year: 1983 ident: CR140 publication-title: The visual display of quantitative information – ident: CR42 – start-page: 123 year: 1988 end-page: 155 ident: CR76 article-title: Performance of scientific software publication-title: Mathematical aspects of scientific software. The IMA volumes in mathematics and its applications doi: 10.1007/978-1-4684-7074-1_6 – year: 1982 ident: CR101 publication-title: Evaluating mathematical programming techniques – volume: 181 start-page: 157 issue: 1 year: 2005 end-page: 175 ident: CR144 article-title: CONDOR, a new parallel, constrained extension of powell’s UOBYQA algorithm: experimental results and comparison with the DFO algorithm publication-title: J Comput Appl Math doi: 10.1016/j.cam.2004.11.029 – volume: 21 start-page: 133 issue: 1 year: 2013 end-page: 162 ident: CR113 article-title: Benchmarking nonlinear optimization software in technical computing environments publication-title: Top doi: 10.1007/s11750-011-0209-5 – volume: 102 start-page: 547 issue: 3 year: 1980 end-page: 551 ident: CR126 article-title: The utility of nonlinear programming algorithms: a comparative study, part 2 publication-title: J Mech Des doi: 10.1115/1.3254783 – start-page: 20 year: 1969 end-page: 29 ident: CR19 article-title: A comparison of some recent iterative methods for the numerical solution of nonlinear programs publication-title: Computing methods in optimization problems. Lecture notes in operations research and mathematical economics doi: 10.1007/978-3-642-85974-8_3 – volume: 21 start-page: 123 issue: 1 year: 1995 end-page: 160 ident: CR26 article-title: CUTE: constrained and unconstrained testing environment publication-title: ACM Trans Math Softw doi: 10.1145/200979.201043 – year: 1980 ident: CR128 publication-title: Nonlinear programming codes: information, tests, performance. Lecture notes in economics and mathematical systems doi: 10.1007/978-3-642-46424-9 – start-page: 95 year: 2003 end-page: 127 ident: CR20 article-title: A comparative study of large-scale nonlinear optimization algorithms publication-title: High performance algorithms and software for nonlinear optimization. Applied optimization doi: 10.1007/978-1-4613-0241-4_5 – volume: 29 start-page: 469 issue: 4 year: 2003 end-page: 480 ident: CR54 article-title: Algorithm 829: software for generation of classes of test functions with known local and global minima for global optimization publication-title: ACM Trans Math Softw doi: 10.1145/962437.962444 – start-page: 489 year: 2002 end-page: 513 ident: CR95 article-title: Experimental analysis of algorithms publication-title: Handbook of global optimization. Nonconvex optimization and its applications doi: 10.1007/978-1-4757-5362-2_14 – volume: 51 start-page: 338 year: 2014 end-page: 349 ident: CR107 article-title: Test problem generator for unconstrained global optimization publication-title: Comput Oper Res doi: 10.1016/j.cor.2014.06.019 – volume: 7 start-page: 77 year: 2010 end-page: 82 ident: CR10 article-title: Experiment design and administration for computer clusters for SAT-solvers (EDACC), system description publication-title: J Satisf Boolean Model Comput – volume: 26 start-page: 149 issue: 2 year: 1978 end-page: 183 ident: CR38 article-title: Current state of the art of algorithms and computer software for geometric programming publication-title: J Optim Theory Appl doi: 10.1007/BF00933402 – volume: 33 start-page: 1024 issue: 5 year: 1985 end-page: 1049 ident: CR39 article-title: Using confidence limits for the global optimum in combinatorial optimization publication-title: Oper Res doi: 10.1287/opre.33.5.1024 – volume: 7 start-page: 261 issue: 3 year: 2001 end-page: 304 ident: CR116 article-title: Experimental evaluation of heuristic optimization algorithms: a tutorial publication-title: J Heuristics doi: 10.1023/A:1011319115230 – volume: 7 start-page: 157 issue: 1 year: 1970 end-page: 186 ident: CR11 article-title: Comparison of gradient methods for the solution of nonlinear parameter estimation problems publication-title: SIAM J Numer Anal doi: 10.1137/0707011 – start-page: 211 year: 2003 end-page: 222 ident: CR136 article-title: Benchmarking global optimization and constraint satisfaction codes publication-title: Global optimization and constraint satisfaction. Lecture notes in computer science doi: 10.1007/978-3-540-39901-8_16 – volume: 39 start-page: 197 issue: 2 year: 2007 end-page: 219 ident: CR146 article-title: A particle swarm pattern search method for bound constrained global optimization publication-title: J Glob Optim doi: 10.1007/s10898-007-9133-5 – ident: CR18 – volume: 16 start-page: 229 issue: 3 year: 2000 end-page: 243 ident: CR93 article-title: Test problem generator by neural network for algorithms that try solving nonlinear programming problems globally publication-title: J Glob Optim doi: 10.1023/A:1008306323448 – volume: 19 start-page: 497 issue: 4 year: 2007 end-page: 509 ident: CR117 article-title: A stochastic radial basis function method for the global optimization of expensive functions publication-title: INFORMS J Comput doi: 10.1287/ijoc.1060.0182 – volume: 55 start-page: 273 issue: 1–3 year: 1992 end-page: 292 ident: CR63 article-title: Global optimization of univariate Lipschitz functions: II. New algorithms and computational comparison publication-title: Math Program doi: 10.1007/BF01581203 – volume: 141 start-page: 96 year: 2016 end-page: 109 ident: CR135 article-title: Operational zones for comparing metaheuristic and deterministic one-dimensional global optimization algorithms publication-title: Math Comput Simul doi: 10.1016/j.matcom.2016.05.006 – ident: CR33 – volume: 45 start-page: 385 issue: 3 year: 2003 end-page: 482 ident: CR86 article-title: Optimization by direct search: new perspectives on some classical and modern methods publication-title: SIAM Rev doi: 10.1137/S003614450242889 – volume: 32 start-page: 1 issue: 3 year: 2017 end-page: 29 ident: CR118 article-title: onorbit: constrained optimization by radial basis function interpolation in trust regions publication-title: Optim Methods Softw doi: 10.1080/10556788.2016.1226305 – start-page: 16 year: 1982 end-page: 23 ident: CR47 article-title: NETGEN-II: a system for generating structured network-based mathematical programming test problems publication-title: Evaluating mathematical programming techniques. Lecture notes in economics and mathematical systems doi: 10.1007/978-3-642-95406-1_3 – volume: 13 start-page: 231 issue: 1–3 year: 1999 end-page: 252 ident: CR145 article-title: An interior-point algorithm for nonconvex nonlinear programming publication-title: Comput Optim Appl doi: 10.1023/A:1008677427361 – volume: 7 start-page: 17 issue: 1 year: 1981 end-page: 41 ident: CR99 article-title: Testing unconstrained optimization software publication-title: ACM Trans Math Softw doi: 10.1145/355934.355936 – volume: 10 start-page: 192 issue: 1 year: 1976 end-page: 213 ident: CR37 article-title: A set of geometric programming test problems and their solutions publication-title: Math Program doi: 10.1007/BF01580667 – start-page: 60 year: 1982 end-page: 71 ident: CR46 article-title: Evidence of fundamental difficulties in nonlinear optimization code comparisons publication-title: Evaluating mathematical programming techniques. Lecture notes in economics and mathematical systems doi: 10.1007/978-3-642-95406-1_7 – ident: CR27 – volume: 96 start-page: 196 issue: 1 year: 1974 end-page: 200 ident: CR45 article-title: A comparison of numerical optimization methods for engineering design publication-title: J Manuf Sci Eng – ident: CR123 – ident: CR108 – volume: 146 start-page: 77 issue: 1–2 year: 2014 end-page: 96 ident: CR148 article-title: Sobolev seminorm of quadratic functions with applications to derivative-free optimization publication-title: Math Program doi: 10.1007/s10107-013-0679-3 – volume: 3 start-page: 103 issue: 2 year: 2011 end-page: 163 ident: CR85 article-title: MIPLIB 2010 publication-title: Math Program Comput doi: 10.1007/s12532-011-0025-9 – start-page: 207 year: 1985 end-page: 249 ident: CR57 article-title: Empirical analysis of heuristics publication-title: The traveling salesman problem: a guided tour of combinatorial optimization – start-page: 177 year: 1982 end-page: 199 ident: CR73 article-title: In pursuit of a methodology for testing mathematical programming software publication-title: Evaluating mathematical programming techniques. Lecture notes in economics and mathematical systems doi: 10.1007/978-3-642-95406-1_19 – volume: 18 start-page: 375 issue: 4 year: 1992 end-page: 391 ident: CR29 article-title: Algorithm 709: testing algorithm implementations publication-title: ACM Trans Math Softw doi: 10.1145/138351.138378 – volume: 30 start-page: 335 issue: 4 year: 1983 end-page: 358 ident: CR72 article-title: A comparative performance evaluation of 27 nonlinear programming codes publication-title: Computing doi: 10.1007/BF02242139 – volume: 49 start-page: 413 issue: 1–3 year: 1990 end-page: 425 ident: CR80 article-title: Guidelines for reporting results of computational experiments. Report of the ad hoc committee publication-title: Math Program doi: 10.1007/BF01588801 – ident: CR55 – volume: 8 start-page: 1 issue: 1 year: 1996 end-page: 15 ident: CR94 article-title: Toward an experimental method for algorithm simulation publication-title: INFORMS J Comput doi: 10.1287/ijoc.8.1.1 – year: 2007 ident: CR119 publication-title: Design of experiments for the tuning of optimisation algorithms – volume: 3 start-page: 305 issue: 4 year: 1977 end-page: 315 ident: CR69 article-title: A simulation test approach to the evaluation of nonlinear optimization algorithms publication-title: ACM Trans Math Softw doi: 10.1145/355759.355760 – year: 1981 ident: CR71 publication-title: Test examples for nonlinear programming codes. Lecture notes in economics and mathematical systems doi: 10.1007/978-3-642-48320-2 – start-page: 255 year: 2010 end-page: 274 ident: CR6 article-title: Algorithmic parameter optimization of the DFO method with the OPAL framework publication-title: Software automatic tuning – start-page: 198 year: 2009 end-page: 206 ident: CR61 article-title: Operating characteristics of some global search algorithms publication-title: Problems of stochastic search – volume: 21 start-page: 99 issue: 1–3 year: 2015 end-page: 111 ident: CR133 article-title: A deterministic global optimization using smooth diagonal auxiliary functions publication-title: Commun Nonlinear Sci Numer Simul doi: 10.1016/j.cnsns.2014.08.026 – ident: CR102 – volume: 1 start-page: 17 issue: 1 year: 1953 end-page: 33 ident: CR74 article-title: Computational experience in solving linear programs publication-title: J Soc Ind Appl Math doi: 10.1137/0101002 – volume: 10 start-page: 147 issue: 1 year: 2008 end-page: 161 ident: CR3 article-title: An unconstrained optimization test functions collection publication-title: Adv Model Optim – year: 2000 ident: CR137 publication-title: Global optimization with non-convex constraints: sequential and parallel algorithms doi: 10.1007/978-1-4615-4677-1 – volume: 6 start-page: 269 issue: 3 year: 1970 end-page: 282 ident: CR77 article-title: Numerical experiments on quadratically convergent algorithms for function minimization publication-title: J Optim Theory Appl doi: 10.1007/BF00926604 – volume: 91 start-page: 201 year: 2002 end-page: 213 ident: CR41 article-title: Benchmarking optimization software with performance profiles publication-title: Math Program doi: 10.1007/s101070100263 – volume: 5 start-page: 2 issue: 1 year: 1993 end-page: 18 ident: CR13 article-title: Reporting computational experiments with parallel algorithms: Issues, measures, and experts’ opinions publication-title: INFORMS J Comput doi: 10.1287/ijoc.5.1.2 – volume: 7 start-page: 3 issue: 1 year: 1997 end-page: 25 ident: CR23 article-title: A comparison of large scale mixed complementarity problem solvers publication-title: Comput Optim Appl doi: 10.1023/A:1008632215341 – volume: 5 start-page: 193 issue: 2 year: 1979 end-page: 203 ident: CR35 article-title: On reporting computational experiments with mathematical software publication-title: ACM Trans Math Softw doi: 10.1145/355826.355833 – volume: 14 start-page: 437 issue: 4 year: 1999 end-page: 447 ident: CR142 article-title: Stochastic global optimization: problem classes and solution techniques publication-title: J Glob Optim doi: 10.1023/A:1008395408187 – volume: 26 start-page: 205 issue: 2 year: 1978 end-page: 242 ident: CR121 article-title: Comparison of generalized geometric programming algorithms publication-title: J Optim Theory Appl doi: 10.1007/BF00933404 – volume: 99 start-page: 35 year: 2004 end-page: 48 ident: CR21 article-title: Interior-point methods for nonconvex nonlinear programming: jamming and numerical testing publication-title: Math Progr doi: 10.1007/s10107-003-0418-2 – volume: 20 start-page: 2442 issue: 5 year: 2010 end-page: 2473 ident: CR67 article-title: A redistributed proximal bundle method for nonconvex optimization publication-title: SIAM J Optim doi: 10.1137/090754595 – start-page: 515 year: 2002 end-page: 569 ident: CR114 article-title: Global optimization: software, test problems, and applications publication-title: Handbook of global optimization. Nonconvex optimization and its applications doi: 10.1007/978-1-4757-5362-2_15 – volume: 31 start-page: 743 issue: 5 year: 2008 end-page: 757 ident: CR53 article-title: Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems publication-title: Adv Water Resour doi: 10.1016/j.advwatres.2008.01.010 – volume: 8 start-page: 849 issue: 3 year: 2014 end-page: 860 ident: CR65 article-title: The NC-proximal average for multiple functions publication-title: Optim Lett doi: 10.1007/s11590-013-0641-6 – volume: 1738 start-page: 400012 issue: 1 year: 2016 ident: CR88 article-title: One-dimensional global search: nature-inspired vs. lipschitz methods publication-title: AIP Conf Proc doi: 10.1063/1.4952200 – ident: CR96 – start-page: 205 year: 2014 end-page: 245 ident: CR15 article-title: Experimental analysis of optimization algorithms: tuning and beyond publication-title: Theory and principled methods for the design of metaheuristics. Natural computing series doi: 10.1007/978-3-642-33206-7_10 – volume: 17 start-page: 642 issue: 3 year: 2006 end-page: 664 ident: CR7 article-title: Finding optimal algorithmic parameters using derivative-free optimization publication-title: SIAM J Optim doi: 10.1137/040620886 – year: 1994 ident: CR84 publication-title: The Stanford GraphBase: a platform for combinatorial computing – volume: 38 start-page: 479 issue: 3 year: 2007 end-page: 501 ident: CR1 article-title: A new class of test functions for global optimization publication-title: J Glob Optim doi: 10.1007/s10898-006-9099-8 – ident: CR9 – ident: CR32 – volume: 103 start-page: 335 issue: 2 year: 2005 end-page: 356 ident: CR106 article-title: A comparison of complete global optimization solvers publication-title: Math Program doi: 10.1007/s10107-005-0585-4 – volume: 31 start-page: 613 issue: 4 year: 2005 end-page: 633 ident: CR83 article-title: Comparative assessment of algorithms and software for global optimization publication-title: J Glob Optim doi: 10.1007/s10898-004-9971-3 – volume: 10 start-page: 59 year: 2017 end-page: 70 ident: CR56 article-title: Lipschitz optimization methods for fitting a sum of damped sinusoids to a series of observations publication-title: Stat Interface doi: 10.4310/SII.2017.v10.n1.a6 – volume: 2005 start-page: 165 issue: 2 year: 2005 end-page: 173 ident: CR147 article-title: A comparative study on optimization methods for the constrained nonlinear programming problems publication-title: Math Probl Eng doi: 10.1155/MPE.2005.165 – ident: CR105 – volume: 102 start-page: 540 issue: 3 year: 1980 end-page: 546 ident: CR125 article-title: The utility of nonlinear programming algorithms: a comparative study, part 1 publication-title: J Mech Des doi: 10.1115/1.3254782 – volume: 17 start-page: 72 issue: 1 year: 1977 end-page: 90 ident: CR115 article-title: A comparison of some algorithms for the nonlinear least squares problem publication-title: BIT Numer Math doi: 10.1007/BF01932400 – volume: 16 start-page: 910 issue: 3 year: 2006 end-page: 937 ident: CR132 article-title: Global search based on efficient diagonal partitions and a set of Lipschitz constants publication-title: SIAM J Optim doi: 10.1137/040621132 – volume: 23 start-page: 134 issue: 1 year: 2001 end-page: 156 ident: CR75 article-title: Asynchronous parallel pattern search for nonlinear optimization publication-title: SIAM J Sci Comput doi: 10.1137/S1064827599365823 – ident: CR43 – volume: 27 start-page: 359 issue: 2 year: 2012 end-page: 372 ident: CR127 article-title: Algorithmic differentiation techniques for global optimization in the COCONUT environment publication-title: Optim Methods Softw doi: 10.1080/10556788.2010.547581 – year: 1999 ident: CR51 publication-title: Handbook of test problems in local and global optimization doi: 10.1007/978-1-4757-3040-1 – volume: 4 start-page: 73 year: 2011 end-page: 80 ident: CR109 article-title: Benchmarking procedures for continuous optimization algorithms publication-title: J Telecommun Inf Technol – volume: 36 start-page: 267 issue: 1 year: 2009 end-page: 306 ident: CR79 article-title: ParamILS: an automatic algorithm configuration framework publication-title: J Artif Intell Res – volume: 41 start-page: 611 issue: 6 year: 2013 end-page: 614 ident: CR22 article-title: Measuring the impact of primal heuristics publication-title: Oper Res Lett doi: 10.1016/j.orl.2013.08.007 – ident: CR14 – volume: 4 start-page: 144 issue: 1 year: 1973 end-page: 154 ident: CR4 article-title: A computational comparison of some non-linear programs publication-title: Math Program doi: 10.1007/BF01584657 – volume: 6 start-page: 233 issue: 3 year: 2014 end-page: 254 ident: CR5 article-title: Optimization of algorithms with OPAL publication-title: Math Program Comput doi: 10.1007/s12532-014-0067-x – ident: CR82 – volume: 10 start-page: 86 issue: 1 year: 1984 end-page: 96 ident: CR92 article-title: Randomly generated test problems for positive definite quadratic programming publication-title: ACM Trans Math Softw doi: 10.1145/356068.356075 – volume: 9 start-page: 67 issue: 1 year: 1966 end-page: 77 ident: CR28 article-title: A comparison of several current optimization methods, and the use of transformations in constrained problems publication-title: Comput J doi: 10.1093/comjnl/9.1.67 – volume: 59 start-page: 259 issue: 2–3 year: 2014 end-page: 275 ident: CR30 article-title: PAVER 2.0: an open source environment for automated performance analysis of benchmarking data publication-title: J Glob Optim doi: 10.1007/s10898-013-0131-5 – start-page: 186 year: 2010 end-page: 202 ident: CR78 article-title: Automated configuration of mixed integer programming solvers publication-title: Integration of AI and OR techniques in constraint programming for combinatorial optimization problems. Lecture notes in computer science doi: 10.1007/978-3-642-13520-0_23 – volume: 3 start-page: 133 issue: 2 year: 1993 end-page: 137 ident: CR131 article-title: A wide class of test functions for global optimization publication-title: J Glob Optim doi: 10.1007/BF01096734 – volume: 23 start-page: 1273 issue: 12 year: 1977 end-page: 1283 ident: CR36 article-title: Procedures for estimating optimal solution values for large combinatorial problems publication-title: Manag Sci doi: 10.1287/mnsc.23.12.1273 – volume: 14 start-page: 572 issue: 5 year: 1969 end-page: 572 ident: CR138 article-title: Comparative study of various minimization techniques used in mathematical programming publication-title: IEEE Trans Autom Control doi: 10.1109/TAC.1969.1099262 – year: 1990 ident: CR50 publication-title: A collection of test problems for constrained global optimization algorithms – year: 2008 ident: CR149 publication-title: Stochastic global optimization. Springer optimization and its applications – volume: 38 start-page: 79 issue: 1 year: 2007 end-page: 101 ident: CR112 article-title: Nonlinear optimization with GAMS /LGO publication-title: J Glob Optim doi: 10.1007/s10898-006-9084-2 – ident: CR124 – ident: CR48 – volume: 31 start-page: 635 issue: 4 year: 2005 end-page: 672 ident: CR2 article-title: A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems publication-title: J Glob Optim doi: 10.1007/s10898-004-9972-2 – ident: CR90 – volume: 21 start-page: 105 issue: 1 year: 2006 end-page: 120 ident: CR98 article-title: A server for automated performance analysis of benchmarking data publication-title: Optim Methods Softw doi: 10.1080/10556780500065366 – volume: 73 start-page: 73 issue: 1 year: 1996 end-page: 110 ident: CR34 article-title: Numerical experiments with the LANCELOT package (release A) for large-scale nonlinear optimization publication-title: Math Program doi: 10.1007/BF02592099 – ident: CR130 – ident: CR17 – volume: 95 start-page: 407 issue: 2 year: 2003 end-page: 430 ident: CR97 article-title: An independent benchmarking of SDP and SOCP solvers publication-title: Math Program doi: 10.1007/s10107-002-0355-5 – volume: 20 start-page: 172 issue: 1 year: 2009 end-page: 191 ident: CR100 article-title: Benchmarking derivative-free optimization algorithms publication-title: SIAM J Optim doi: 10.1137/080724083 – volume: 80 start-page: 58 year: 2015 end-page: 66 ident: CR89 article-title: Deterministic approaches for solving practical black-box global optimization problems publication-title: Adv Eng Softw doi: 10.1016/j.advengsoft.2014.09.014 – start-page: 223 year: 2003 end-page: 238 ident: CR31 article-title: Quality assurance and global optimization publication-title: Global optimization and constraint satisfaction. Lecture notes in computer science doi: 10.1007/978-3-540-39901-8_17 – volume: 4 start-page: 150 issue: 2 year: 2013 end-page: 194 ident: CR81 article-title: A literature survey of benchmark functions for global optimisation problems publication-title: Int J Math Model Numer Optim – volume: 11 start-page: 159 issue: 1 year: 2010 end-page: 183 ident: CR139 article-title: Benchmarking multidisciplinary design optimization algorithms publication-title: Optim Eng doi: 10.1007/s11081-009-9082-6 – start-page: 391 year: 2009 end-page: 394 ident: CR62 article-title: Combinatorial test problems and problem generators publication-title: Encyclopedia of optimization doi: 10.1007/978-0-387-74759-0_71 – volume: 2 start-page: 545 issue: 3 year: 2006 end-page: 573 ident: CR66 article-title: Benchmark of some nonsmooth optimization solvers for computing nonconvex proximal points publication-title: Pac J Optim – volume: 1 start-page: 358 issue: 3 year: 1991 end-page: 372 ident: CR103 article-title: A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization publication-title: SIAM J Optim doi: 10.1137/0801023 – volume: 82 start-page: 024313 issue: 2 year: 2010 ident: CR87 article-title: Nuclear energy density optimization publication-title: Phys Rev C doi: 10.1103/PhysRevC.82.024313 – year: 1989 ident: CR143 publication-title: Global optimization. Lecture notes in computer science – volume: 59 start-page: 545 issue: 2–3 year: 2014 end-page: 567 ident: CR111 article-title: Globally-biased Disimpl algorithm for expensive global optimization publication-title: J Glob Optim doi: 10.1007/s10898-014-0180-4 – year: 1993 ident: CR70 publication-title: Convex analysis and minimization algorithms. II. Grundlehren der Mathematischen Wissenschaften [Fundamental principles of mathematical sciences]. Advanced theory and bundle methods – volume: 1 start-page: 9 issue: 1 year: 1995 end-page: 32 ident: CR12 article-title: Designing and reporting on computational experiments with heuristic methods publication-title: J Heuristics doi: 10.1007/BF02430363 – volume: 16 start-page: 910 issue: 3 year: 2006 ident: 9366_CR132 publication-title: SIAM J Optim doi: 10.1137/040621132 – volume-title: Global optimization with non-convex constraints: sequential and parallel algorithms year: 2000 ident: 9366_CR137 doi: 10.1007/978-1-4615-4677-1 – volume: 2 start-page: 545 issue: 3 year: 2006 ident: 9366_CR66 publication-title: Pac J Optim – volume-title: Stochastic global optimization. Springer optimization and its applications year: 2008 ident: 9366_CR149 – volume: 1 start-page: 17 issue: 1 year: 1953 ident: 9366_CR74 publication-title: J Soc Ind Appl Math doi: 10.1137/0101002 – ident: 9366_CR42 doi: 10.2172/775270 – start-page: 186 volume-title: Integration of AI and OR techniques in constraint programming for combinatorial optimization problems. Lecture notes in computer science year: 2010 ident: 9366_CR78 doi: 10.1007/978-3-642-13520-0_23 – volume: 38 start-page: 79 issue: 1 year: 2007 ident: 9366_CR112 publication-title: J Glob Optim doi: 10.1007/s10898-006-9084-2 – volume: 8 start-page: 1 issue: 1 year: 1996 ident: 9366_CR94 publication-title: INFORMS J Comput doi: 10.1287/ijoc.8.1.1 – ident: 9366_CR33 – start-page: 16 volume-title: Evaluating mathematical programming techniques. Lecture notes in economics and mathematical systems year: 1982 ident: 9366_CR47 doi: 10.1007/978-3-642-95406-1_3 – volume: 21 start-page: 133 issue: 1 year: 2013 ident: 9366_CR113 publication-title: Top doi: 10.1007/s11750-011-0209-5 – volume: 38 start-page: 479 issue: 3 year: 2007 ident: 9366_CR1 publication-title: J Glob Optim doi: 10.1007/s10898-006-9099-8 – volume: 5 start-page: 2 issue: 1 year: 1993 ident: 9366_CR13 publication-title: INFORMS J Comput doi: 10.1287/ijoc.5.1.2 – ident: 9366_CR68 – ident: 9366_CR16 doi: 10.1109/CEC.2005.1554761 – volume: 26 start-page: 149 issue: 2 year: 1978 ident: 9366_CR38 publication-title: J Optim Theory Appl doi: 10.1007/BF00933402 – volume-title: Towards global optimisation 2 year: 1978 ident: 9366_CR40 – volume: 103 start-page: 335 issue: 2 year: 2005 ident: 9366_CR106 publication-title: Math Program doi: 10.1007/s10107-005-0585-4 – volume: 23 start-page: 134 issue: 1 year: 2001 ident: 9366_CR75 publication-title: SIAM J Sci Comput doi: 10.1137/S1064827599365823 – volume: 19 start-page: 497 issue: 4 year: 2007 ident: 9366_CR117 publication-title: INFORMS J Comput doi: 10.1287/ijoc.1060.0182 – volume: 146 start-page: 77 issue: 1–2 year: 2014 ident: 9366_CR148 publication-title: Math Program doi: 10.1007/s10107-013-0679-3 – volume: 8 start-page: 849 issue: 3 year: 2014 ident: 9366_CR65 publication-title: Optim Lett doi: 10.1007/s11590-013-0641-6 – volume: 215 start-page: 470 issue: 2 year: 2011 ident: 9366_CR64 publication-title: Eur J Oper Res doi: 10.1016/j.ejor.2011.06.011 – volume-title: A collection of test problems for constrained global optimization algorithms year: 1990 ident: 9366_CR50 doi: 10.1007/3-540-53032-0 – volume: 43 start-page: 15:1 issue: 2 year: 2016 ident: 9366_CR59 publication-title: ACM Trans Math Softw doi: 10.1145/2950048 – ident: 9366_CR27 – volume: 21 start-page: 123 issue: 1 year: 1995 ident: 9366_CR26 publication-title: ACM Trans Math Softw doi: 10.1145/200979.201043 – volume: 7 start-page: 261 issue: 3 year: 2001 ident: 9366_CR116 publication-title: J Heuristics doi: 10.1023/A:1011319115230 – volume: 4 start-page: 73 year: 2011 ident: 9366_CR109 publication-title: J Telecommun Inf Technol – volume: 1 start-page: 358 issue: 3 year: 1991 ident: 9366_CR103 publication-title: SIAM J Optim doi: 10.1137/0801023 – volume: 55 start-page: 273 issue: 1–3 year: 1992 ident: 9366_CR63 publication-title: Math Program doi: 10.1007/BF01581203 – volume: 26 start-page: 205 issue: 2 year: 1978 ident: 9366_CR121 publication-title: J Optim Theory Appl doi: 10.1007/BF00933404 – volume: 29 start-page: 373 issue: 4 year: 2003 ident: 9366_CR58 publication-title: ACM Trans Math Softw doi: 10.1145/962437.962439 – ident: 9366_CR105 – volume: 141 start-page: 96 year: 2016 ident: 9366_CR135 publication-title: Math Comput Simul doi: 10.1016/j.matcom.2016.05.006 – start-page: 95 volume-title: High performance algorithms and software for nonlinear optimization. Applied optimization year: 2003 ident: 9366_CR20 doi: 10.1007/978-1-4613-0241-4_5 – volume: 18 start-page: 375 issue: 4 year: 1992 ident: 9366_CR29 publication-title: ACM Trans Math Softw doi: 10.1145/138351.138378 – volume: 73 start-page: 73 issue: 1 year: 1996 ident: 9366_CR34 publication-title: Math Program doi: 10.1007/BF02592099 – ident: 9366_CR82 – volume: 102 start-page: 547 issue: 3 year: 1980 ident: 9366_CR126 publication-title: J Mech Des doi: 10.1115/1.3254783 – ident: 9366_CR14 doi: 10.1145/318371.318618 – start-page: 123 volume-title: Mathematical aspects of scientific software. The IMA volumes in mathematics and its applications year: 1988 ident: 9366_CR76 doi: 10.1007/978-1-4684-7074-1_6 – volume: 20 start-page: 172 issue: 1 year: 2009 ident: 9366_CR100 publication-title: SIAM J Optim doi: 10.1137/080724083 – volume: 17 start-page: 72 issue: 1 year: 1977 ident: 9366_CR115 publication-title: BIT Numer Math doi: 10.1007/BF01932400 – volume: 31 start-page: 635 issue: 4 year: 2005 ident: 9366_CR2 publication-title: J Glob Optim doi: 10.1007/s10898-004-9972-2 – start-page: 93 volume-title: Stochastic and global optimization. Nonconvex optimization and its applications year: 2002 ident: 9366_CR49 doi: 10.1007/0-306-47648-7_6 – volume: 1738 start-page: 400012 issue: 1 year: 2016 ident: 9366_CR88 publication-title: AIP Conf Proc doi: 10.1063/1.4952200 – start-page: 198 volume-title: Problems of stochastic search year: 2009 ident: 9366_CR61 – volume: 7 start-page: 17 issue: 1 year: 1981 ident: 9366_CR99 publication-title: ACM Trans Math Softw doi: 10.1145/355934.355936 – volume: 27 start-page: 359 issue: 2 year: 2012 ident: 9366_CR127 publication-title: Optim Methods Softw doi: 10.1080/10556788.2010.547581 – volume-title: Test examples for nonlinear programming codes. Lecture notes in economics and mathematical systems year: 1981 ident: 9366_CR71 doi: 10.1007/978-3-642-48320-2 – volume-title: Introduction to global optimization exploiting space-filling curves. Springer briefs in optimization year: 2013 ident: 9366_CR134 doi: 10.1007/978-1-4614-8042-6 – start-page: 207 volume-title: The traveling salesman problem: a guided tour of combinatorial optimization year: 1985 ident: 9366_CR57 – volume-title: Nonlinear programming codes: information, tests, performance. Lecture notes in economics and mathematical systems year: 1980 ident: 9366_CR128 doi: 10.1007/978-3-642-46424-9 – volume: 56 start-page: 1247 issue: 3 year: 2013 ident: 9366_CR122 publication-title: J Glob Optim doi: 10.1007/s10898-012-9951-y – volume: 33 start-page: 1024 issue: 5 year: 1985 ident: 9366_CR39 publication-title: Oper Res doi: 10.1287/opre.33.5.1024 – volume: 4 start-page: 144 issue: 1 year: 1973 ident: 9366_CR4 publication-title: Math Program doi: 10.1007/BF01584657 – volume: 1 start-page: 4 year: 1996 ident: 9366_CR91 publication-title: J Exp Algorithmics doi: 10.1145/235141.235145 – volume: 30 start-page: 335 issue: 4 year: 1983 ident: 9366_CR72 publication-title: Computing doi: 10.1007/BF02242139 – volume: 16 start-page: 527 issue: 3 year: 2012 ident: 9366_CR110 publication-title: Soft Comput doi: 10.1007/s00500-011-0754-8 – volume-title: Design of experiments for the tuning of optimisation algorithms year: 2007 ident: 9366_CR119 – volume-title: The visual display of quantitative information year: 1983 ident: 9366_CR140 – volume: 10 start-page: 192 issue: 1 year: 1976 ident: 9366_CR37 publication-title: Math Program doi: 10.1007/BF01580667 – start-page: 489 volume-title: Handbook of global optimization. Nonconvex optimization and its applications year: 2002 ident: 9366_CR95 doi: 10.1007/978-1-4757-5362-2_14 – start-page: 20 volume-title: Computing methods in optimization problems. Lecture notes in operations research and mathematical economics year: 1969 ident: 9366_CR19 doi: 10.1007/978-3-642-85974-8_3 – volume: 96 start-page: 196 issue: 1 year: 1974 ident: 9366_CR45 publication-title: J Manuf Sci Eng – volume: 39 start-page: 197 issue: 2 year: 2007 ident: 9366_CR146 publication-title: J Glob Optim doi: 10.1007/s10898-007-9133-5 – start-page: 600 volume-title: Learning and intelligent optimization. Lecture notes in computer science year: 2011 ident: 9366_CR104 doi: 10.1007/978-3-642-25566-3_47 – ident: 9366_CR90 doi: 10.1016/j.amc.2017.05.014 – ident: 9366_CR130 – volume: 3 start-page: 103 issue: 2 year: 2011 ident: 9366_CR85 publication-title: Math Program Comput doi: 10.1007/s12532-011-0025-9 – volume: 36 start-page: 267 issue: 1 year: 2009 ident: 9366_CR79 publication-title: J Artif Intell Res doi: 10.1613/jair.2861 – volume: 80 start-page: 58 year: 2015 ident: 9366_CR89 publication-title: Adv Eng Softw doi: 10.1016/j.advengsoft.2014.09.014 – volume: 17 start-page: 642 issue: 3 year: 2006 ident: 9366_CR7 publication-title: SIAM J Optim doi: 10.1137/040620886 – start-page: 211 volume-title: Global optimization and constraint satisfaction. Lecture notes in computer science year: 2003 ident: 9366_CR136 doi: 10.1007/978-3-540-39901-8_16 – volume: 60 start-page: 545 issue: 3 year: 2015 ident: 9366_CR60 publication-title: Comput Optim Appl doi: 10.1007/s10589-014-9687-3 – volume: 6 start-page: 269 issue: 3 year: 1970 ident: 9366_CR77 publication-title: J Optim Theory Appl doi: 10.1007/BF00926604 – ident: 9366_CR25 – start-page: 515 volume-title: Handbook of global optimization. Nonconvex optimization and its applications year: 2002 ident: 9366_CR114 doi: 10.1007/978-1-4757-5362-2_15 – volume: 21 start-page: 99 issue: 1–3 year: 2015 ident: 9366_CR133 publication-title: Commun Nonlinear Sci Numer Simul doi: 10.1016/j.cnsns.2014.08.026 – ident: 9366_CR124 – ident: 9366_CR9 doi: 10.2172/79972 – volume: 181 start-page: 157 issue: 1 year: 2005 ident: 9366_CR144 publication-title: J Comput Appl Math doi: 10.1016/j.cam.2004.11.029 – volume: 95 start-page: 407 issue: 2 year: 2003 ident: 9366_CR97 publication-title: Math Program doi: 10.1007/s10107-002-0355-5 – volume-title: Exploratory data analysis year: 1977 ident: 9366_CR141 – ident: 9366_CR17 – volume: 16 start-page: 229 issue: 3 year: 2000 ident: 9366_CR93 publication-title: J Glob Optim doi: 10.1023/A:1008306323448 – volume: 51 start-page: 338 year: 2014 ident: 9366_CR107 publication-title: Comput Oper Res doi: 10.1016/j.cor.2014.06.019 – start-page: 223 volume-title: Global optimization and constraint satisfaction. Lecture notes in computer science year: 2003 ident: 9366_CR31 doi: 10.1007/978-3-540-39901-8_17 – volume: 7 start-page: 3 issue: 1 year: 1997 ident: 9366_CR23 publication-title: Comput Optim Appl doi: 10.1023/A:1008632215341 – volume: 31 start-page: 431 issue: 4 year: 1978 ident: 9366_CR129 publication-title: Numer Math doi: 10.1007/BF01404569 – volume-title: Evaluating mathematical programming techniques year: 1982 ident: 9366_CR101 – volume: 14 start-page: 572 issue: 5 year: 1969 ident: 9366_CR138 publication-title: IEEE Trans Autom Control doi: 10.1109/TAC.1969.1099262 – volume: 91 start-page: 201 year: 2002 ident: 9366_CR41 publication-title: Math Program doi: 10.1007/s101070100263 – volume: 49 start-page: 413 issue: 1–3 year: 1990 ident: 9366_CR80 publication-title: Math Program doi: 10.1007/BF01588801 – volume: 6 start-page: 233 issue: 3 year: 2014 ident: 9366_CR5 publication-title: Math Program Comput doi: 10.1007/s12532-014-0067-x – volume-title: Convex analysis and minimization algorithms. II. Grundlehren der Mathematischen Wissenschaften [Fundamental principles of mathematical sciences]. Advanced theory and bundle methods year: 1993 ident: 9366_CR70 – volume: 21 start-page: 105 issue: 1 year: 2006 ident: 9366_CR98 publication-title: Optim Methods Softw doi: 10.1080/10556780500065366 – volume: 7 start-page: 157 issue: 1 year: 1970 ident: 9366_CR11 publication-title: SIAM J Numer Anal doi: 10.1137/0707011 – ident: 9366_CR43 doi: 10.2172/834714 – volume-title: The Stanford GraphBase: a platform for combinatorial computing year: 1994 ident: 9366_CR84 – volume: 14 start-page: 437 issue: 4 year: 1999 ident: 9366_CR142 publication-title: J Glob Optim doi: 10.1023/A:1008395408187 – volume: 5 start-page: 193 issue: 2 year: 1979 ident: 9366_CR35 publication-title: ACM Trans Math Softw doi: 10.1145/355826.355833 – volume: 10 start-page: 86 issue: 1 year: 1984 ident: 9366_CR92 publication-title: ACM Trans Math Softw doi: 10.1145/356068.356075 – ident: 9366_CR55 – volume: 10 start-page: 147 issue: 1 year: 2008 ident: 9366_CR3 publication-title: Adv Model Optim – volume: 99 start-page: 35 year: 2004 ident: 9366_CR21 publication-title: Math Progr doi: 10.1007/s10107-003-0418-2 – volume: 10 start-page: 59 year: 2017 ident: 9366_CR56 publication-title: Stat Interface doi: 10.4310/SII.2017.v10.n1.a6 – volume: 11 start-page: 159 issue: 1 year: 2010 ident: 9366_CR139 publication-title: Optim Eng doi: 10.1007/s11081-009-9082-6 – volume: 31 start-page: 743 issue: 5 year: 2008 ident: 9366_CR53 publication-title: Adv Water Resour doi: 10.1016/j.advwatres.2008.01.010 – volume: 45 start-page: 385 issue: 3 year: 2003 ident: 9366_CR86 publication-title: SIAM Rev doi: 10.1137/S003614450242889 – start-page: 391 volume-title: Encyclopedia of optimization year: 2009 ident: 9366_CR62 doi: 10.1007/978-0-387-74759-0_71 – volume-title: Global optimization. Lecture notes in computer science year: 1989 ident: 9366_CR143 – volume: 3 start-page: 133 issue: 2 year: 1993 ident: 9366_CR131 publication-title: J Glob Optim doi: 10.1007/BF01096734 – start-page: 205 volume-title: Theory and principled methods for the design of metaheuristics. Natural computing series year: 2014 ident: 9366_CR15 doi: 10.1007/978-3-642-33206-7_10 – start-page: 60 volume-title: Evaluating mathematical programming techniques. Lecture notes in economics and mathematical systems year: 1982 ident: 9366_CR46 doi: 10.1007/978-3-642-95406-1_7 – volume: 9 start-page: 67 issue: 1 year: 1966 ident: 9366_CR28 publication-title: Comput J doi: 10.1093/comjnl/9.1.67 – volume-title: AMPL: a modeling language for mathematical programming year: 2002 ident: 9366_CR52 – ident: 9366_CR96 – volume: 13 start-page: 231 issue: 1–3 year: 1999 ident: 9366_CR145 publication-title: Comput Optim Appl doi: 10.1023/A:1008677427361 – ident: 9366_CR108 – start-page: 255 volume-title: Software automatic tuning year: 2010 ident: 9366_CR6 – volume: 41 start-page: 611 issue: 6 year: 2013 ident: 9366_CR22 publication-title: Oper Res Lett doi: 10.1016/j.orl.2013.08.007 – start-page: 177 volume-title: Evaluating mathematical programming techniques. Lecture notes in economics and mathematical systems year: 1982 ident: 9366_CR73 doi: 10.1007/978-3-642-95406-1_19 – volume-title: Handbook of test problems in local and global optimization year: 1999 ident: 9366_CR51 doi: 10.1007/978-1-4757-3040-1 – volume: 7 start-page: 77 year: 2010 ident: 9366_CR10 publication-title: J Satisf Boolean Model Comput – volume: 29 start-page: 469 issue: 4 year: 2003 ident: 9366_CR54 publication-title: ACM Trans Math Softw doi: 10.1145/962437.962444 – volume: 23 start-page: 1273 issue: 12 year: 1977 ident: 9366_CR36 publication-title: Manag Sci doi: 10.1287/mnsc.23.12.1273 – ident: 9366_CR102 doi: 10.1145/1143997.1144029 – ident: 9366_CR123 – volume-title: Tuning metaheuristics: a machine learning perspective year: 2009 ident: 9366_CR24 doi: 10.1007/978-3-642-00483-4 – volume: 59 start-page: 259 issue: 2–3 year: 2014 ident: 9366_CR30 publication-title: J Glob Optim doi: 10.1007/s10898-013-0131-5 – volume: 15 start-page: 443 issue: 2 year: 2014 ident: 9366_CR44 publication-title: Optim Eng doi: 10.1007/s11081-013-9234-6 – ident: 9366_CR18 – volume: 32 start-page: 1 issue: 3 year: 2017 ident: 9366_CR118 publication-title: Optim Methods Softw doi: 10.1080/10556788.2016.1226305 – ident: 9366_CR8 doi: 10.1007/s10589-014-9708-2 – volume: 20 start-page: 2442 issue: 5 year: 2010 ident: 9366_CR67 publication-title: SIAM J Optim doi: 10.1137/090754595 – start-page: 265 volume-title: Experimental methods for the analysis of optimization algorithms year: 2010 ident: 9366_CR120 doi: 10.1007/978-3-642-02538-9_11 – volume: 82 start-page: 024313 issue: 2 year: 2010 ident: 9366_CR87 publication-title: Phys Rev C doi: 10.1103/PhysRevC.82.024313 – volume: 3 start-page: 305 issue: 4 year: 1977 ident: 9366_CR69 publication-title: ACM Trans Math Softw doi: 10.1145/355759.355760 – volume: 59 start-page: 545 issue: 2–3 year: 2014 ident: 9366_CR111 publication-title: J Glob Optim doi: 10.1007/s10898-014-0180-4 – volume: 2005 start-page: 165 issue: 2 year: 2005 ident: 9366_CR147 publication-title: Math Probl Eng doi: 10.1155/MPE.2005.165 – volume: 1 start-page: 9 issue: 1 year: 1995 ident: 9366_CR12 publication-title: J Heuristics doi: 10.1007/BF02430363 – ident: 9366_CR32 – volume: 4 start-page: 150 issue: 2 year: 2013 ident: 9366_CR81 publication-title: Int J Math Model Numer Optim – volume: 102 start-page: 540 issue: 3 year: 1980 ident: 9366_CR125 publication-title: J Mech Des doi: 10.1115/1.3254782 – ident: 9366_CR48 doi: 10.1007/978-1-4612-5022-7 – volume: 31 start-page: 613 issue: 4 year: 2005 ident: 9366_CR83 publication-title: J Glob Optim doi: 10.1007/s10898-004-9971-3
SSID	ssj0016528
Score	2.530356
Snippet	Comparing, or benchmarking, of optimization algorithms is a complicated task that involves many subtle considerations to yield a fair and unbiased evaluation....
SourceID	proquest crossref springer
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	815
SubjectTerms	Algorithms Benchmarks Control Engineering Environmental Management Financial Engineering Mathematics Mathematics and Statistics Operations Research/Decision Theory Optimization Optimization algorithms Systems Theory
Title	Best practices for comparing optimization algorithms
URI	https://link.springer.com/article/10.1007/s11081-017-9366-1 https://www.proquest.com/docview/2063567116
Volume	18
WOSCitedRecordID	wos000414427400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1573-2924 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0016528 issn: 1389-4420 databaseCode: RSV dateStart: 20000601 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LSwMxEA5SPejBt1itsgdPSiDPfRxVLF4sokV6W5JsooXaSnf19zvZ7kNFBb3tsrMhzCTzYGa-QehEa8Y14QprIiBAUTrGsaPwpA3EW8pkkuty2EQ0GMSjUXJb9XHndbV7nZIsNXXb7EbBfGGvVRMehhhCnmWwdrGf13B3_9CkDkJZDlT1CTgsBGtSmd8t8dkYtR7ml6RoaWv6G__a5SZar1zL4HxxFrbQkp1uo7UPgIPwdtOgtOY7SFzANoK6TyoPwH8NFjXpQBzMQJk8V12agZo8zubj4uk530XD_tXw8hpXUxSwEZQV2DFpeQaeRUYTKxS10lopGLURBCcm1gmYSaXALREuUY5Y6QxEzEYqkhETOb6HOtPZ1O6jQHKpImtio8JMOGI0pY4bFtskIjRLSBeRmpupqRDG_aCLSdpiI3vupMCd1HMnpV102vzysoDX-I24V4sorW5anjLiEfYiSsMuOqtF0n7-cbGDP1EfolXmZVrWsfRQp5i_2iO0Yt6KcT4_Lg_gO_JF1J4
linkProvider	Springer Nature
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LS8QwEA6ignrwLa6u2oMnpZA0SR9HFZcVdxfRRfYWkjTVhX3Itvr7nXT7UFFBby2dhjCTzIOZ-QahU6U8qjCVrsIMAhSpQjdMCDwpDfGW1DGnKh82EfR64WAQ3RV93GlZ7V6mJHNNXTe7ETBfrtWqEfV9F0KeJQYGywLm3z88VqkDn-cDVW0CzmXMq1KZ3y3x2RjVHuaXpGhua1ob_9rlJlovXEvnYn4WttCCmWyjtQ-Ag_DWrVBa0x3ELmEbTtknlTrgvzrzmnQgdqagTMZFl6YjR0_T2TB7Hqe7qN-67l-13WKKgqsZ8TI38bihMXgWMYkMk8RwYzgwzgQQnOhQRWAmpQS3hCWRTLDhiYaIWXOJY6yDhO6hxcl0YvaRwymXgdGhln7MEqwVIQnVXmiiAJM4wg2ES24KXSCM20EXI1FjI1vuCOCOsNwRpIHOql9e5vAavxE3SxGJ4qalwsMWYS8gxG-g81Ik9ecfFzv4E_UJWmn3ux3RuendHqJVz8o3r2lposVs9mqO0LJ-y4bp7Dg_jO_zCteC
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3dS8MwEA8yRfTBb3E6tQ8-KWFJmvTj0a-hqGPgkL2VJE10sHVjrf79Jv1YVVQQ31qahnCX5O7H3f0OgBMhiCuQy6FA1AAULgIYaGyehDR4i8uYuSJvNuF3u8FgEPbKPqdple1ehSSLmgbL0pRk7Wms23XhGzamDNobNnQ9Dxr4s0htHr2F649P8zCCx_LmqjYYBykl87Dmd1N8Nky1t_klQJrbnc76v1e8AdZKl9M5L_bIJlhQyRZY_UBEaN4e5uyt6TagF2ZJTlU_lTrGr3WKXHUz2JmYS2ZcVm86fPQ8mQ2zl3G6A_qd6_7lDSy7K0BJMcmgJky5sfE4YhwqyrFiSjFKsPINaJGBCI355Ny4K1SHXCPFtDRIWjKOYiR97e6CRjJJ1B5wmMu4r2QguRdTjaTAWLuSBCr0EY5D1ASokmwkS-Zx2wBjFNWcyVY6kZFOZKUT4SY4nf8yLWg3fhvcqtQVlScwjQiyzHs-xl4TnFXqqT__ONn-n0Yfg-XeVSe6v-3eHYAVYtWbp7q0QCObvapDsCTfsmE6O8r35Tt3ReBm
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Best+practices+for+comparing+optimization+algorithms&rft.jtitle=Optimization+and+engineering&rft.au=Beiranvand%2C+Vahid&rft.au=Hare%2C+Warren&rft.au=Lucet%2C+Yves&rft.date=2017-12-01&rft.pub=Springer+Nature+B.V&rft.issn=1389-4420&rft.eissn=1573-2924&rft.volume=18&rft.issue=4&rft.spage=815&rft.epage=848&rft_id=info:doi/10.1007%2Fs11081-017-9366-1&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1389-4420&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1389-4420&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1389-4420&client=summon