Analysing differences between algorithm configurations through ablation

Developers of high-performance algorithms for hard computational problems increasingly take advantage of automated parameter tuning and algorithm configuration tools, and consequently often create solvers with many parameters and vast configuration spaces. However, there has been very little work to...

Full description

Saved in:

Bibliographic Details
Published in:	Journal of heuristics Vol. 22; no. 4; pp. 431 - 458
Main Authors:	Fawcett, Chris, Hoos, Holger H.
Format:	Journal Article
Language:	English
Published:	New York Springer US 01.08.2016 Springer Nature B.V
Subjects:	Ablation Algorithms Artificial Intelligence Automation Calculus of Variations and Optimal Control; Optimization Design of experiments Factorial experiments Genetic algorithms Heuristic Integer programming Management Science Mathematical analysis Mathematics Mathematics and Statistics Operations Research Operations Research/Decision Theory Optimization algorithms Production planning Studies Variance analysis Parameter importance Empirical analysis Ablation analysis Automated algorithm configuration
ISSN:	1381-1231, 1572-9397
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Developers of high-performance algorithms for hard computational problems increasingly take advantage of automated parameter tuning and algorithm configuration tools, and consequently often create solvers with many parameters and vast configuration spaces. However, there has been very little work to help these algorithm developers answer questions about the high-quality configurations produced by these tools, specifically about which parameter changes contribute most to improved performance. In this work, we present an automated technique for answering such questions by performing ablation analysis between two algorithm configurations. We perform an extensive empirical analysis of our technique on five scenarios from propositional satisfiability, mixed-integer programming and AI planning, and show that in all of these scenarios more than 95 % of the performance gains between default configurations and optimised configurations obtained from automated configuration tools can be explained by modifying the values of a small number of parameters (1–4 in the scenarios we studied). We also investigate the use of our ablation analysis procedure for producing configurations that generalise well to previously-unseen problem domains, as well as for analysing the structure of the algorithm parameter response surface near and between high-performance configurations.
AbstractList	Issue Title: Special Issue: MIC 2013 Developers of high-performance algorithms for hard computational problems increasingly take advantage of automated parameter tuning and algorithm configuration tools, and consequently often create solvers with many parameters and vast configuration spaces. However, there has been very little work to help these algorithm developers answer questions about the high-quality configurations produced by these tools, specifically about which parameter changes contribute most to improved performance. In this work, we present an automated technique for answering such questions by performing ablation analysis between two algorithm configurations. We perform an extensive empirical analysis of our technique on five scenarios from propositional satisfiability, mixed-integer programming and AI planning, and show that in all of these scenarios more than 95 % of the performance gains between default configurations and optimised configurations obtained from automated configuration tools can be explained by modifying the values of a small number of parameters (1-4 in the scenarios we studied). We also investigate the use of our ablation analysis procedure for producing configurations that generalise well to previously-unseen problem domains, as well as for analysing the structure of the algorithm parameter response surface near and between high-performance configurations. Developers of high-performance algorithms for hard computational problems increasingly take advantage of automated parameter tuning and algorithm configuration tools, and consequently often create solvers with many parameters and vast configuration spaces. However, there has been very little work to help these algorithm developers answer questions about the high-quality configurations produced by these tools, specifically about which parameter changes contribute most to improved performance. In this work, we present an automated technique for answering such questions by performing ablation analysis between two algorithm configurations. We perform an extensive empirical analysis of our technique on five scenarios from propositional satisfiability, mixed-integer programming and AI planning, and show that in all of these scenarios more than 95 % of the performance gains between default configurations and optimised configurations obtained from automated configuration tools can be explained by modifying the values of a small number of parameters (1–4 in the scenarios we studied). We also investigate the use of our ablation analysis procedure for producing configurations that generalise well to previously-unseen problem domains, as well as for analysing the structure of the algorithm parameter response surface near and between high-performance configurations.
Author	Fawcett, Chris Hoos, Holger H.
Author_xml	– sequence: 1 givenname: Chris surname: Fawcett fullname: Fawcett, Chris email: fawcettc@cs.ubc.ca organization: Computer Science Department, University of British Columbia – sequence: 2 givenname: Holger H. surname: Hoos fullname: Hoos, Holger H. organization: Computer Science Department, University of British Columbia
BookMark	eNp9kMFKAzEQhoNUsK0-gLcFz6uZzW6THEvRKhS86Dkk2WSbsk1qsov07d12PYigpxmG_xtmvhma-OANQreA7wFj-pAAU1LkGMqcF7TK-QWaQkWLnBNOJ0NPGORQELhCs5R2GGPOKjJF66WX7TE532S1s9ZE47VJmTLdpzE-k20Touu2-0wHb13TR9m54FPWbWPom20mVXueXKNLK9tkbr7rHL0_Pb6tnvPN6_pltdzkmsCiyw0vuFYU88LWShtMmLWYQg2VLKpK2bqsCQXOqaHUMMrwgvIF06qsFCe2VGSO7sa9hxg-epM6sQt9HH5IAhgAxyUjMKTomNIxpBSNFdp15zu7KF0rAIuTNTFaE4M1cbIm-EDCL_IQ3V7G479MMTJpyPrGxB83_Ql9Ae5vgaE
CitedBy_id	crossref_primary_10_1145_3612933 crossref_primary_10_1093_bib_bbac344 crossref_primary_10_1186_s13054_020_03403_6 crossref_primary_10_1016_j_eswa_2021_115140 crossref_primary_10_1017_S0033291722003452 crossref_primary_10_3233_JIFS_200682 crossref_primary_10_1016_j_cor_2023_106491 crossref_primary_10_3390_eng4010032 crossref_primary_10_1016_j_aca_2020_10_038 crossref_primary_10_1109_TEVC_2022_3211336 crossref_primary_10_1016_j_addma_2021_101965 crossref_primary_10_1016_j_eswa_2021_115899 crossref_primary_10_1016_j_cor_2020_104886 crossref_primary_10_1016_j_jisa_2020_102718 crossref_primary_10_1016_j_bspc_2024_106305 crossref_primary_10_1016_j_jss_2021_111133 crossref_primary_10_1007_s10732_016_9313_x crossref_primary_10_1109_TEVC_2022_3215013 crossref_primary_10_1007_s10143_019_01163_8 crossref_primary_10_1007_s10994_017_5683_z crossref_primary_10_1016_j_swevo_2024_101623 crossref_primary_10_1016_j_swevo_2022_101224 crossref_primary_10_1016_j_cor_2025_107050 crossref_primary_10_1016_j_asoc_2018_08_030 crossref_primary_10_3390_electronics11101640 crossref_primary_10_1016_j_future_2024_107524 crossref_primary_10_1111_itor_12631 crossref_primary_10_1162_evco_a_00263 crossref_primary_10_1080_1206212X_2021_1974663 crossref_primary_10_1088_1361_6501_ad0691 crossref_primary_10_1007_s12517_021_06465_0 crossref_primary_10_1109_TVCG_2020_3030380 crossref_primary_10_1109_TC_2020_2968888 crossref_primary_10_1016_j_cor_2025_106999 crossref_primary_10_1007_s00500_023_09107_7 crossref_primary_10_1016_j_cor_2024_106911 crossref_primary_10_1016_j_cor_2021_105300 crossref_primary_10_1007_s11721_023_00227_2 crossref_primary_10_1007_s10479_018_3122_6 crossref_primary_10_1016_j_asoc_2022_109798 crossref_primary_10_1016_j_asoc_2020_106269 crossref_primary_10_1016_j_jisa_2025_104002 crossref_primary_10_1145_3506695 crossref_primary_10_1007_s00521_020_05455_9 crossref_primary_10_3390_telecom4020015 crossref_primary_10_1145_3558774 crossref_primary_10_1007_s41060_025_00764_5 crossref_primary_10_3390_en17030553 crossref_primary_10_1016_j_compind_2022_103830 crossref_primary_10_1038_s41746_022_00613_w crossref_primary_10_1109_TIFS_2018_2825958 crossref_primary_10_1186_s12967_024_05138_2 crossref_primary_10_1007_s10462_024_10726_1 crossref_primary_10_1016_j_artint_2019_08_001 crossref_primary_10_1007_s10951_023_00799_1
Cites_doi	10.1007/978-0-387-76731-4 10.1198/106186007X237892 10.1007/BF00175357 10.1007/978-1-4615-6089-0 10.1007/0-306-48056-5_1 10.1186/1471-2105-14-139 10.1007/978-1-4615-4102-8_1 10.1145/2076450.2076469 10.1613/jair.2861 10.1109/FAMCAD.2007.9 10.1109/CEC.2007.4424460 10.1145/2463372.2463438 10.1007/978-3-642-04244-7_14 10.1609/socs.v4i1.18293 10.1007/978-3-540-68155-7_27 10.32614/CRAN.package.irace 10.1007/978-3-642-44973-4_40 10.1007/978-3-642-02538-9_10 10.1007/978-3-642-25566-3_21 10.1007/978-3-642-25566-3_40 10.1007/978-3-642-13520-0_23 10.1007/978-3-642-25566-3_47
ContentType	Journal Article
Copyright	Springer Science+Business Media New York 2015 Springer Science+Business Media New York 2016
Copyright_xml	– notice: Springer Science+Business Media New York 2015 – notice: Springer Science+Business Media New York 2016
DBID	AAYXX CITATION 3V. 7WY 7WZ 7XB 87Z 8AL 8FE 8FG 8FK 8FL ABUWG AFKRA ARAPS AZQEC BENPR BEZIV BGLVJ CCPQU DWQXO FRNLG F~G GNUQQ HCIFZ JQ2 K60 K6~ K7- L.- M0C M0N P5Z P62 PHGZM PHGZT PKEHL PQBIZ PQBZA PQEST PQGLB PQQKQ PQUKI PYYUZ Q9U
DOI	10.1007/s10732-014-9275-9
DatabaseName	CrossRef ProQuest Central (Corporate) ABI/INFORM Collection ABI/INFORM Global (PDF only) ProQuest Central (purchase pre-March 2016) ABI/INFORM Collection Computing Database (Alumni Edition) ProQuest SciTech Collection ProQuest Technology Collection ProQuest Central (Alumni) (purchase pre-March 2016) ABI/INFORM Collection (Alumni) ProQuest Central (Alumni) ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Essentials - QC ProQuest Central Business Premium Collection Technology collection ProQuest One Community College ProQuest Central Business Premium Collection (Alumni) ABI/INFORM Global (Corporate) ProQuest Central Student SciTech Premium Collection ProQuest Computer Science Collection ProQuest Business Collection (Alumni Edition) ProQuest Business Collection Computer Science Database ABI/INFORM Professional Advanced ABI/INFORM Global Computing Database Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection Proquest Central Premium ProQuest One Academic ProQuest One Academic Middle East (New) ProQuest One Business (UW System Shared) ProQuest One Business (Alumni) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ABI/INFORM Collection China ProQuest Central Basic
DatabaseTitle	CrossRef ABI/INFORM Global (Corporate) ProQuest Business Collection (Alumni Edition) ProQuest One Business Computer Science Database ProQuest Central Student Technology Collection ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ABI/INFORM Complete ProQuest Central ABI/INFORM Professional Advanced ProQuest One Applied & Life Sciences ProQuest Central Korea ProQuest Central (New) ABI/INFORM Complete (Alumni Edition) Advanced Technologies & Aerospace Collection Business Premium Collection ABI/INFORM Global ProQuest Computing ABI/INFORM Global (Alumni Edition) ProQuest Central Basic ProQuest Computing (Alumni Edition) ProQuest One Academic Eastern Edition ABI/INFORM China ProQuest Technology Collection ProQuest SciTech Collection ProQuest Business Collection Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition ProQuest One Business (Alumni) ProQuest One Academic ProQuest Central (Alumni) ProQuest One Academic (New) Business Premium Collection (Alumni)
DatabaseTitleList	ABI/INFORM Global (Corporate)
Database_xml	– sequence: 1 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Mathematics Computer Science
EISSN	1572-9397
EndPage	458
ExternalDocumentID	4149460761 10_1007_s10732_014_9275_9
GroupedDBID	-52 -5D -5G -BR -EM -Y2 -~C .4S .86 .DC .VR 06D 0R~ 0VY 1N0 1SB 2.D 203 28- 29K 2J2 2JN 2JY 2KG 2KM 2LR 2P1 2VQ 2~H 30V 3V. 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 7WY 8FE 8FG 8FL 8TC 8UJ 95- 95. 95~ 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYOK AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDZT ABECU ABFTV ABHLI ABHQN ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABUWG ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACHSB ACHXU ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADMLS ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFGCZ AFKRA AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARAPS ARCSS ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN AZQEC B-. BA0 BAPOH BBWZM BDATZ BENPR BEZIV BGLVJ BGNMA BPHCQ BSONS CAG CCPQU COF CS3 CSCUP D-I DDRTE DL5 DNIVK DPUIP DU5 DWQXO EBLON EBS EDO EIOEI EJD ESBYG FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRNLG FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNUQQ GNWQR GQ6 GQ7 GQ8 GROUPED_ABI_INFORM_COMPLETE GROUPED_ABI_INFORM_RESEARCH GXS H13 HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I-F I09 IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z JBSCW JCJTX JZLTJ K60 K6V K6~ K7- KDC KOV KOW LAK LLZTM M0C M0N M4Y MA- N2Q NB0 NDZJH NPVJJ NQJWS NU0 O9- O93 O9G O9I O9J OAM OVD P19 P2P P62 P9O PF0 PQBIZ PQBZA PQQKQ PROAC PT4 PT5 Q2X QOK QOS R4E R89 R9I RHV RNI ROL RPX RSV RZC RZD RZK S16 S1Z S26 S27 S28 S3B SAP SCLPG SDD SDH SDM SHX SISQX SJYHP SMT SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TEORI TSG TSK TSV TUC TUS U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WK8 YLTOR Z45 Z81 Z83 Z86 Z88 Z8U Z8W Z92 ZMTXR ZWQNP ~EX AAPKM AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC ADHKG ADKFA AEZWR AFDZB AFFHD AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION PHGZM PHGZT PQGLB 7XB 8AL 8FK JQ2 L.- PKEHL PQEST PQUKI Q9U
ID	FETCH-LOGICAL-c316t-e929cb7092fdbce038ff071d15a255bfd4d371997e77e878067968cb45b93f4b3
IEDL.DBID	K7-
ISICitedReferencesCount	78
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000380273700005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1381-1231
IngestDate	Tue Nov 04 21:50:56 EST 2025 Tue Nov 18 22:19:04 EST 2025 Sat Nov 29 07:53:06 EST 2025 Fri Feb 21 02:36:37 EST 2025
IsPeerReviewed	true
IsScholarly	true
Issue	4
Keywords	Parameter importance Empirical analysis Ablation analysis Automated algorithm configuration
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c316t-e929cb7092fdbce038ff071d15a255bfd4d371997e77e878067968cb45b93f4b3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
PQID	1811904831
PQPubID	26068
PageCount	28
ParticipantIDs	proquest_journals_1811904831 crossref_citationtrail_10_1007_s10732_014_9275_9 crossref_primary_10_1007_s10732_014_9275_9 springer_journals_10_1007_s10732_014_9275_9
PublicationCentury	2000
PublicationDate	20160800 2016-8-00 20160801
PublicationDateYYYYMMDD	2016-08-01
PublicationDate_xml	– month: 8 year: 2016 text: 20160800
PublicationDecade	2010
PublicationPlace	New York
PublicationPlace_xml	– name: New York – name: Boston
PublicationTitle	Journal of heuristics
PublicationTitleAbbrev	J Heuristics
PublicationYear	2016
Publisher	Springer US Springer Nature B.V
Publisher_xml	– name: Springer US – name: Springer Nature B.V
References	Hoos (CR15) 2012; 55 CR19 CR17 CR16 Bartz-Beielstein (CR3) 2006 CR13 CR12 Maron, Moore (CR24) 1994; 6 Hooker (CR14) 2007; 16 CR30 Glover (CR7) 1994; 4 Aghaeepour, Hoos (CR1) 2013; 14 Glover, Laguna (CR9) 1997 Glover, Barr, Helgason, Kennington (CR8) 1997 Conover (CR6) 1999 CR2 Glover, Laguna, Martí (CR10) 2000; 39 CR4 CR5 Reidys (CR27) 2011 CR29 CR28 CR26 CR25 Glover, Laguna, Martí, Glover, Kochenberger (CR11) 2003 CR23 CR22 CR21 CR20 Hutter, Hoos, Leyton-Brown, Stützle (CR18) 2009; 36 F Glover (9275_CR7) 1994; 4 9275_CR23 9275_CR26 CM Reidys (9275_CR27) 2011 9275_CR25 9275_CR28 F Hutter (9275_CR18) 2009; 36 9275_CR29 9275_CR20 9275_CR22 9275_CR21 9275_CR4 9275_CR5 W Conover (9275_CR6) 1999 O Maron (9275_CR24) 1994; 6 9275_CR2 F Glover (9275_CR11) 2003 F Glover (9275_CR10) 2000; 39 9275_CR13 9275_CR12 N Aghaeepour (9275_CR1) 2013; 14 9275_CR17 9275_CR16 F Glover (9275_CR8) 1997 9275_CR19 9275_CR30 G Hooker (9275_CR14) 2007; 16 HH Hoos (9275_CR15) 2012; 55 F Glover (9275_CR9) 1997 T Bartz-Beielstein (9275_CR3) 2006
References_xml	– ident: CR22 – year: 2011 ident: CR27 publication-title: Combinatorial Computational Biology of RNA: Pseudoknots and Neutral Networks doi: 10.1007/978-0-387-76731-4 – ident: CR4 – ident: CR2 – volume: 16 start-page: 709 year: 2007 end-page: 732 ident: CR14 article-title: Generalized functional ANOVA diagnostics for high dimensional functions of dependent variables publication-title: J. Comput. Graph. Stat. doi: 10.1198/106186007X237892 – ident: CR16 – year: 2006 ident: CR3 publication-title: Experimental Research in Evolutionary Computation-The New Experimentalism. Natural Computing Series – ident: CR12 – ident: CR30 – volume: 4 start-page: 131 year: 1994 end-page: 140 ident: CR7 article-title: Genetic algorithms and scatter search: unsuspected potentials publication-title: Stat. Comput. doi: 10.1007/BF00175357 – ident: CR29 – volume: 36 start-page: 267 year: 2009 end-page: 306 ident: CR18 article-title: ParamILS: an automatic algorithm configuration framework publication-title: J. Artif. Intel. Res. – ident: CR25 – year: 1997 ident: CR9 publication-title: Tabu Search doi: 10.1007/978-1-4615-6089-0 – ident: CR23 – start-page: 1 year: 2003 end-page: 35 ident: CR11 article-title: Scatter search and path relinking: advances and applications publication-title: Handbook of Metaheuristics, International Series in Operations Research & Management Science doi: 10.1007/0-306-48056-5_1 – ident: CR21 – ident: CR19 – volume: 14 start-page: 139 year: 2013 ident: CR1 article-title: Ensemble-based prediction of RNA secondary structures publication-title: BMC Bioinform. doi: 10.1186/1471-2105-14-139 – start-page: 1 year: 1997 end-page: 75 ident: CR8 article-title: Tabu search and adaptive memory programing—advances, applications and challenges publication-title: Interfaces in Computer Science and Operations Research doi: 10.1007/978-1-4615-4102-8_1 – ident: CR17 – ident: CR13 – ident: CR5 – volume: 6 start-page: 59 year: 1994 end-page: 66 ident: CR24 article-title: Hoeffding races: accelerating model selection search for classification and function approximation publication-title: Adv. Neural Inf. Process. Syst. – volume: 55 start-page: 70 issue: 2 year: 2012 end-page: 80 ident: CR15 article-title: Programming by optimization publication-title: Commun. ACM doi: 10.1145/2076450.2076469 – ident: CR28 – ident: CR26 – year: 1999 ident: CR6 publication-title: Practical Nonparametric Statistics – volume: 39 start-page: 653 year: 2000 end-page: 684 ident: CR10 article-title: Fundamentals of scatter search and path relinking publication-title: Control Cybern. – ident: CR20 – volume-title: Experimental Research in Evolutionary Computation-The New Experimentalism. Natural Computing Series year: 2006 ident: 9275_CR3 – ident: 9275_CR17 – volume: 36 start-page: 267 year: 2009 ident: 9275_CR18 publication-title: J. Artif. Intel. Res. doi: 10.1613/jair.2861 – start-page: 1 volume-title: Handbook of Metaheuristics, International Series in Operations Research & Management Science year: 2003 ident: 9275_CR11 doi: 10.1007/0-306-48056-5_1 – volume: 14 start-page: 139 year: 2013 ident: 9275_CR1 publication-title: BMC Bioinform. doi: 10.1186/1471-2105-14-139 – ident: 9275_CR16 doi: 10.1109/FAMCAD.2007.9 – volume-title: Practical Nonparametric Statistics year: 1999 ident: 9275_CR6 – ident: 9275_CR25 doi: 10.1109/CEC.2007.4424460 – ident: 9275_CR28 doi: 10.1145/2463372.2463438 – ident: 9275_CR2 doi: 10.1007/978-3-642-04244-7_14 – volume: 39 start-page: 653 year: 2000 ident: 9275_CR10 publication-title: Control Cybern. – ident: 9275_CR30 doi: 10.1609/socs.v4i1.18293 – volume: 4 start-page: 131 year: 1994 ident: 9275_CR7 publication-title: Stat. Comput. doi: 10.1007/BF00175357 – ident: 9275_CR12 doi: 10.1007/978-3-540-68155-7_27 – ident: 9275_CR23 doi: 10.32614/CRAN.package.irace – volume-title: Tabu Search year: 1997 ident: 9275_CR9 doi: 10.1007/978-1-4615-6089-0 – ident: 9275_CR21 doi: 10.1007/978-3-642-44973-4_40 – ident: 9275_CR22 – volume: 6 start-page: 59 year: 1994 ident: 9275_CR24 publication-title: Adv. Neural Inf. Process. Syst. – ident: 9275_CR5 doi: 10.1007/978-3-642-02538-9_10 – ident: 9275_CR13 doi: 10.1007/978-3-642-25566-3_21 – ident: 9275_CR20 doi: 10.1007/978-3-642-25566-3_40 – ident: 9275_CR19 doi: 10.1007/978-3-642-13520-0_23 – volume: 55 start-page: 70 issue: 2 year: 2012 ident: 9275_CR15 publication-title: Commun. ACM doi: 10.1145/2076450.2076469 – ident: 9275_CR4 – volume-title: Combinatorial Computational Biology of RNA: Pseudoknots and Neutral Networks year: 2011 ident: 9275_CR27 doi: 10.1007/978-0-387-76731-4 – volume: 16 start-page: 709 year: 2007 ident: 9275_CR14 publication-title: J. Comput. Graph. Stat. doi: 10.1198/106186007X237892 – ident: 9275_CR26 doi: 10.1007/978-3-642-25566-3_47 – ident: 9275_CR29 – start-page: 1 volume-title: Interfaces in Computer Science and Operations Research year: 1997 ident: 9275_CR8 doi: 10.1007/978-1-4615-4102-8_1
SSID	ssj0009853
Score	2.4603972
Snippet	Developers of high-performance algorithms for hard computational problems increasingly take advantage of automated parameter tuning and algorithm configuration... Issue Title: Special Issue: MIC 2013 Developers of high-performance algorithms for hard computational problems increasingly take advantage of automated...
SourceID	proquest crossref springer
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	431
SubjectTerms	Ablation Algorithms Artificial Intelligence Automation Calculus of Variations and Optimal Control; Optimization Design of experiments Factorial experiments Genetic algorithms Heuristic Integer programming Management Science Mathematical analysis Mathematics Mathematics and Statistics Operations Research Operations Research/Decision Theory Optimization algorithms Production planning Studies Variance analysis
SummonAdditionalLinks	– databaseName: SpringerLINK Contemporary 1997-Present dbid: RSV link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3PS8MwFH7o9KAHp1NxOiUHT0qgP9ImOYo4vTjEX-xWmrSZg7nJ2vn3m7TJNkUFPZYmobzk5X2vefk-gNNAMR7lUmHhZQSTQDKceoHEETGiOJzEHssqsQna67F-n9_Ze9yFq3Z3R5LVTr102Y2GpoyAYB7QCPNVWNPRjhlvvH94XjDtsop60tehCOtt2XdHmd8N8TkYLRDml0PRKtZ0m__6ym3YstASXdRrYQdW8nELmk62AVkvbsHmEgehfrqdE7cWu3BtSUrGA-SkU_RGgmw1F0pHg8l0WL68Ip1Gq-FgVq-fAlm5H5SKurZuD566V4-XN9hqLWAZ-nGJcw2TpKAeD1QmjIYYU0qjj8yPUp10CJWRLKSmKCWnNGeUVf-fmBQkEjxURIT70BhPxvkBIO3IIo7T0CApIikVOgVTkSkAJcrkQ23wnNETaYnIjR7GKFlQKBsjJtqIiTFiwttwNu_yVrNw_Na442YysQ5ZJBrIaOhDWOi34dzN3NLrnwY7_FPrI9jQgCquCwQ70Cins_wY1uV7OSymJ9U6_QCTAOA0 priority: 102 providerName: Springer Nature
Title	Analysing differences between algorithm configurations through ablation
URI	https://link.springer.com/article/10.1007/s10732-014-9275-9 https://www.proquest.com/docview/1811904831
Volume	22
WOSCitedRecordID	wos000380273700005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVPQU databaseName: ABI/INFORM Collection customDbUrl: eissn: 1572-9397 dateEnd: 20171231 omitProxy: false ssIdentifier: ssj0009853 issn: 1381-1231 databaseCode: 7WY dateStart: 19970301 isFulltext: true titleUrlDefault: https://www.proquest.com/abicomplete providerName: ProQuest – providerCode: PRVPQU databaseName: ABI/INFORM Global customDbUrl: eissn: 1572-9397 dateEnd: 20171231 omitProxy: false ssIdentifier: ssj0009853 issn: 1381-1231 databaseCode: M0C dateStart: 19970301 isFulltext: true titleUrlDefault: https://search.proquest.com/abiglobal providerName: ProQuest – providerCode: PRVPQU databaseName: Advanced Technologies & Aerospace Database customDbUrl: eissn: 1572-9397 dateEnd: 20171231 omitProxy: false ssIdentifier: ssj0009853 issn: 1381-1231 databaseCode: P5Z dateStart: 19970301 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: Computer Science Database customDbUrl: eissn: 1572-9397 dateEnd: 20171231 omitProxy: false ssIdentifier: ssj0009853 issn: 1381-1231 databaseCode: K7- dateStart: 19970301 isFulltext: true titleUrlDefault: http://search.proquest.com/compscijour providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1572-9397 dateEnd: 20171231 omitProxy: false ssIdentifier: ssj0009853 issn: 1381-1231 databaseCode: BENPR dateStart: 19970301 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1572-9397 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0009853 issn: 1381-1231 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3JTsMwEB1BywEOLAVEWSofOIEssjixc0KACkiIqio7lyh24lIJypKW72ecOBSQ4MLFl6zKjGexX94D2Pa0iIJMaSqdlFHmKUETx1M0YEYUJ2KhI9JCbIJ3OuL2NuraBbfcwiqrmFgE6vRZmTXyPcxEmLuY8N39l1dqVKPM7qqV0JiGuut5rvHzM04npLuiYKF0MStRjNButatZ_jrHfQNKYDTyeECj73lpUmz-2B8t0s7xwn9feBHmbcFJDkoPWYKpbNiAhUrMgdi53YC5L8yEy3BiyUqGfVJJqGBAIRbVRZLHPj5q9PBEsJ3Wg_649KOcWNkfksgSY7cCV8fty6NTajUXqPLdcEQzLJeU5E7k6VQaLTGhNVYhqRsk2HxInbLU5wacknGeCS6KdSihJAtk5Gsm_VWoDZ-H2RoQnNAyDBPfVFRMcS6xFdOBAYIybfqiJjjVF4-VJSQ3uhiP8YRK2RgpRiPFxkhx1ISdz0teSjaOv07erAwT24mZxxOrNGG3Mu2Xw7_dbP3vm23ALFZSYYkM3ITa6G2cbcGMeh8N8rcWTPObuxbUD9udbq9V-CiO584Rjt3gHsfexfUH7YnrbA
linkProvider	ProQuest
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3JTsMwEB1BQQIO7Iiy-gAXkEUWJ3YOCCF2ARUHkLiF2IlLJUihLSB-im9knMQUkODGgXOSkRy_WWw_zwNY87SIgkxpKp2UUeYpQRPHUzRgRhQnYqEj0kJsgjca4vo6uhiAN3sXxtAqbUwsAnXaVmaPfAszEeYuJnx35-GRGtUoc7pqJTRKWJxmry-4ZOtun-zj_K573uHB5d4xrVQFqPLdsEczLAiU5E7k6VQatSyhNebZ1A0SLK-lTlnqc0O_yDjPBBfFTotQkgUy8jWTPtodhCHG0B0MVdDZ6zf5FUXXSxezIMWM4NpT1PKqHvcNCYLRyOMBjb7mwX5x--08tkhzhxP_7QdNwnhVUJPd0gOmYCDLp2HCilWQKnZNw9inzoszcFQ1Y8mbxErEYMAkFWuNJHdNHFrv9p6odq5bzafST7qkkjUiiSw5hLNw9SeDm4Na3s6zeSAYsGQYJr6pGJniXOJSUweG6Mq0WffVwbEzHKuq4brR_biL-62iDShiBEVsQBFHddj4-OSh7Dby28tLFghxFXi6cR8Fddi0UPr0-CdjC78bW4WR48vzs_jspHG6CKNYNYYlC3IJar3OU7YMw-q51-p2VgqPIHDz1wh7B4TpQnw
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3JTuQwEC2xCTEHdkQPmw9wAVmdxYmdwwghoAGBWn0ACXEJsRM3SJAGupnR_Np83ZQTmwYkuHHgnKQkx8-12M_1ADYDLZKoUJpKL2eUBUrQzAsUjZgRxUlY7Im8Epvg7ba4vEw6I_DP3YUxtErnEytHnfeU2SNvYiTC2MVE6De1pUV0Dlq7D4_UKEiZk1Ynp1FD5LT4-wfLt_6vkwOc660gaB2e7x9TqzBAVejHA1pgcqAk95JA59IoZwmtMebmfpRhqi11zvKQGypGwXkhuKh2XYSSLJJJqJkM0e4ojHOsMU3h14muhg1_RdUB08eISDE6-O5Etb62x0NDiGA0CXhEk7cxcZjovjubrUJea-Y7_6xZmLaJNtmrV8YcjBTlPMw4EQtifdo8_HjVkXEBjmyTlrJLnHQMOlJi2Wwku-vi0AY390T1Sn3bfa7XT59YuSOSyZpbuAgXXzK4JRgre2WxDAQdmYzjLDSZJFOcSyxBdWQIsEyberABnpvtVNlG7EYP5C4dtpA2AEkRIKkBSJo0YPvlk4e6C8lnL686UKTWIfXTISIasONg9erxR8Z-fm5sAyYRWOnZSft0BaYwmYxrcuQqjA2enos1mFC_B7f9p_VqcRC4_mqA_QcXFEuN
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Analysing+differences+between+algorithm+configurations+through+ablation&rft.jtitle=Journal+of+heuristics&rft.au=Fawcett%2C+Chris&rft.au=Hoos%2C+Holger+H.&rft.date=2016-08-01&rft.issn=1381-1231&rft.eissn=1572-9397&rft.volume=22&rft.issue=4&rft.spage=431&rft.epage=458&rft_id=info:doi/10.1007%2Fs10732-014-9275-9&rft.externalDBID=n%2Fa&rft.externalDocID=10_1007_s10732_014_9275_9
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1381-1231&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1381-1231&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1381-1231&client=summon