Analysing differences between algorithm configurations through ablation

Developers of high-performance algorithms for hard computational problems increasingly take advantage of automated parameter tuning and algorithm configuration tools, and consequently often create solvers with many parameters and vast configuration spaces. However, there has been very little work to...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of heuristics Ročník 22; číslo 4; s. 431 - 458
Hlavní autoři: Fawcett, Chris, Hoos, Holger H.
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.08.2016
Springer Nature B.V
Témata:
Ablation
Algorithms
Artificial Intelligence
Automation
Calculus of Variations and Optimal Control; Optimization
Design of experiments
Factorial experiments
Genetic algorithms
Heuristic
Integer programming
Management Science
Mathematical analysis
Mathematics
Mathematics and Statistics
Operations Research
Operations Research/Decision Theory
Optimization algorithms
Production planning
Studies
Variance analysis
Parameter importance
Empirical analysis
Ablation analysis
Automated algorithm configuration
ISSN:1381-1231, 1572-9397
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Developers of high-performance algorithms for hard computational problems increasingly take advantage of automated parameter tuning and algorithm configuration tools, and consequently often create solvers with many parameters and vast configuration spaces. However, there has been very little work to help these algorithm developers answer questions about the high-quality configurations produced by these tools, specifically about which parameter changes contribute most to improved performance. In this work, we present an automated technique for answering such questions by performing ablation analysis between two algorithm configurations. We perform an extensive empirical analysis of our technique on five scenarios from propositional satisfiability, mixed-integer programming and AI planning, and show that in all of these scenarios more than 95 % of the performance gains between default configurations and optimised configurations obtained from automated configuration tools can be explained by modifying the values of a small number of parameters (1–4 in the scenarios we studied). We also investigate the use of our ablation analysis procedure for producing configurations that generalise well to previously-unseen problem domains, as well as for analysing the structure of the algorithm parameter response surface near and between high-performance configurations.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1381-1231
1572-9397
DOI:10.1007/s10732-014-9275-9