Benchmarking Procedures for Continuous Optimization Algorithms
Reliable comparison of optimization algorithms requires the use of specialized benchmarking procedures. This paper highlights motivations which influence their structure, discusses evaluation criteria of algorithms, typical ways of presenting and interpreting results as well as related statistical p...
Uloženo v:
| Vydáno v: | Journal of Telecommunications and Information Technology číslo 4; s. 73 - 80 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
National Institute of Telecommunications
01.06.2023
|
| Témata: | |
| ISSN: | 1509-4553, 1899-8852 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Shrnutí: | Reliable comparison of optimization algorithms requires the use of specialized benchmarking procedures. This paper highlights motivations which influence their structure, discusses evaluation criteria of algorithms, typical ways of presenting and interpreting results as well as related statistical procedures. Discussions are based on examples from CEC and BBOB benchmarks. Moreover, attention is drawn to these features of comparison procedures, which make them susceptible to manipulation. In particular, novel application of the weak axiom of revealed preferences to the field of benchmarking shows why it may be misleading to assess algorithms on basis of their ranks for each of test problems. Additionally, an idea is presented of developing massively parallel implementation of benchmarks. Not only would this provide faster computation but also open the door to improving reliability of benchmarking procedures and promoting research into parallel implementations of optimization algorithms. |
|---|---|
| ISSN: | 1509-4553 1899-8852 |
| DOI: | 10.26636/jtit.2011.4.1180 |