Maximum number of generations as a stopping criterion considered harmful

Evolutionary algorithms have been shown to be very effective in solving complex optimization problems. This has driven the research community in the development of novel, even more efficient evolutionary algorithms. The newly proposed algorithms need to be evaluated and compared with existing state-...

Full description

Saved in:
Bibliographic Details
Published in:Applied soft computing Vol. 128; p. 109478
Main Authors: Ravber, Miha, Liu, Shih-Hsi, Mernik, Marjan, Črepinšek, Matej
Format: Journal Article
Language:English
Published: Elsevier B.V 01.10.2022
Subjects:
Algorithm comparison
Algorithm termination
Benchmarking
Evolutionary algorithms
Stopping criteria
Benchmarking
Algorithm comparison
Evolutionary algorithms
Stopping criteria
Algorithm termination
ISSN:1568-4946, 1872-9681
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Evolutionary algorithms have been shown to be very effective in solving complex optimization problems. This has driven the research community in the development of novel, even more efficient evolutionary algorithms. The newly proposed algorithms need to be evaluated and compared with existing state-of-the-art algorithms, usually by employing benchmarks. However, comparing evolutionary algorithms is a complicated task, which involves many factors that must be considered to ensure a fair and unbiased comparison. In this paper, we focus on the impact of stopping criteria in the comparison process. Their job is to stop the algorithms in such a way that each algorithm has a fair opportunity to solve the problem. Although they are not given much attention, they play a vital role in the comparison process. In the paper, we compared different stopping criteria with different settings, to show their impact on the comparison results. The results show that stopping criteria play a vital role in the comparison, as they can produce statistically significant differences in the rankings of evolutionary algorithms. The experiments have shown that in one case an algorithm consumed 50 times more evaluations in a single generation, giving it a considerable advantage when max gen was used as the stopping criterion, which puts the validity of most published work in question. •Overview of four existing stopping criteria used in benchmarking.•Comparison of seven evolutionary algorithms by using a chess rating system.•Two benchmarks consisting of 36 standard problems and a real-world problem.•Presented guidelines for applying stopping criteria in benchmarking.
ISSN:1568-4946
1872-9681
DOI:10.1016/j.asoc.2022.109478