Paradox-Free Data Analysis in Numerical Comparisons of Optimization Algorithms: A Review and Case Study

Numerical comparison is essential for evaluating an optimization algorithm. Unfortunately, recent research has shown that two paradoxes may occur, namely the cycle ranking paradox and survival of the nonfittest paradox. Further exploitation reveals that these paradoxes stem from the method of data a...

Full description

Saved in:
Bibliographic Details
Published in:International journal of swarm intelligence research Vol. 15; no. 1; pp. 1 - 22
Main Authors: Zhi, Yunchao, Liu, Qunfeng, Yan, Yuan, Jing, Yunpeng
Format: Journal Article
Language:English
Published: Hershey IGI Global 01.01.2024
Subjects:
Algorithms
Case studies
Data analysis
Design optimization
Distribution functions
Hypotheses
Hypothesis testing
Methods
Optimization algorithms
Paradoxes
Probability
ISSN:1947-9263, 1947-9271
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Numerical comparison is essential for evaluating an optimization algorithm. Unfortunately, recent research has shown that two paradoxes may occur, namely the cycle ranking paradox and survival of the nonfittest paradox. Further exploitation reveals that these paradoxes stem from the method of data analysis, especially its comparison strategy. Therefore, the design and widespread use of paradox-free data analysis methods has become urgent. This paper is dedicated to reviewing the recent progress in paradox in numerical comparisons of optimization algorithms, especially the reasons for paradoxes and the ways to eliminate them. Specifically, significant progress has been made in eliminating paradoxes from two popular data analysis methods, including hypothesis testing and methods based on the cumulative distribution function. Furthermore, in this paper the authors provide case studies, aiming to show that paradoxes are common and how to eliminate them with paradox-free data analysis methods.
AbstractList Numerical comparison is essential for evaluating an optimization algorithm. Unfortunately, recent research has shown that two paradoxes may occur, namely the cycle ranking paradox and survival of the nonfittest paradox. Further exploitation reveals that these paradoxes stem from the method of data analysis, especially its comparison strategy. Therefore, the design and widespread use of paradox-free data analysis methods has become urgent. This paper is dedicated to reviewing the recent progress in paradox in numerical comparisons of optimization algorithms, especially the reasons for paradoxes and the ways to eliminate them. Specifically, significant progress has been made in eliminating paradoxes from two popular data analysis methods, including hypothesis testing and methods based on the cumulative distribution function. Furthermore, in this paper the authors provide case studies, aiming to show that paradoxes are common and how to eliminate them with paradox-free data analysis methods.
Author Zhi, Yunchao
Jing, Yunpeng
Liu, Qunfeng
Yan, Yuan
AuthorAffiliation Dongguan University of Technology, Dongguan, China
University of Electronic Science and Technology of China, Shenzhen, China
AuthorAffiliation_xml – sequence: 0
  name: Dongguan University of Technology, Dongguan, China
– sequence: 0
  name: University of Electronic Science and Technology of China, Shenzhen, China
Author_xml – sequence: 1
  givenname: Yunchao
  surname: Zhi
  fullname: Zhi, Yunchao
  organization: Dongguan University of Technology, Dongguan, China
– sequence: 2
  givenname: Qunfeng
  surname: Liu
  fullname: Liu, Qunfeng
  organization: Dongguan University of Technology, Dongguan, China
– sequence: 3
  givenname: Yuan
  surname: Yan
  fullname: Yan, Yuan
  organization: University of Electronic Science and Technology of China, Shenzhen, China
– sequence: 4
  givenname: Yunpeng
  surname: Jing
  fullname: Jing, Yunpeng
  organization: Dongguan University of Technology, Dongguan, China
BookMark eNpt0M9PwjAUB_DGYCIiN_-AJl4dtmvLqDcyRTFEDOi5eds6LGErtpuKf73D-ePiu7x3-OabvM8x6pS21AidUjLghI4upnfL6WLAxFBQfoC6VPIokGFEO7_3kB2hvvdr0ozgUSRYF60ewEFm34OJ0xpfQQV4XMJm543HpsT3daGdSWGDY1tswRlvS49tjufbyhTmAypjSzzerKwz1XPhL_EYL_Sr0W8YygzH4DVeVnW2O0GHOWy87n_vHnqaXD_Gt8FsfjONx7MgDUekCoDl6TAhUlAJQoo8yYSmOZeEcC4jljASEhnmkkud6lGmgYosyTRlIxhCmnLWQ2dt79bZl1r7Sq1t7ZqPvGKURoTLsHHoofM2lTrrvdO52jpTgNspStReU31pqlazicdt3KzMX1-jpvZqaq-mftSUKf_roIJ9ApN0gBQ
Cites_doi 10.17877/DE290R-15667
10.7551/mitpress/1090.001.0001
10.1109/CEC.2019.8790158
10.1007/s10462-020-09906-6
10.1023/A:1008202821328
10.1162/EVCO_a_00134
10.1093/comjnl/7.4.308
10.1126/science.220.4598.671
10.1007/BF00143070
10.1007/s13042-019-01053-x
10.1145/3319619.3321996
10.1080/10556780902909948
10.1016/j.ins.2024.120120
10.1145/962437.962444
10.1016/j.matcom.2016.05.006
10.1007/s101070100263
10.1023/A:1026065325419
10.1016/j.swevo.2020.100665
10.1111/j.1541-0420.2005.00389.x
10.1016/j.swevo.2021.100888
10.1016/j.ins.2010.12.006
10.1109/TEVC.2019.2955110
10.1137/080724083
10.1287/ijoc.1060.0175
10.1109/ICNN.1995.488968
10.1109/TEVC.2022.3199647
10.1007/s00500-016-2471-9
10.1145/1830761.1830790
10.1109/CEC55065.2022.9870433
10.1007/BF01742498
ContentType Journal Article
Copyright 2024. This work is published under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2024. This work is published under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID AAYXX
CITATION
7SC
8FD
8FE
8FG
ABJCF
AFKRA
ARAPS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
GNUQQ
HCIFZ
JQ2
K7-
L6V
L7M
L~C
L~D
M7S
P62
PHGZM
PHGZT
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.4018/IJSIR.356514
DatabaseName CrossRef
Computer and Information Systems Abstracts
Technology Research Database
ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central UK/Ireland
Advanced Technologies & Computer Science Collection
ProQuest Central Essentials
ProQuest Central
Technology collection
ProQuest One
ProQuest Central
ProQuest Central Student
SciTech Premium Collection
ProQuest Computer Science Collection
Computer Science Database
ProQuest Engineering Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
Engineering Database
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Premium
ProQuest One Academic
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle CrossRef
Computer Science Database
ProQuest Central Student
Technology Collection
Technology Research Database
Computer and Information Systems Abstracts – Academic
ProQuest One Academic Middle East (New)
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Essentials
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
SciTech Premium Collection
ProQuest One Community College
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest Central Korea
ProQuest Central (New)
Advanced Technologies Database with Aerospace
Engineering Collection
Advanced Technologies & Aerospace Collection
Engineering Database
ProQuest One Academic Eastern Edition
ProQuest Technology Collection
ProQuest SciTech Collection
Computer and Information Systems Abstracts Professional
ProQuest One Academic UKI Edition
Materials Science & Engineering Collection
ProQuest One Academic
ProQuest One Academic (New)
DatabaseTitleList Computer Science Database

CrossRef
Database_xml – sequence: 1
  dbid: BENPR
  name: ProQuest Central
  url: https://www.proquest.com/central
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1947-9271
EndPage 22
ExternalDocumentID 10_4018_IJSIR_356514
dox_Free_Data_Analysis_in10_4018_IJSIR_35651415
GroupedDBID 0R~
4.4
AAYVP
ABEPT
ABGRR
ABJCF
ABPHS
ACOJC
AFKRA
ALMA_UNASSIGNED_HOLDINGS
ARAPS
BAAKF
BENPR
BGLVJ
BYHXH
CBWLS
CCPQU
CDTDJ
CIGCI
CKMBR
CNQXE
COVLG
CTSEY
EBS
H13
HCIFZ
HZ~
IAO
IGYUU
ITC
JRD
K7-
M7S
MV1
N95
NEEBM
O9-
PTHSS
RIF
AAYXX
ADMLS
AFFHD
CITATION
ICD
IMI
PHGZM
PHGZT
PQGLB
7SC
8FD
8FE
8FG
AZQEC
DWQXO
GNUQQ
JQ2
L6V
L7M
L~C
L~D
P62
PKEHL
PQEST
PQQKQ
PQUKI
PRINS
ID FETCH-LOGICAL-c280t-a3fc6b09519a595fbd5e1f490044973b302092f949ece8dea15dbde138a6acc43
IEDL.DBID M7S
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001336693800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1947-9263
IngestDate Fri Jul 25 11:41:11 EDT 2025
Sat Nov 29 06:00:57 EST 2025
Thu Dec 05 09:33:41 EST 2024
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 1
Language English
License http://creativecommons.org/licenses/by/3.0/deed.en_US
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c280t-a3fc6b09519a595fbd5e1f490044973b302092f949ece8dea15dbde138a6acc43
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0003-0769-1716
0000-0002-6286-941X
0009-0005-3229-2532
0000-0002-7838-5719
PQID 3117049226
PQPubID 2045809
PageCount 22
ParticipantIDs igi_journals_dox_Free_Data_Analysis_in10_4018_IJSIR_35651415
proquest_journals_3117049226
crossref_primary_10_4018_IJSIR_356514
PublicationCentury 2000
PublicationDate 2024-01-01
PublicationDateYYYYMMDD 2024-01-01
PublicationDate_xml – month: 01
  year: 2024
  text: 2024-01-01
  day: 01
PublicationDecade 2020
PublicationPlace Hershey
PublicationPlace_xml – name: Hershey
PublicationTitle International journal of swarm intelligence research
PublicationYear 2024
Publisher IGI Global
Publisher_xml – name: IGI Global
References IJSIR.356514-30
IJSIR.356514-32
IJSIR.356514-10
IJSIR.356514-31
IJSIR.356514-11
IJSIR.356514-33
IJSIR.356514-13
IJSIR.356514-14
IJSIR.356514-35
IJSIR.356514-15
IJSIR.356514-16
IJSIR.356514-17
K.Hamacher (IJSIR.356514-9) 2014
IJSIR.356514-19
IJSIR.356514-3
IJSIR.356514-2
IJSIR.356514-5
IJSIR.356514-4
IJSIR.356514-1
IJSIR.356514-0
A.Memari (IJSIR.356514-18) 2017; 4
Y.Yan (IJSIR.356514-34) 2021; 28
IJSIR.356514-20
IJSIR.356514-21
IJSIR.356514-22
IJSIR.356514-23
IJSIR.356514-24
IJSIR.356514-25
IJSIR.356514-26
IJSIR.356514-27
IJSIR.356514-7
IJSIR.356514-28
IJSIR.356514-6
IJSIR.356514-29
IJSIR.356514-8
W.Hare (IJSIR.356514-12) 2006; 3
References_xml – ident: IJSIR.356514-0
  doi: 10.17877/DE290R-15667
– ident: IJSIR.356514-13
  doi: 10.7551/mitpress/1090.001.0001
– ident: IJSIR.356514-29
  doi: 10.1109/CEC.2019.8790158
– ident: IJSIR.356514-8
  doi: 10.1007/s10462-020-09906-6
– ident: IJSIR.356514-31
  doi: 10.1023/A:1008202821328
– ident: IJSIR.356514-19
  doi: 10.1162/EVCO_a_00134
– ident: IJSIR.356514-23
  doi: 10.1093/comjnl/7.4.308
– ident: IJSIR.356514-15
  doi: 10.1126/science.220.4598.671
– ident: IJSIR.356514-7
  doi: 10.1007/BF00143070
– ident: IJSIR.356514-21
  doi: 10.1007/s13042-019-01053-x
– ident: IJSIR.356514-30
  doi: 10.1145/3319619.3321996
– ident: IJSIR.356514-33
  doi: 10.1080/10556780902909948
– ident: IJSIR.356514-17
  doi: 10.1016/j.ins.2024.120120
– ident: IJSIR.356514-6
  doi: 10.1145/962437.962444
– ident: IJSIR.356514-27
  doi: 10.1016/j.matcom.2016.05.006
– ident: IJSIR.356514-3
  doi: 10.1007/s101070100263
– ident: IJSIR.356514-20
  doi: 10.1023/A:1026065325419
– ident: IJSIR.356514-2
  doi: 10.1016/j.swevo.2020.100665
– start-page: 169
  year: 2014
  ident: IJSIR.356514-9
  article-title: Online performance measures for metaheuristic optimization.
  publication-title: Proceedings of the International Workshop on Hybrid Metaheuristics
– volume: 28
  start-page: 31
  issue: 1
  year: 2021
  ident: IJSIR.356514-34
  article-title: A modified data profile technology for optimization algorithms competition.
  publication-title: Journal of Dongguan University of Technology
– volume: 4
  start-page: 1
  issue: 6
  year: 2017
  ident: IJSIR.356514-18
  article-title: Metaheuristic algorithms: Guidelines for implementation.
  publication-title: Journal of Soft Computing and Decision Support Systems
– ident: IJSIR.356514-10
– ident: IJSIR.356514-25
  doi: 10.1111/j.1541-0420.2005.00389.x
– ident: IJSIR.356514-24
  doi: 10.1016/j.swevo.2021.100888
– ident: IJSIR.356514-28
  doi: 10.1016/j.ins.2010.12.006
– ident: IJSIR.356514-4
– ident: IJSIR.356514-16
  doi: 10.1109/TEVC.2019.2955110
– ident: IJSIR.356514-22
  doi: 10.1137/080724083
– ident: IJSIR.356514-32
  doi: 10.1287/ijoc.1060.0175
– ident: IJSIR.356514-14
  doi: 10.1109/ICNN.1995.488968
– ident: IJSIR.356514-35
  doi: 10.1109/TEVC.2022.3199647
– volume: 3
  start-page: 545
  year: 2006
  ident: IJSIR.356514-12
  article-title: Benchmark of some nonsmooth optimization solvers for computing nonconvex proximal points.
  publication-title: Pacific Journal of Optimization
– ident: IJSIR.356514-5
  doi: 10.1007/s00500-016-2471-9
– ident: IJSIR.356514-11
  doi: 10.1145/1830761.1830790
– ident: IJSIR.356514-1
  doi: 10.1109/CEC55065.2022.9870433
– ident: IJSIR.356514-26
  doi: 10.1007/BF01742498
SSID ssj0000547753
Score 2.24219
Snippet Numerical comparison is essential for evaluating an optimization algorithm. Unfortunately, recent research has shown that two paradoxes may occur, namely the...
SourceID proquest
crossref
igi
SourceType Aggregation Database
Index Database
Publisher
StartPage 1
SubjectTerms Algorithms
Case studies
Data analysis
Design optimization
Distribution functions
Hypotheses
Hypothesis testing
Methods
Optimization algorithms
Paradoxes
Probability
Title Paradox-Free Data Analysis in Numerical Comparisons of Optimization Algorithms: A Review and Case Study
URI http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/IJSIR.356514
https://www.proquest.com/docview/3117049226
Volume 15
WOSCitedRecordID wos001336693800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVPQU
  databaseName: Computer Science Database
  customDbUrl:
  eissn: 1947-9271
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000547753
  issn: 1947-9263
  databaseCode: K7-
  dateStart: 20230101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/compscijour
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Engineering Database
  customDbUrl:
  eissn: 1947-9271
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000547753
  issn: 1947-9263
  databaseCode: M7S
  dateStart: 20230101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: ProQuest Central
  customDbUrl:
  eissn: 1947-9271
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000547753
  issn: 1947-9263
  databaseCode: BENPR
  dateStart: 20230101
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/central
  providerName: ProQuest
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LTxsxEB41wIFLoQVECkQ-lKMhu7bjNaqEQiAqRUojHhI3y-tHuhJsaHap2n-PvQ8QatUL55VX1ow9883D8wF81koYSinD1orwJIenWGnBsNGDvonjNFSHKrIJPpkkt7di2iTciqatsrWJlaE2cx1y5IckUKRQ4dHC8cNPHFijQnW1odDowHKYkhBVrXtXzzkWD0d4PYjSh-oci3hA6t53H1Qkh-ffrs4vD4hHNBF95ZU62Sz7yzRX_ma89tadrsP7BmmiYX00PsA7m3-EtZbFATWXegNmU7VQZv4bjxfWolNVKtROKkFZjiaPdU3nDo2eKQsLNHfouzc2980rTjS8m_ktlD_uiyM0RHXBAancoJH3kig0K_7ZhJvx2fXoK27oF7COk36JFXF6kAYIJhQTzKWG2chREWrAgpOUeKQpYieosNomxqqImdTYiCRqoLSmZAuW8nlutwH5IMbwuO8cM4yqhAriLLc87dMoSpWzXdhvxS8f6ikb0kcnQU2yUpOs1dSFL143srlmhfTCkUE4MghHtsKRWf6vxRHrwm6rsZd_vKjr0_8_78Bq7BFNnX_ZhaVy8Wj3YEX_KrNi0YPlk7PJ9LIHnQuOe9U5fAKyseOB
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Nb9QwEB21BQkuLZ9iSwEf6NF0E9vrGIHQasuqy5algiL15jr-WCK12bJJgf4pfiN2nLRCIG49cI5iyePnmTczth_Ac62EoZQybK0IV3J4jpUWDBs96Js0zUN3qBGb4LNZdnQkDlbgZ3cXJhyr7Hxi46jNQoca-Q4JEilUeLbw5uwrDqpRobvaSWhEWEztxXefslWvJ7t-fbfTdPz2cLSHW1UBrNOsX2NFnB7kgVkIxQRzuWE2cVSE1qbgJCeeQInUCSqstpmxKmEmNzYhmRoorSnx467CDUoyHvbVlOPLmo6nPzw-fJkIyrFIBySetfdJTLYzefdp8vEF8Qwqob9FwdViXvwRCpr4Nt743yxzB9ZbJo2GEfp3YcWW92CjU6lArdO6D_MDtVRm8QOPl9aiXVUr1L3EgooSzc5jz-oEjS4lGSu0cOiDd6an7S1VNDyZ-ynXX06rl2iIYkMFqdKgkWcBKBzGvHgAn69lug9hrVyU9hEgn6QZnvadY4ZRlVFBnOWW532aJLlytgfb3XLLs_iKiPTZV4CFbGAhIyx68MpjQbZupJLeODIYRwbjyM44sij_9nPCerDVIeRqjCt4bP778zO4tXf4fl_uT2bTx3A79ewt1pq2YK1entsncFN_q4tq-bRBPYLj6wbTL8KgPps
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Paradox-Free+Data+Analysis+in+Numerical+Comparisons+of+Optimization+Algorithms%3A+A+Review+and+Case+Study&rft.jtitle=International+journal+of+swarm+intelligence+research&rft.au=Yunchao+Zhi&rft.au=Liu%2C+Qunfeng&rft.au=Yuan%2C+Yan&rft.au=Jing%2C+Yunpeng&rft.date=2024-01-01&rft.pub=IGI+Global&rft.issn=1947-9263&rft.eissn=1947-9271&rft.volume=15&rft.issue=1&rft.spage=1&rft.epage=22&rft_id=info:doi/10.4018%2FIJSIR.356514
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1947-9263&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1947-9263&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1947-9263&client=summon