A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation

Metaheuristic search techniques have been extensively used to automate the process of generating test cases, and thus providing solutions for a more cost-effective testing process. This approach to test automation, often coined "Search-based Software Testing" (SBST), has been used for a wi...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on software engineering Vol. 36; no. 6; pp. 742 - 762
Main Authors:	Ali, Shaukat, Briand, Lionel C, Hemmati, Hadi, Panesar-Walawege, Rajwinder Kaur
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.11.2010 IEEE Computer Society
Subjects:	Algorithm design and analysis Algorithms Automatic testing Automation Collection Computer programs Cost analysis Costs Digital Object Identifier Empirical analysis Evolutionary computing and genetic algorithms frameworks Genetic algorithms Guidelines Heuristic heuristics design Logic testing Optimization review and evaluation Scalability Searching Software Software engineering Software testing Studies System testing Systematic review test generation testing strategies validation
ISSN:	0098-5589, 1939-3520
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Metaheuristic search techniques have been extensively used to automate the process of generating test cases, and thus providing solutions for a more cost-effective testing process. This approach to test automation, often coined "Search-based Software Testing" (SBST), has been used for a wide variety of test case generation purposes. Since SBST techniques are heuristic by nature, they must be empirically investigated in terms of how costly and effective they are at reaching their test objectives and whether they scale up to realistic development artifacts. However, approaches to empirically study SBST techniques have shown wide variation in the literature. This paper presents the results of a systematic, comprehensive review that aims at characterizing how empirical studies have been designed to investigate SBST cost-effectiveness and what empirical evidence is available in the literature regarding SBST cost-effectiveness and scalability. We also provide a framework that drives the data collection process of this systematic review and can be the starting point of guidelines on how SBST techniques can be empirically assessed. The intent is to aid future researchers doing empirical studies in SBST by providing an unbiased view of the body of empirical evidence and by guiding them in performing well-designed and executed empirical studies.
AbstractList	Metaheuristic search techniques have been extensively used to automate the process of generating test cases, and thus providing solutions for a more cost-effective testing process. This approach to test automation, often coined "Search-based Software Testing" (SBST), has been used for a wide variety of test case generation purposes. Since SBST techniques are heuristic by nature, they must be empirically investigated in terms of how costly and effective they are at reaching their test objectives and whether they scale up to realistic development artifacts. However, approaches to empirically study SBST techniques have shown wide variation in the literature. This paper presents the results of a systematic, comprehensive review that aims at characterizing how empirical studies have been designed to investigate SBST cost-effectiveness and what empirical evidence is available in the literature regarding SBST cost-effectiveness and scalability. We also provide a framework that drives the data collection process of this systematic review and can be the starting point of guidelines on how SBST techniques can be empirically assessed. The intent is to aid future researchers doing empirical studies in SBST by providing an unbiased view of the body of empirical evidence and by guiding them in performing well-designed and executed empirical studies. Metaheuristic search techniques have been extensively used to automate the process of generating test cases, and thus providing solutions for a more cost-effective testing process. This approach to test automation, often coined "Search-based Software Testing" (SBST), has been used for a wide variety of test case generation purposes. Since SBST techniques are heuristic by nature, they must be empirically investigated in terms of how costly and effective they are at reaching their test objectives and whether they scale up to realistic development artifacts. However, approaches to empirically study SBST techniques have shown wide variation in the literature. This paper presents the results of a systematic, comprehensive review that aims at characterizing how empirical studies have been designed to investigate SBST cost-effectiveness and what empirical evidence is available in the literature regarding SBST cost-effectiveness and scalability. We also provide a framework that drives the data collection process of this systematic review and can be the starting point of guidelines on how SBST techniques can be empirically assessed. The intent is to aid future researchers doing empirical studies in SBST by providing an unbiased view of the body of empirical evidence and by guiding them in performing well-designed and executed empirical studies. [PUBLICATION ABSTRACT]
Author	Panesar-Walawege, Rajwinder Kaur Ali, Shaukat Briand, Lionel C Hemmati, Hadi
Author_xml	– sequence: 1 givenname: Shaukat surname: Ali fullname: Ali, Shaukat email: shaukat@simula.no organization: Simula Res. Lab., Lysaker, Norway – sequence: 2 givenname: Lionel C surname: Briand fullname: Briand, Lionel C email: briand@simula.no organization: Simula Res. Lab., Lysaker, Norway – sequence: 3 givenname: Hadi surname: Hemmati fullname: Hemmati, Hadi email: hemmati@simula.no organization: Simula Res. Lab., Lysaker, Norway – sequence: 4 givenname: Rajwinder Kaur surname: Panesar-Walawege fullname: Panesar-Walawege, Rajwinder Kaur email: rpanesar@simula.no organization: Simula Res. Lab., Lysaker, Norway
BookMark	eNp1kEFrGzEQhUVwIY6bU465iJ4KZR3NarWSjq5x04AhULtnIWtHicJ6dyutE_zvq8Shh0BPEvO-ebx5F2TS9R0ScgVsDsD0zXazmpeM6bkoz8gUNNcFFyWbkGkeqkIIpc_JRUpPjDEhpZiShwXdHNOIezsGR3_hc8AX2ns6PiJdDEMbXBb6jtquoav9EGIetPSue8Y0hoeTlvEN2ugei-82YUO3WaPL_KW32GF8gz6TT962CS_f3xn5_WO1Xf4s1ve3d8vFunAc-FhwB1YCV1KAr-yuUV5KKL2vuHCutq7httkpQMuVUGpXgfPal1XlG6ZrlJrPyNeT7xD7P4ccxOxDcti2tsP-kAzU2U_pUtcZ_fIBfeoPscvpjGKyBlBCZujbCXKxTymiN0MMexuPBph57dzkzs1r50aUmYYPtAvj2_1jtKH9z871aScg4j93UQLLCfhf-kyOvw
CODEN	IESEDJ
CitedBy_id	crossref_primary_10_1016_j_infsof_2013_07_007 crossref_primary_10_1007_s10515_016_0197_7 crossref_primary_10_1109_TR_2018_2832072 crossref_primary_10_1002_stvr_1495 crossref_primary_10_1016_j_infsof_2016_09_010 crossref_primary_10_1007_s13369_019_03996_3 crossref_primary_10_1145_3391533 crossref_primary_10_1093_iwc_iww043 crossref_primary_10_1007_s11334_020_00378_z crossref_primary_10_1007_s00500_017_2906_y crossref_primary_10_1016_j_jss_2012_11_045 crossref_primary_10_1016_j_infsof_2019_05_001 crossref_primary_10_1145_3182659 crossref_primary_10_1007_s11063_020_10288_7 crossref_primary_10_1007_s12065_021_00618_6 crossref_primary_10_1007_s13369_017_2796_4 crossref_primary_10_1016_j_infsof_2011_05_004 crossref_primary_10_1016_j_infsof_2018_05_003 crossref_primary_10_1002_stvr_1486 crossref_primary_10_1145_3498326 crossref_primary_10_1007_s11432_020_3403_2 crossref_primary_10_1016_j_infsof_2021_106620 crossref_primary_10_1016_j_infsof_2016_09_002 crossref_primary_10_1109_TSE_2010_62 crossref_primary_10_1007_s10515_016_0203_0 crossref_primary_10_1109_TSE_2014_2327020 crossref_primary_10_1007_s10664_013_9247_y crossref_primary_10_1016_j_jss_2014_08_024 crossref_primary_10_1145_3280988 crossref_primary_10_1007_s10664_024_10451_x crossref_primary_10_1109_TII_2018_2856881 crossref_primary_10_1016_j_jss_2019_110495 crossref_primary_10_1145_3532182 crossref_primary_10_1016_j_swevo_2017_12_009 crossref_primary_10_1145_3616372 crossref_primary_10_1016_j_infsof_2013_03_004 crossref_primary_10_1016_j_infsof_2020_106295 crossref_primary_10_2196_56370 crossref_primary_10_1016_j_infsof_2013_02_006 crossref_primary_10_1109_TSE_2012_14 crossref_primary_10_1007_s13198_017_0646_0 crossref_primary_10_1145_2430536_2430540 crossref_primary_10_1145_3464940 crossref_primary_10_1002_stvr_1470 crossref_primary_10_1016_j_asoc_2024_111586 crossref_primary_10_1016_j_infsof_2014_05_020 crossref_primary_10_1016_j_jss_2016_07_001 crossref_primary_10_1016_j_jss_2018_09_055 crossref_primary_10_1145_3293455 crossref_primary_10_1145_3375636 crossref_primary_10_1145_3617175 crossref_primary_10_1155_2014_642630 crossref_primary_10_1016_j_infsof_2015_01_008 crossref_primary_10_1016_j_jss_2013_07_040 crossref_primary_10_1002_stvr_1627 crossref_primary_10_1007_s10515_025_00539_z crossref_primary_10_1049_iet_sen_2016_0066 crossref_primary_10_1109_TSE_2014_2372785 crossref_primary_10_1016_j_jss_2014_01_010 crossref_primary_10_1016_j_swevo_2016_10_002 crossref_primary_10_1016_j_jcss_2015_12_001 crossref_primary_10_1007_s11219_016_9322_x crossref_primary_10_1016_j_eswa_2025_126634 crossref_primary_10_1016_j_jss_2014_09_042 crossref_primary_10_1007_s40747_020_00226_4 crossref_primary_10_1016_j_jss_2019_04_049 crossref_primary_10_1145_1735223_1735248 crossref_primary_10_1016_j_knosys_2020_106446 crossref_primary_10_4018_JITR_2018010110 crossref_primary_10_1109_ACCESS_2019_2917913 crossref_primary_10_1109_TSE_2010_107 crossref_primary_10_1002_smr_2520 crossref_primary_10_1049_iet_sen_2018_5133 crossref_primary_10_1145_3408301 crossref_primary_10_1016_j_infsof_2016_01_017 crossref_primary_10_1016_j_jss_2023_111669 crossref_primary_10_1007_s10515_017_0215_4 crossref_primary_10_1145_3609427 crossref_primary_10_1016_j_jss_2019_03_011 crossref_primary_10_1007_s10664_020_09937_1 crossref_primary_10_1007_s10664_021_10048_8 crossref_primary_10_1002_smr_2679 crossref_primary_10_1016_j_jss_2020_110570 crossref_primary_10_1145_3652157 crossref_primary_10_1145_3477271 crossref_primary_10_1007_s10664_013_9249_9 crossref_primary_10_1109_TEVC_2017_2778560 crossref_primary_10_1155_2015_617685 crossref_primary_10_1109_ACCESS_2018_2809600 crossref_primary_10_1049_iet_sen_2018_5143 crossref_primary_10_3390_app112110166 crossref_primary_10_1002_spe_2654 crossref_primary_10_1109_TSE_2013_17 crossref_primary_10_1109_TSE_2015_2487958 crossref_primary_10_1007_s10664_012_9219_7 crossref_primary_10_1016_j_infsof_2014_06_009 crossref_primary_10_1109_TSE_2022_3227418 crossref_primary_10_1145_3593801 crossref_primary_10_1186_2195_1721_2_3 crossref_primary_10_1109_ACCESS_2022_3178438 crossref_primary_10_1007_s10664_015_9392_6 crossref_primary_10_3390_sym11091145 crossref_primary_10_1007_s10664_017_9500_x crossref_primary_10_1109_TSE_2024_3381015 crossref_primary_10_1002_stvr_1838 crossref_primary_10_1145_3624745 crossref_primary_10_1016_j_asoc_2021_107246 crossref_primary_10_1109_MCI_2017_2670462 crossref_primary_10_1109_TSE_2023_3343753 crossref_primary_10_1016_j_asoc_2016_08_030 crossref_primary_10_1049_trit_2017_0004 crossref_primary_10_1016_j_infsof_2016_04_015 crossref_primary_10_1016_j_infsof_2017_01_001 crossref_primary_10_1145_3381032 crossref_primary_10_1007_s10664_015_9424_2 crossref_primary_10_1016_j_jss_2012_10_027 crossref_primary_10_1145_3585009 crossref_primary_10_1049_iet_sen_2017_0112 crossref_primary_10_1016_j_jss_2021_111166 crossref_primary_10_1016_j_jss_2011_06_004 crossref_primary_10_4018_IJAMC_2017100102 crossref_primary_10_1109_TSE_2025_3539458 crossref_primary_10_3390_sym14081525 crossref_primary_10_1007_s10664_014_9301_4 crossref_primary_10_1016_j_jss_2012_07_041 crossref_primary_10_1007_s10664_011_9154_z crossref_primary_10_1007_s10664_022_10127_4 crossref_primary_10_1007_s13198_021_01099_7 crossref_primary_10_1007_s10515_018_0246_5 crossref_primary_10_1016_j_jestch_2020_12_011 crossref_primary_10_1007_s42979_021_00631_7 crossref_primary_10_1016_j_swevo_2014_10_003 crossref_primary_10_1002_stvr_1701 crossref_primary_10_1109_TSE_2017_2702606 crossref_primary_10_1002_cae_22430 crossref_primary_10_1007_s10664_013_9287_3 crossref_primary_10_1007_s10270_021_00924_8 crossref_primary_10_1002_stvr_1660 crossref_primary_10_1007_s11704_014_3496_9 crossref_primary_10_1109_ACCESS_2020_2971712 crossref_primary_10_1109_MC_2011_263 crossref_primary_10_1109_ACCESS_2023_3312556 crossref_primary_10_1109_TSE_2011_93 crossref_primary_10_1109_ACCESS_2017_2771562 crossref_primary_10_1145_2492248_2492277 crossref_primary_10_1145_2379776_2379787 crossref_primary_10_1007_s10664_021_10024_2 crossref_primary_10_1007_s00766_015_0225_3 crossref_primary_10_1016_j_jss_2013_02_061 crossref_primary_10_1145_2492248_2492274 crossref_primary_10_1016_j_jss_2015_11_036 crossref_primary_10_1155_2016_5309348 crossref_primary_10_1007_s10515_018_0232_y crossref_primary_10_1145_3491038 crossref_primary_10_1016_j_jss_2023_111629 crossref_primary_10_1109_TSE_2011_18
Cites_doi	10.1007/0-387-28356-0 10.1016/j.jss.2007.08.028 10.1145/1287624.1287647 10.1109/MS.2005.6 10.1109/FOSE.2007.29 10.1109/2.294849 10.1201/9781420036268 10.1016/j.infsof.2008.12.005 10.1109/ASE.2000.873665 10.1109/CMPSAC.2004.1342808 10.1145/1276958.1277175 10.1002/spe.684 10.1007/s10710-006-9003-9 10.1109/TSE.2002.1027796 10.1109/REAL.1998.739738 10.1145/1068009.1068183 10.1002/stvr.294 10.1007/s10664-006-9026-0 10.1016/j.asoc.2004.08.004 10.1109/CEC.2003.1299837 10.1049/sej.1996.0040 10.1145/271771.271792 10.1109/ICSE.2004.1317449 10.1145/1143997.1144319 10.1145/1273463.1273475 10.1016/j.infsof.2005.06.006 10.1109/FOSE.2007.25 10.1016/S0950-5849(01)00189-6 10.1016/S0950-5849(01)00190-2 10.1145/154183.154265 10.1145/1321631.1321689 10.1016/j.infsof.2005.08.009 10.1145/1292414.1292419 10.1049/ip-sen:20030559 10.1109/32.988709 10.1007/978-1-4615-4625-2
ContentType	Journal Article
Copyright	Copyright IEEE Computer Society Nov 2010
Copyright_xml	– notice: Copyright IEEE Computer Society Nov 2010
DBID	97E RIA RIE AAYXX CITATION 3V. 7WY 7WZ 7X7 7XB 87Z 88E 88F 88I 88K 8AL 8FE 8FG 8FI 8FJ 8FK 8FL 8G5 ABJCF ABUWG AFKRA ARAPS AZQEC BENPR BEZIV BGLVJ CCPQU DWQXO FRNLG FYUFA F~G GHDGH GNUQQ GUQSH HCIFZ JQ2 K60 K6~ K7- K9. L.- L6V M0C M0N M0S M1P M1Q M2O M2P M2T M7S MBDVC P5Z P62 PHGZM PHGZT PJZUB PKEHL PPXIY PQBIZ PQBZA PQEST PQGLB PQQKQ PQUKI PRINS PTHSS Q9U 7SC 7SP 8FD F28 FR3 L7M L~C L~D
DOI	10.1109/TSE.2009.52
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library (IEL) (UW System Shared) CrossRef ProQuest Central (Corporate) ABI/INFORM Collection ABI/INFORM Global (PDF only) Health & Medical Collection ProQuest Central (purchase pre-March 2016) ABI/INFORM Global (Alumni Edition) Medical Database (Alumni Edition) Military Database (Alumni Edition) Science Database (Alumni Edition) Telecommunications (Alumni Edition) Computing Database (Alumni Edition) ProQuest SciTech Collection ProQuest Technology Collection Hospital Premium Collection Hospital Premium Collection (Alumni Edition) ProQuest Central (Alumni) (purchase pre-March 2016) ABI/INFORM Collection (Alumni Edition) Research Library (Alumni Edition) Materials Science & Engineering Collection (subscription) ProQuest Central (Alumni) ProQuest Central UK/Ireland ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Central (subscription) Business Premium Collection Technology collection ProQuest One ProQuest Central Business Premium Collection (Alumni) Health Research Premium Collection ABI/INFORM Global (Corporate) Health Research Premium Collection (Alumni) ProQuest Central Student Research Library Prep SciTech Premium Collection ProQuest Computer Science Collection ProQuest Business Collection (Alumni Edition) ProQuest Business Collection Computer Science Database ProQuest Health & Medical Complete (Alumni) ABI/INFORM Professional Advanced ProQuest Engineering Collection ABI/INFORM Global Computing Database Health & Medical Collection (Alumni Edition) PML(ProQuest Medical Library) Military Database ProQuest Research Library Science Database (subscription) Telecommunications Database Engineering Database (subscription) Research Library (Corporate) ProQuest advanced technologies & aerospace journals ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic ProQuest Health & Medical Research Collection ProQuest One Academic Middle East (New) ProQuest One Health & Nursing ProQuest One Business (UW System Shared) ProQuest One Business (Alumni) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering collection ProQuest Central Basic Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ANTE: Abstracts in New Technology & Engineering Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef ProQuest Business Collection (Alumni Edition) Research Library Prep Computer Science Database ProQuest Central Student ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection SciTech Premium Collection ProQuest Military Collection ProQuest Central China ABI/INFORM Complete ProQuest Telecommunications ProQuest One Applied & Life Sciences Health Research Premium Collection Health & Medical Research Collection ProQuest Central (New) ProQuest Medical Library (Alumni) Engineering Collection Advanced Technologies & Aerospace Collection Business Premium Collection ABI/INFORM Global Engineering Database ProQuest Science Journals (Alumni Edition) ProQuest One Academic Eastern Edition ProQuest Hospital Collection ProQuest Technology Collection Health Research Premium Collection (Alumni) ProQuest Telecommunications (Alumni Edition) ProQuest Business Collection ProQuest Hospital Collection (Alumni) ProQuest Health & Medical Complete ProQuest One Academic UKI Edition ProQuest One Academic ProQuest One Academic (New) ABI/INFORM Global (Corporate) ProQuest One Business Technology Collection ProQuest One Academic Middle East (New) ProQuest Health & Medical Complete (Alumni) ProQuest Central (Alumni Edition) ProQuest One Community College ProQuest One Health & Nursing Research Library (Alumni Edition) ProQuest Central ABI/INFORM Professional Advanced ProQuest Health & Medical Research Collection ProQuest Engineering Collection Health and Medicine Complete (Alumni Edition) ProQuest Central Korea ProQuest Research Library ABI/INFORM Complete (Alumni Edition) ProQuest Computing ABI/INFORM Global (Alumni Edition) ProQuest Central Basic ProQuest Science Journals ProQuest Computing (Alumni Edition) ProQuest Military Collection (Alumni Edition) ProQuest SciTech Collection Advanced Technologies & Aerospace Database ProQuest Medical Library Materials Science & Engineering Collection ProQuest One Business (Alumni) ProQuest Central (Alumni) Business Premium Collection (Alumni) Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace ANTE: Abstracts in New Technology & Engineering Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database ProQuest Business Collection (Alumni Edition)
Database_xml	– sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher – sequence: 2 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	1939-3520
EndPage	762
ExternalDocumentID	2194227931 10_1109_TSE_2009_52 5210118
Genre	orig-research Feature
GroupedDBID	--Z -DZ -~X .4S .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 7WY 7X7 85S 88E 88I 8FE 8FG 8FI 8FJ 8FL 8G5 8R4 8R5 97E 9M8 AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABJCF ABPPZ ABQJQ ABUWG ABVLG ACGFO ACGOD ACIWK ACNCT ADBBV AENEX AETIX AFKRA AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ARAPS ARCSS ASUFR ATWAV AZQEC BEFXN BENPR BEZIV BFFAM BGLVJ BGNUA BKEBE BKOMP BPEOZ BPHCQ BVXVI CCPQU CS3 DU5 DWQXO E.L EBS EDO EJD FRNLG FYUFA GNUQQ GROUPED_ABI_INFORM_RESEARCH GUQSH HCIFZ HMCUK HZ~ H~9 I-F IBMZZ ICLAB IEDLZ IFIPE IFJZH IPLJI ITG ITH JAVBF K60 K6V K6~ K7- L6V LAI M0C M1P M1Q M2O M2P M43 M7S MS~ O9- OCL OHT P2P P62 PHGZM PHGZT PJZUB PPXIY PQBIZ PQBZA PQGLB PQQKQ PROAC PSQYO PTHSS PUEGO Q2X RIA RIE RNI RNS RXW RZB S10 TAE TN5 TWZ UHB UKHRP UPT UQL VH1 WH7 XOL YYP YZZ ZCG AAYXX AFFHD CITATION 3V. 7XB 88K 8AL 8FK JQ2 K9. L.- M0N M2T MBDVC PKEHL PQEST PQUKI PRINS Q9U 7SC 7SP 8FD F28 FR3 L7M L~C L~D
ID	FETCH-LOGICAL-c313t-3c1a7138751f4abd8f7712ff435cc6acd3adb81ea38588b41cf9f244fd096e793
IEDL.DBID	RIE
ISICitedReferencesCount	279
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000284424100003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0098-5589
IngestDate	Sat Sep 27 22:29:31 EDT 2025 Mon Oct 06 17:00:17 EDT 2025 Sat Nov 29 08:10:00 EST 2025 Tue Nov 18 21:05:11 EST 2025 Wed Aug 27 02:47:42 EDT 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	6
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c313t-3c1a7138751f4abd8f7712ff435cc6acd3adb81ea38588b41cf9f244fd096e793
Notes	SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-1 ObjectType-Feature-2 content type line 23
PQID	807611857
PQPubID	21418
PageCount	21
ParticipantIDs	proquest_journals_807611857 ieee_primary_5210118 crossref_citationtrail_10_1109_TSE_2009_52 crossref_primary_10_1109_TSE_2009_52 proquest_miscellaneous_1671289296
PublicationCentury	2000
PublicationDate	2010-11-01
PublicationDateYYYYMMDD	2010-11-01
PublicationDate_xml	– month: 11 year: 2010 text: 2010-11-01 day: 01
PublicationDecade	2010
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on software engineering
PublicationTitleAbbrev	TSE
PublicationYear	2010
Publisher	IEEE IEEE Computer Society
Publisher_xml	– name: IEEE – name: IEEE Computer Society
References	ref13 ref12 ref15 ref53 Zhan (ref54) ref52 ref11 ref55 ref10 (ref1) 2008 Drechsler (ref17) 2002 Haupt (ref26) 1997 Capon (ref14) 1988 ref19 ref18 Moore (ref40) 2002 Kitchenham (ref30) 2007 ref51 ref50 ref46 ref45 ref48 ref47 ref44 Mathur (ref36) 2008 ref43 Johnson (ref27) ref49 (ref2) 2004 ref8 ref7 ref9 ref4 Osman (ref41) 1996 Ali (ref5) 2008 Beizer (ref6) 1990 ref35 ref37 ref31 ref33 ref32 ref39 ref38 Coley (ref16) 1997 Luke (ref34) 2007 (ref3) 2008 Pohlheim (ref42) 2007 Hart (ref25) 1999 ref24 ref23 ref20 ref22 ref21 ref28 Khan (ref29) 2003
References_xml	– ident: ref12 doi: 10.1007/0-387-28356-0 – ident: ref13 doi: 10.1016/j.jss.2007.08.028 – ident: ref22 doi: 10.1145/1287624.1287647 – volume-title: Systematic Review to Support Evidence-Based Medicine: How to Review and Apply Findings of Healthcare Research year: 2003 ident: ref29 – ident: ref19 doi: 10.1109/MS.2005.6 – ident: ref21 doi: 10.1109/FOSE.2007.29 – ident: ref46 doi: 10.1109/2.294849 – ident: ref44 doi: 10.1201/9781420036268 – volume-title: “Genetic Algorithms Framework, “ Rubicite Interactive year: 2004 ident: ref2 – ident: ref4 doi: 10.1016/j.infsof.2008.12.005 – start-page: 215 volume-title: Proc. Data Structures, Near Neighbor Searches, and Methodology: Fifth and Sixth DIMACS Implementation Challenges ident: ref27 article-title: A Theoreticians Guide to the Experimental Analysis of Algorithms – ident: ref10 doi: 10.1109/ASE.2000.873665 – volume-title: Metaheuristics: Theory and Applications year: 1996 ident: ref41 – ident: ref45 doi: 10.1109/CMPSAC.2004.1342808 – volume-title: Introduction to the Practice of Statistics year: 2002 ident: ref40 – volume-title: “UML Profile for Modeling and Analysis of Real-Time and Embedded Systems (MARTE), “ Object Management Group (OMG) year: 2008 ident: ref3 – volume-title: Elementary Statistics for the Social Sciences year: 1988 ident: ref14 – volume-title: Technical Report Simula.SE.293, Simula Research Laboratory year: 2008 ident: ref5 article-title: A Systematic Review of the Application and Empirical Investigation of Evolutionary Testing – year: 2008 ident: ref1 article-title: Computer Science Conference Ranking – ident: ref33 doi: 10.1145/1276958.1277175 – ident: ref50 doi: 10.1002/spe.684 – ident: ref9 doi: 10.1007/s10710-006-9003-9 – ident: ref32 doi: 10.1109/TSE.2002.1027796 – ident: ref43 doi: 10.1109/REAL.1998.739738 – ident: ref8 doi: 10.1145/1068009.1068183 – ident: ref37 doi: 10.1002/stvr.294 – ident: ref53 doi: 10.1007/s10664-006-9026-0 – year: 2007 ident: ref34 article-title: A Java-Based Evolutionary Computation Research System – ident: ref35 doi: 10.1016/j.asoc.2004.08.004 – ident: ref20 doi: 10.1109/CEC.2003.1299837 – ident: ref28 doi: 10.1049/sej.1996.0040 – volume-title: Technical Report EBSE-2007-01 year: 2007 ident: ref30 article-title: Guidelines for Performing Systematic Literature Reviews in Software Engineering – ident: ref47 doi: 10.1145/271771.271792 – volume-title: An Introduction to Genetic Algorithms for Scientists and Engineers year: 1997 ident: ref16 – ident: ref31 doi: 10.1109/ICSE.2004.1317449 – volume-title: Evolutionary Algorithms for Embedded System Design year: 2002 ident: ref17 – ident: ref55 doi: 10.1145/1143997.1144319 – ident: ref24 doi: 10.1145/1273463.1273475 – ident: ref39 doi: 10.1016/j.infsof.2005.06.006 – ident: ref7 doi: 10.1109/FOSE.2007.25 – volume-title: Foundations of Software Testing year: 2008 ident: ref36 – volume-title: Software Testing Techniques year: 1990 ident: ref6 – volume-title: Doing a Literature Review: Releasing the Social Science Research Imagination year: 1999 ident: ref25 – volume-title: Proc. Genetic and Evolutionary Computation Conf ident: ref54 article-title: Search-Based Mutation Testing for Simulink Models – ident: ref23 doi: 10.1016/S0950-5849(01)00189-6 – ident: ref51 doi: 10.1016/S0950-5849(01)00190-2 – ident: ref48 doi: 10.1145/154183.154265 – ident: ref49 doi: 10.1145/1321631.1321689 – ident: ref18 doi: 10.1016/j.infsof.2005.08.009 – volume-title: Practical Genetic Algorithms year: 1997 ident: ref26 – year: 2007 ident: ref42 article-title: GEATbx—The Genetic and Evolutionary Algorithm Toolbox for Matlab – ident: ref11 doi: 10.1145/1292414.1292419 – ident: ref15 doi: 10.1049/ip-sen:20030559 – ident: ref38 doi: 10.1109/32.988709 – ident: ref52 doi: 10.1007/978-1-4615-4625-2
SSID	ssj0005775
Score	2.4830172
SecondaryResourceType	review_article
Snippet	Metaheuristic search techniques have been extensively used to automate the process of generating test cases, and thus providing solutions for a more...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	742
SubjectTerms	Algorithm design and analysis Algorithms Automatic testing Automation Collection Computer programs Cost analysis Costs Digital Object Identifier Empirical analysis Evolutionary computing and genetic algorithms frameworks Genetic algorithms Guidelines Heuristic heuristics design Logic testing Optimization review and evaluation Scalability Searching Software Software engineering Software testing Studies System testing Systematic review test generation testing strategies validation
SummonAdditionalLinks	– databaseName: Military Database dbid: M1Q link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8NAEB58Hbz4FuuLFXoSlmbbJtk9SS0tXhSlFXoLm32IoGm11d_v7GYTFcWLt8AMIcns7MxkZr8PoIkxWeBS0DRPc027eaRoznNFTawSpaLUJMZ6son05oZPJuI2zObMw1hltSf6jVpPlftH3uKu4HbARRezF-pIo1xzNTBoLMMqioSDzr9md58THmkaV5CZccxFOJ7HItEajwYlVmXc_haQPMPKj23Zx5rh5j-fcgs2QpJJeuWq2IYlU-zAZkXgQII_78JDj4xqJGdStgnI1BJMCknvs7NNZKHJ4Hn26OFEyBdoDpShejmyTC8xIGoyRhnp4yUpEa2d0h7cDwfj_hUNzAtUdVhnQTuKSaxesZZhtitzzW2asra1mFsplUilO1LnnBnp2oo87zJlhcVEwWqsiAy6_D6sFNPCHADhkW0rzEKklnHXJlhBWR4bI3mSCGZY1IDz6vNnKsCSO3aMp8yXJ5HI0FaOLFNkcbsBzVp5VqJx_K626-xYq2CK4g7YNuCoMloWPHWe1RZrwFktRRdzfRNZmOnbPGMJvjzHPDI5_PMOR7DuZwv8ScVjWFm8vpkTWFPvi8f566lfmx_AceyT priority: 102 providerName: ProQuest
Title	A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation
URI	https://ieeexplore.ieee.org/document/5210118 https://www.proquest.com/docview/807611857 https://www.proquest.com/docview/1671289296
Volume	36
WOSCitedRecordID	wos000284424100003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE/IET Electronic Library (IEL) (UW System Shared) customDbUrl: eissn: 1939-3520 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: RIE dateStart: 19750101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVPQU databaseName: ABI/INFORM Collection customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: 7WY dateStart: 19870101 isFulltext: true titleUrlDefault: https://www.proquest.com/abicomplete providerName: ProQuest – providerCode: PRVPQU databaseName: ABI/INFORM Global (OCUL) customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: M0C dateStart: 19870101 isFulltext: true titleUrlDefault: https://search.proquest.com/abiglobal providerName: ProQuest – providerCode: PRVPQU databaseName: Computer Science Database customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: K7- dateStart: 19870101 isFulltext: true titleUrlDefault: http://search.proquest.com/compscijour providerName: ProQuest – providerCode: PRVPQU databaseName: Engineering Database (subscription) customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: M7S dateStart: 19870101 isFulltext: true titleUrlDefault: http://search.proquest.com providerName: ProQuest – providerCode: PRVPQU databaseName: Health & Medical Collection customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: 7X7 dateStart: 19870101 isFulltext: true titleUrlDefault: https://search.proquest.com/healthcomplete providerName: ProQuest – providerCode: PRVPQU databaseName: Military Database customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: M1Q dateStart: 19870101 isFulltext: true titleUrlDefault: https://search.proquest.com/military providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest advanced technologies & aerospace journals customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: P5Z dateStart: 19870101 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: BENPR dateStart: 19870101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Research Library customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: M2O dateStart: 19870101 isFulltext: true titleUrlDefault: https://search.proquest.com/pqrl providerName: ProQuest – providerCode: PRVPQU databaseName: Science Database (subscription) customDbUrl: eissn: 1939-3520 dateEnd: 20120331 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: M2P dateStart: 19870101 isFulltext: true titleUrlDefault: https://search.proquest.com/sciencejournals providerName: ProQuest
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb9swDCbabodd-lg3LO0WqEBOw7RaTvTwMQlSDCiWZkuGZbsYsh5DgNYpmmS_f5TsuC3ayy6EDBGGbUomKZIfATqokzNcCpYWsrC0VySGFqow1HEjjEmkE87HZhNyPFbzeTbZgU9NLYxzLiafuc9hGGP5dmk24ajsHFVNKJTchV0pRVWrdZ_OISXf4mNyrrK6Fo8l2flsOqqAKXn6SPvEdipP_sFRsVwc_N8jHcJ-bUCSfiXxI9hx5Ws42DZnIPVePYY_fTJtUJpJFQIgS0_Q4CP9-6g10aUlo5vbRYQKIQ9gN3AO2at0ZDpAZWfJDOfIEIekQqsOTG_gx8VoNvxC664K1HRZd027hmn0TNFPYb6nC6u8lCz1Hu0mY4Q2tqttoZjTIWSoih4zPvNoBHiL3o7D7fwW9spl6d4BUYlPDVoY2mre8wK9I6-4c1oJkTHHkhZ83H7t3NSQ46HzxXUeXY8ky1E0oRFmlvO0BZ2G-bZC2nie7ThIoWGpBdCC060Y83oXrnIVDmkC2FULzppZ3D4hJqJLt9yscibw5RXaiOLk-RufwquYMBDLD9_D3vpu4z7AS_N3vVjdtXEl_vwV6Fy24cVgNJ58x6tLSZF-TYaBsm-BpleRTiKdBSqnSCf8dzuu43_9s-bw
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1LSxxBEC5EA3pRExXXR9KCXgKN07Mz0z0HkY2uKGsWwQ14G3v6IQvJ7OquCf4o_6PVPQ-VSG4echvooqFnvq7HVNVXALtok1OEgqY5zzWN8kDRXOSKmlglSgXcJMb6YRO83xdXV-nFDDzWvTCurLLWiV5R65Fy_8j3hQu4HXHR4fiWuqFRLrlaT9AoUdEzD38wYpscnB3j590Lw5Pu4OiUVkMFqGqz9pS2FZMYmKGbzmwkcy0s5yy0Ft0GpRKpdFvqXDAjXcZM5BFTNrVoA61GZ99wx72EGn8uwh3ctepx-lxRwnlcU3TGsUirdkAWpPuDy27JjRmHrwygn-jylxnwtu1k6T97K8uwWDnRpFOi_iPMmOITLNUDKkilr1bgpkMuG6ZqUqZByMgSdHpJ5zlzT2ShSffXeOjpUsgL6hFcQ_GyJJt-Q4OvyQDXyBE-kpKx2wmtwo93Oe4azBajwqwDEYENFXpZUss4sglGiFbExkiRJCkzLGjB1_pzZ6qiXXfTP35mPvwK0gyx4YaBplkctmC3ER6XbCNvi6043DQi6IK5BuIWbNYgySpNNMkahLRgp1lFFeLyQrIwo_tJxhI8vEA_Odn45w5fYP508P08Oz_r9zZhwddR-K7MLZid3t2bbfigfk-Hk7vP_l4QuH5vyD0BzV5KOw
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Systematic+Review+of+the+Application+and+Empirical+Investigation+of+Search-Based+Test+Case+Generation&rft.jtitle=IEEE+transactions+on+software+engineering&rft.au=Ali%2C+Shaukat&rft.au=Briand%2C+Lionel+C&rft.au=Hemmati%2C+Hadi&rft.au=Panesar-Walawege%2C+Rajwinder+Kaur&rft.date=2010-11-01&rft.issn=0098-5589&rft.volume=36&rft.issue=6&rft.spage=742&rft.epage=762&rft_id=info:doi/10.1109%2FTSE.2009.52&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0098-5589&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0098-5589&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0098-5589&client=summon