The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs

The field of automated software repair lacks a set of common benchmark problems. Although benchmark sets are used widely throughout computer science, existing benchmarks are not easily adapted to the problem of automatic defect repair, which has several special requirements. Most important of these...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on software engineering Ročník 41; číslo 12; s. 1236 - 1256
Hlavní autoři:	Le Goues, Claire, Holtschulte, Neal, Smith, Edward K., Brun, Yuriy, Devanbu, Premkumar, Forrest, Stephanie, Weimer, Westley
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York IEEE 01.12.2015 IEEE Computer Society
Témata:	<sc xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">IntroClass <sc xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">ManyBugs Algorithms Automated program repair Automation benchmark Benchmark testing Benchmarks C (programming language) C language Categories Computer bugs Computer programs Computer science Datasets Debugging Defects Electronic mail Maintenance engineering Repair Reproducibility Software Software systems Studies subject defect IntroClass Automated program repair ManyBugs reproducibility benchmark subject defect
ISSN:	0098-5589, 1939-3520
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	The field of automated software repair lacks a set of common benchmark problems. Although benchmark sets are used widely throughout computer science, existing benchmarks are not easily adapted to the problem of automatic defect repair, which has several special requirements. Most important of these is the need for benchmark programs with reproducible, important defects and a deterministic method for assessing if those defects have been repaired. This article details the need for a new set of benchmarks, outlines requirements, and then presents two datasets, ManyBugs and IntroClass, consisting between them of 1,183 defects in 15 C programs. Each dataset is designed to support the comparative evaluation of automatic repair algorithms asking a variety of experimental questions. The datasets have empirically defined guarantees of reproducibility and benchmark quality, and each study object is categorized to facilitate qualitative evaluation and comparisons by category of bug or program. The article presents baseline experimental results on both datasets for three existing repair methods, GenProg, AE, and TrpAutoRepair, to reduce the burden on researchers who adopt these datasets for their own comparative evaluations.
AbstractList	The field of automated software repair lacks a set of common benchmark problems. Although benchmark sets are used widely throughout computer science, existing benchmarks are not easily adapted to the problem of automatic defect repair, which has several special requirements. Most important of these is the need for benchmark programs with reproducible, important defects and a deterministic method for assessing if those defects have been repaired. This article details the need for a new set of benchmarks, outlines requirements, and then presents two datasets, ManyBugs and IntroClass, consisting between them of 1,183 defects in 15 C programs. Each dataset is designed to support the comparative evaluation of automatic repair algorithms asking a variety of experimental questions. The datasets have empirically defined guarantees of reproducibility and benchmark quality, and each study object is categorized to facilitate qualitative evaluation and comparisons by category of bug or program. The article presents baseline experimental results on both datasets for three existing repair methods, GenProg, AE, and TrpAutoRepair, to reduce the burden on researchers who adopt these datasets for their own comparative evaluations.
Author	Holtschulte, Neal Weimer, Westley Brun, Yuriy Forrest, Stephanie Smith, Edward K. Le Goues, Claire Devanbu, Premkumar
Author_xml	– sequence: 1 givenname: Claire surname: Le Goues fullname: Le Goues, Claire email: clegoues@cs.cmu.edu organization: Sch. of Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, USA – sequence: 2 givenname: Neal surname: Holtschulte fullname: Holtschulte, Neal email: neal.holts@cs.unm.edu organization: Dept. of Comput. Sci., Univ. of New Mexico, Albuquerque, NM, USA – sequence: 3 givenname: Edward K. surname: Smith fullname: Smith, Edward K. email: tedks@cs.umass.edu organization: Coll. of Inf. & Comput. Sci., Univ. of Massachusetts at Amherst, Amherst, MA, USA – sequence: 4 givenname: Yuriy surname: Brun fullname: Brun, Yuriy email: brun@cs.umass.edu organization: Coll. of Inf. & Comput. Sci., Univ. of Massachusetts at Amherst, Amherst, MA, USA – sequence: 5 givenname: Premkumar surname: Devanbu fullname: Devanbu, Premkumar email: devanbu@cs.ucdavis.edu organization: Dept. of Comput. Sci., Univ. of California at Davis, Davis, CA, USA – sequence: 6 givenname: Stephanie surname: Forrest fullname: Forrest, Stephanie email: forrest@cs.unm.edu organization: Dept. of Comput. Sci., Univ. of New Mexico, Albuquerque, NM, USA – sequence: 7 givenname: Westley surname: Weimer fullname: Weimer, Westley email: weimer@cs.virginia.edu organization: Dept. of Comput. Sci., Univ. of Virginia, Charlottesville, VA, USA
BookMark	eNp9kD1PwzAQQC0EEuVjR2KxxMKS4vNXnJFWBSoVgaDMkbEvEEjjYidD_z2pihgYmLy8d3d-R2S_DS0ScgZsDMCKq-XzbMwZqDGXSioQe2QEhSgyoTjbJyPGCpMpZYpDcpTSB2NM5bkakfnyHem9bTeT_i1R23o6b7sYpo1NiU6wde8rGz8TrUKk130XVrZDT59wbetIQ0Wn9DGGt2hX6YQcVLZJePrzHpOXm9lyepctHm7n0-tF5kTBuswaX4F_NQ68cUZ5qRSXUgoN2mrtjHG5fZXIvCh47q0Cz1ACGOEQpUYpjsnlbu46hq8eU1eu6uSwaWyLoU8l5EZzYJybAb34g36EPrbDdQOlAYBzLQaK7SgXQ0oRq3Id6-HXmxJYuW1bDm3Lbdvyp-2g6D-Kqzvb1WFoZ-vmP_F8J9aI-LsnByVUzsQ3xNGGPQ
CODEN	IESEDJ
CitedBy_id	crossref_primary_10_1145_3340544 crossref_primary_10_1145_3418461 crossref_primary_10_1007_s10664_020_09931_7 crossref_primary_10_1109_ACCESS_2019_2901951 crossref_primary_10_1145_3576039 crossref_primary_10_1109_TSE_2024_3492204 crossref_primary_10_1145_3383390 crossref_primary_10_1007_s10664_025_10614_4 crossref_primary_10_1016_j_jss_2024_112083 crossref_primary_10_1145_3702971 crossref_primary_10_1145_3631974 crossref_primary_10_1145_3695997 crossref_primary_10_1109_TSE_2019_2944914 crossref_primary_10_1109_TSE_2021_3067156 crossref_primary_10_1109_ACCESS_2024_3449106 crossref_primary_10_1007_s10489_016_0804_0 crossref_primary_10_1145_3733599 crossref_primary_10_1109_TR_2020_3003313 crossref_primary_10_1145_3241980 crossref_primary_10_1109_TSE_2022_3164662 crossref_primary_10_1145_3597617 crossref_primary_10_1007_s11432_023_4127_5 crossref_primary_10_1145_3680466 crossref_primary_10_1016_j_jss_2018_03_046 crossref_primary_10_3390_systems8010008 crossref_primary_10_1145_3712190 crossref_primary_10_1007_s10664_022_10189_4 crossref_primary_10_1016_j_jss_2023_111918 crossref_primary_10_1109_TSE_2022_3154717 crossref_primary_10_1145_3744900 crossref_primary_10_1016_j_eswa_2020_114503 crossref_primary_10_1016_j_cola_2019_03_006 crossref_primary_10_1145_3729404 crossref_primary_10_1007_s10664_021_10100_7 crossref_primary_10_1016_j_cosrev_2025_100732 crossref_primary_10_1109_TSE_2018_2884955 crossref_primary_10_1016_j_jss_2016_04_002 crossref_primary_10_1109_TSE_2022_3218264 crossref_primary_10_1145_3022671_2984031 crossref_primary_10_1007_s10664_017_9562_9 crossref_primary_10_1007_s10664_024_10440_0 crossref_primary_10_1109_TSE_2020_3013438 crossref_primary_10_1016_j_jss_2022_111423 crossref_primary_10_3390_sym10040092 crossref_primary_10_1007_s11859_018_1358_2 crossref_primary_10_1007_s10664_016_9470_4 crossref_primary_10_1016_j_jss_2023_111805 crossref_primary_10_1145_3360004 crossref_primary_10_1016_j_jss_2020_110817 crossref_primary_10_1002_smr_2181 crossref_primary_10_1145_3624741 crossref_primary_10_1002_stvr_1751 crossref_primary_10_1109_TSE_2020_2998785 crossref_primary_10_1016_j_ijcip_2022_100527 crossref_primary_10_1016_j_jss_2019_01_069 crossref_primary_10_1109_TSE_2016_2560811 crossref_primary_10_1145_3105906 crossref_primary_10_1145_3731559 crossref_primary_10_3390_sym12122102 crossref_primary_10_1007_s10664_020_09807_w crossref_primary_10_1007_s10115_025_02383_9 crossref_primary_10_1007_s10664_017_9550_0 crossref_primary_10_1002_smr_2173 crossref_primary_10_1007_s10664_024_10506_z crossref_primary_10_1145_3715107 crossref_primary_10_1007_s10664_017_9577_2 crossref_primary_10_1007_s10664_021_10010_8 crossref_primary_10_1016_j_csi_2024_103951 crossref_primary_10_1016_j_jss_2023_111934 crossref_primary_10_1145_3715786 crossref_primary_10_1016_j_cola_2019_100927 crossref_primary_10_1109_MS_2021_3070552 crossref_primary_10_1016_j_jss_2020_110825 crossref_primary_10_1109_TSE_2017_2755013 crossref_primary_10_1109_TSE_2017_2755651 crossref_primary_10_1109_TSE_2018_2874648 crossref_primary_10_1007_s10664_021_09989_x crossref_primary_10_1109_TSE_2017_2775634 crossref_primary_10_1002_spe_2672 crossref_primary_10_1016_j_jss_2017_06_039 crossref_primary_10_1007_s10664_023_10291_1 crossref_primary_10_1016_j_infsof_2022_106865 crossref_primary_10_1109_TSE_2021_3124323 crossref_primary_10_1016_j_jss_2019_06_076 crossref_primary_10_1109_TSE_2020_2970009 crossref_primary_10_1186_s40537_022_00654_8 crossref_primary_10_1145_3729352 crossref_primary_10_1007_s10664_020_09843_6 crossref_primary_10_1145_3561382 crossref_primary_10_1007_s10664_017_9584_3 crossref_primary_10_1016_j_infsof_2022_107043 crossref_primary_10_1145_3296979_3192387 crossref_primary_10_1109_TSE_2023_3305052 crossref_primary_10_1007_s10664_021_10083_5 crossref_primary_10_1109_TSE_2022_3177713 crossref_primary_10_1145_3716167
Cites_doi	10.1145/1830483.1830654 10.1145/2568225.2568324 10.1145/2568225.2568258 10.1109/ICST.2010.66 10.1007/s10664-005-3861-2 10.1145/2666356.2594297 10.1007/978-3-642-39742-4_1 10.1145/1368088.1368222 10.1109/ICSM.2013.29 10.1145/1993498.1993544 10.1109/ICSE.2013.6606623 10.1145/1629575.1629585 10.1109/MSP.2005.144 10.1109/ASE.2009.15 10.1145/1806799.1806833 10.1109/ICSE.2003.1201189 10.1007/978-3-642-23716-4_6 10.1145/2330163.2330333 10.1145/2330163.2330296 10.1145/2610384.2610404 10.1109/ICSE.2013.6606624 10.1007/s11219-013-9208-0 10.1109/ICSE.2013.6606625 10.1145/1454115.1454128 10.1109/ICSE.2012.6227184 10.1145/2338965.2336775 10.1145/2635868.2635929 10.1145/2568225.2568303 10.1007/s10710-013-9195-8 10.1109/TSE.2014.2312918 10.1145/1167473.1167488 10.1109/ICSE.1994.296778 10.1109/ICPC.2009.5090029 10.1145/2338967.2336812 10.1109/TEVC.2010.2052622 10.1145/1134285.1134336 10.1109/CEC.2008.4630793 10.1145/2483760.2483785 10.1145/1370175.1370223 10.1145/2786805.2786825 10.1109/TSE.2010.62 10.1109/ASE.2013.6693094 10.1109/ICSE.2012.6227211 10.1109/FOSE.2007.29 10.1145/1882291.1882327 10.1109/ICSM.2010.5609691 10.1145/1065010.1065014 10.1109/TSE.2011.104 10.1109/ICSE.2013.6606626 10.1145/1831708.1831716 10.1145/1146238.1146266 10.1145/1321631.1321702 10.1109/ICSE.2009.5070536 10.1145/2001420.2001445
ContentType	Journal Article
Copyright	Copyright IEEE Computer Society Dec 1, 2015
Copyright_xml	– notice: Copyright IEEE Computer Society Dec 1, 2015
DBID	97E RIA RIE AAYXX CITATION JQ2 K9. 7SC 7SP 8FD F28 FR3 L7M L~C L~D
DOI	10.1109/TSE.2015.2454513
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef ProQuest Computer Science Collection ProQuest Health & Medical Complete (Alumni) Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ANTE: Abstracts in New Technology & Engineering Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef ProQuest Health & Medical Complete (Alumni) ProQuest Computer Science Collection Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace ANTE: Abstracts in New Technology & Engineering Computer and Information Systems Abstracts Professional
DatabaseTitleList	ProQuest Health & Medical Complete (Alumni) Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	1939-3520
EndPage	1256
ExternalDocumentID	3936541631 10_1109_TSE_2015_2454513 7153570
Genre	orig-research Feature
GrantInformation_xml	– fundername: US National Science Foundation (NSF) grantid: CCF-0729097; CCF-0905236; CCF-1446683; CNS-0905222) funderid: 10.13039/100000001 – fundername: US Department of Energy (DOE) grantid: DE-AC02-05CH11231 funderid: 10.13039/100000015 – fundername: AFOSR grantid: FA9550-07-1-0532; FA9550-10-1-0277 funderid: 10.13039/100000181 – fundername: US Defense Advanced Research Projects Agency (DARPA) grantid: P-1070-113237 funderid: 10.13039/100000185
GroupedDBID	--Z -DZ -~X .4S .DC 0R~ 29I 4.4 5GY 6IK 85S 8R4 8R5 97E AAJGR AARMG AASAJ AAWTH ABAZT ABPPZ ABQJQ ABVLG ACGFO ACGOD ACIWK ACNCT AENEX AGQYO AGSQL AHBIQ AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BKOMP BPEOZ CS3 DU5 EBS EDO EJD HZ~ I-F IEDLZ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P Q2X RIA RIE RNS RXW S10 TAE TN5 TWZ UHB UPT WH7 YZZ AAYXX CITATION JQ2 K9. 7SC 7SP 8FD F28 FR3 L7M L~C L~D
ID	FETCH-LOGICAL-c390t-a8df1db8c1d8c85d45524443616a66c88c7ab4e0d3927da51d0e41183cee46e43
IEDL.DBID	RIE
ISICitedReferencesCount	216
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000366470700006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0098-5589
IngestDate	Sat Sep 27 23:57:30 EDT 2025 Fri Oct 03 04:01:16 EDT 2025 Tue Nov 18 20:58:10 EST 2025 Sat Nov 29 03:10:23 EST 2025 Wed Aug 27 02:47:43 EDT 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	12
Keywords	IntroClass Automated program repair ManyBugs reproducibility benchmark subject defect
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c390t-a8df1db8c1d8c85d45524443616a66c88c7ab4e0d3927da51d0e41183cee46e43
Notes	SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-1 ObjectType-Feature-2 content type line 23
PQID	1761112263
PQPubID	21418
PageCount	21
ParticipantIDs	proquest_miscellaneous_1786210228 proquest_journals_1761112263 crossref_citationtrail_10_1109_TSE_2015_2454513 ieee_primary_7153570 crossref_primary_10_1109_TSE_2015_2454513
PublicationCentury	2000
PublicationDate	2015-Dec.-1 2015-12-1 20151201
PublicationDateYYYYMMDD	2015-12-01
PublicationDate_xml	– month: 12 year: 2015 text: 2015-Dec.-1 day: 01
PublicationDecade	2010
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on software engineering
PublicationTitleAbbrev	TSE
PublicationYear	2015
Publisher	IEEE IEEE Computer Society
Publisher_xml	– name: IEEE – name: IEEE Computer Society
References	ref13 ref12 ref15 (ref5) 2013 ref14 ref52 ref55 ref11 ref10 britton (ref7) 2013 carbin (ref68) 0 (ref29) 2014 ref17 ref16 egly (ref19) 0 (ref56) 2014 world (ref31) 2014 ref51 ref50 (ref33) 2014 ref45 ref48 lu (ref73) 0 ref42 ref41 bradbury (ref59) 2010 ref44 ref43 (ref28) 2014 gopinath (ref67) 0 brun (ref24) 2013 ref49 (ref30) 2014 ref8 (ref2) 0 ref9 ref4 ref3 ref6 knight (ref76) 0 (ref57) 2014 ref40 ref80 ref35 ref78 ref34 ref37 ref36 ref75 ref74 cadar (ref58) 0 ref77 durieux (ref47) 2015 ref39 ref38 (ref53) 0 järvisalo (ref18) 0 ref71 ref70 (ref1) 2013; 409 ref72 ref23 ref26 ref69 ref25 ref64 ref20 ref63 ref66 ref22 ref65 ref21 bird (ref54) 0 ref27 ref60 he (ref79) 0 (ref32) 2014 ref62 ref61 (ref46) 0
References_xml	– ident: ref55 doi: 10.1145/1830483.1830654 – year: 2013 ident: ref24 – ident: ref36 doi: 10.1145/2568225.2568324 – ident: ref38 doi: 10.1145/2568225.2568258 – ident: ref10 doi: 10.1109/ICST.2010.66 – ident: ref44 doi: 10.1007/s10664-005-3861-2 – year: 0 ident: ref53 – ident: ref72 doi: 10.1145/2666356.2594297 – year: 2013 ident: ref5 article-title: SPEC open systems group policies and procedures document – ident: ref41 doi: 10.1007/978-3-642-39742-4_1 – ident: ref69 doi: 10.1145/1368088.1368222 – year: 2014 ident: ref32 – ident: ref22 doi: 10.1109/ICSM.2013.29 – ident: ref11 doi: 10.1145/1993498.1993544 – ident: ref14 doi: 10.1109/ICSE.2013.6606623 – ident: ref15 doi: 10.1145/1629575.1629585 – ident: ref70 doi: 10.1109/MSP.2005.144 – year: 2013 ident: ref7 article-title: Reversible debugging software – year: 0 ident: ref18 publication-title: Proc Int Conf Theory and Appl Satisfiability Testing – ident: ref34 doi: 10.1109/ASE.2009.15 – ident: ref71 doi: 10.1145/1806799.1806833 – start-page: 267 year: 0 ident: ref79 article-title: Automated debugging using path-based weakest preconditions publication-title: Proc 8th Int Conf Fundam Approaches Softw Eng – ident: ref6 doi: 10.1109/ICSE.2003.1201189 – ident: ref51 doi: 10.1007/978-3-642-23716-4_6 – ident: ref64 doi: 10.1145/2330163.2330333 – start-page: 609 year: 0 ident: ref68 article-title: Detecting and escaping infinite loops with Jolt publication-title: Proc 11 European Conf on Object-Oriented Programming – ident: ref50 doi: 10.1145/2330163.2330296 – start-page: 121 year: 0 ident: ref54 article-title: Fair and balanced? Bias in bug-fix datasets publication-title: Proc 2nd ACM SIGSOFT Symp on Foundations of Software Engineering – year: 2014 ident: ref56 – ident: ref49 doi: 10.1145/2610384.2610404 – ident: ref8 doi: 10.1109/ICSE.2013.6606624 – year: 2014 ident: ref31 – ident: ref20 doi: 10.1007/s11219-013-9208-0 – ident: ref9 doi: 10.1109/ICSE.2013.6606625 – year: 2014 ident: ref30 – ident: ref3 doi: 10.1145/1454115.1454128 – year: 0 ident: ref19 publication-title: Proc Int Conf Theory and Appl Satisfiability Testing – start-page: 209 year: 0 ident: ref58 article-title: KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs publication-title: Proc 8th USENIX Conf Operating Syst Des Implementation – ident: ref62 doi: 10.1109/ICSE.2012.6227184 – ident: ref40 doi: 10.1145/2338965.2336775 – ident: ref43 doi: 10.1145/2635868.2635929 – ident: ref37 doi: 10.1145/2568225.2568303 – start-page: 1 year: 2010 ident: ref59 article-title: Automatic repair of concurrency bugs publication-title: Proc Int Symp Search Based Softw Eng -Fast Abstracts – ident: ref74 doi: 10.1007/s10710-013-9195-8 – year: 2014 ident: ref57 article-title: Python (programming language) – ident: ref61 doi: 10.1109/TSE.2014.2312918 – ident: ref4 doi: 10.1145/1167473.1167488 – year: 2014 ident: ref33 – ident: ref45 doi: 10.1109/ICSE.1994.296778 – ident: ref63 doi: 10.1109/ICPC.2009.5090029 – ident: ref48 doi: 10.1145/2338967.2336812 – start-page: 16 year: 0 ident: ref73 article-title: Bugbench: Benchmarks for evaluating bug detection tools publication-title: Workshop on the Evaluation of Software Defect Detection Tools – ident: ref66 doi: 10.1109/TEVC.2010.2052622 – ident: ref26 doi: 10.1145/1134285.1134336 – ident: ref60 doi: 10.1109/CEC.2008.4630793 – volume: 409 year: 2013 ident: ref1 publication-title: Economist – year: 0 ident: ref2 – year: 2014 ident: ref28 – ident: ref52 doi: 10.1145/2483760.2483785 – ident: ref80 doi: 10.1145/1370175.1370223 – ident: ref25 doi: 10.1145/2786805.2786825 – ident: ref77 doi: 10.1109/TSE.2010.62 – start-page: 337 year: 0 ident: ref76 article-title: An experimental evaluation of simple methods for seeding program errors publication-title: Proc 8th Int Conf on Software Eng – year: 2014 ident: ref29 – ident: ref21 doi: 10.1109/ASE.2013.6693094 – ident: ref23 doi: 10.1109/ICSE.2012.6227211 – ident: ref39 doi: 10.1109/FOSE.2007.29 – ident: ref35 doi: 10.1145/1882291.1882327 – ident: ref75 doi: 10.1109/ICSM.2010.5609691 – year: 2015 ident: ref47 publication-title: CoRR – start-page: 173 year: 0 ident: ref67 article-title: Specification-based program repair using SAT publication-title: Proc Int Conf Tools Algorithms Construction Anal Syst – ident: ref27 doi: 10.1145/1065010.1065014 – ident: ref13 doi: 10.1109/TSE.2011.104 – ident: ref12 doi: 10.1109/ICSE.2013.6606626 – ident: ref16 doi: 10.1145/1831708.1831716 – year: 0 ident: ref46 article-title: Samate-Software software assurance metrics and tool evaluation – ident: ref65 doi: 10.1145/1146238.1146266 – ident: ref42 doi: 10.1145/1321631.1321702 – ident: ref17 doi: 10.1109/ICSE.2009.5070536 – ident: ref78 doi: 10.1145/2001420.2001445
SSID	ssj0005775 ssib053395008
Score	2.610706
Snippet	The field of automated software repair lacks a set of common benchmark problems. Although benchmark sets are used widely throughout computer science, existing...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1236
SubjectTerms	<sc xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">IntroClass <sc xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">ManyBugs Algorithms Automated program repair Automation benchmark Benchmark testing Benchmarks C (programming language) C language Categories Computer bugs Computer programs Computer science Datasets Debugging Defects Electronic mail Maintenance engineering Repair Reproducibility Software Software systems Studies subject defect
Title	The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs
URI	https://ieeexplore.ieee.org/document/7153570 https://www.proquest.com/docview/1761112263 https://www.proquest.com/docview/1786210228
Volume	41
WOSCitedRecordID	wos000366470700006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1939-3520 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: RIE dateStart: 19750101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEA6rePDiW1xfRPAiWLexefWoi-KCLoIPvJVsJlVRW9lu_f1O-lgRRfBW6KQtM5nM9zWTGUL2IYZIWiYDkA4CX1ApiFMEctIpHTEZAqsKz99fquFQPzzE1x1yOD0L45yrks_ckb-s9vIht6X_VdZT6J5CIUGfUUrWZ7W-0jmUEm19TCF03G5JhnHv9ubM53CJo2OOeIFF30JQ1VPlx0JcRZfzxf991xJZaFAkPanNvkw6Llshi22HBto47CoZ4CygV-jvp-VjQU0GdOAz06tOmPQUZZ7ezPiloIhc6Uk5yRG-OqAIys3zmOYp7dPrOn-rWCN352e3_YugaZ4Q2CgOJ4HRkDIYactAWy2AC4GRnEeSSSOl1doqM-IuBARICoxgEDqObCPCqMml49E6mc3yzG0QmgLKIhDSRksukdOmI98IgVsbgQpT3SW9Vp-JbSqL-wYXr0nFMMI4QQsk3gJJY4EuOZiOeK-ravwhu-o1PpVrlN0l263JksbtioQpiWs3IkoctTe9jQ7jd0FM5vLSyyCJ8zxXb_7-5C0y799f56xsk9nJuHQ7ZM5-TJ6L8W416z4B-lbSXQ
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dS9xAEB_ECvpStVZ6rdUt9EVovKzZrzyqKB6eh-C1-Bb2djattE3K5dK_v7P5uCKK4Fsgs2HZ2dn5Tea3MwCfMcVEOa4iVB6jUFApSnMCcsprk3AVI28Kz38b68nE3N2lNyvwZXkXxnvfkM_8UXhscvlYujr8KhtqMk-pKUB_JYU4jtvbWv8JHVrLvkKmlCbtk5JxOpzengcWlzw6FoQYePLACTVdVR4dxY1_udh82cy24HWHI9lJq_htWPHFG9jsezSwzmR3YET7gF2TxZ_W3ytmC2SjwE1vemGyU5L58dvOf1aMsCs7qRclAViPjGC5vZ-zMmdn7KZlcFVv4evF-fTsMuraJ0QuSeNFZA3mHGfGcTTOSBRSki8XieLKKuWMcdrOhI-RIJJGKznGXlC8kZDfFMqLZBdWi7Lw74DlSLIEhYw1SiiKavNZaIUgnEtQx7kZwLBfz8x1tcVDi4tfWRNjxGlGGsiCBrJOAwM4XI7409bVeEZ2J6z4Uq5b7AHs9SrLOsOrMq4Vnd6EKWnUp-VrMpmQB7GFL-sgQ2FciHTN-6e_fADrl9PrcTYeTa4-wEaYS8tg2YPVxbz2H2HN_V3cV_P9Zgf-AxWQ1aQ
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+ManyBugs+and+IntroClass+Benchmarks+for+Automated+Repair+of+C+Programs&rft.jtitle=IEEE+transactions+on+software+engineering&rft.au=Le+Goues%2C+Claire&rft.au=Holtschulte%2C+Neal&rft.au=Smith%2C+Edward+K.&rft.au=Brun%2C+Yuriy&rft.date=2015-12-01&rft.issn=0098-5589&rft.eissn=1939-3520&rft.volume=41&rft.issue=12&rft.spage=1236&rft.epage=1256&rft_id=info:doi/10.1109%2FTSE.2015.2454513&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TSE_2015_2454513
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0098-5589&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0098-5589&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0098-5589&client=summon