NLP-assisted software testing: A systematic mapping of the literature

To reduce manual effort of extracting test cases from natural-language requirements, many approaches based on Natural Language Processing (NLP) have been proposed in the literature. Given the large amount of approaches in this area, and since many practitioners are eager to utilize such techniques,...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	Information and software technology Ročník 126; s. 106321
Hlavní autori:	Garousi, Vahid, Bauer, Sara, Felderer, Michael
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Elsevier B.V 01.10.2020
Predmet:	Body of knowledge Industrial collaborators Large amounts Mapping NAtural language processing Natural Language Processing (NLP) Natural language processing systems Natural language requirements Review papers Software testing State of the art Systematic literature mapping Systematic literature review Systematic mapping Software testing Systematic literature mapping Natural Language Processing (NLP) Systematic literature review
ISSN:	0950-5849, 1873-6025, 1873-6025
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	To reduce manual effort of extracting test cases from natural-language requirements, many approaches based on Natural Language Processing (NLP) have been proposed in the literature. Given the large amount of approaches in this area, and since many practitioners are eager to utilize such techniques, it is important to synthesize and provide an overview of the state-of-the-art in this area. Our objective is to summarize the state-of-the-art in NLP-assisted software testing which could benefit practitioners to potentially utilize those NLP-based techniques. Moreover, this can benefit researchers in providing an overview of the research landscape. To address the above need, we conducted a survey in the form of a systematic literature mapping (classification). After compiling an initial pool of 95 papers, we conducted a systematic voting, and our final pool included 67 technical papers. This review paper provides an overview of the contribution types presented in the papers, types of NLP approaches used to assist software testing, types of required input requirements, and a review of tool support in this area. Some key results we have detected are: (1) only four of the 38 tools (11%) presented in the papers are available for download; (2) a larger ratio of the papers (30 of 67) provided a shallow exposure to the NLP aspects (almost no details). This paper would benefit both practitioners and researchers by serving as an “index” to the body of knowledge in this area. The results could help practitioners utilizing the existing NLP-based techniques; this in turn reduces the cost of test-case design and decreases the amount of human resources spent on test activities. After sharing this review with some of our industrial collaborators, initial insights show that this review can indeed be useful and beneficial to practitioners.
AbstractList	To reduce manual effort of extracting test cases from natural-language requirements, many approaches based on Natural Language Processing (NLP) have been proposed in the literature. Given the large amount of approaches in this area, and since many practitioners are eager to utilize such techniques, it is important to synthesize and provide an overview of the state-of-the-art in this area. Our objective is to summarize the state-of-the-art in NLP-assisted software testing which could benefit practitioners to potentially utilize those NLP-based techniques. Moreover, this can benefit researchers in providing an overview of the research landscape. To address the above need, we conducted a survey in the form of a systematic literature mapping (classification). After compiling an initial pool of 95 papers, we conducted a systematic voting, and our final pool included 67 technical papers. This review paper provides an overview of the contribution types presented in the papers, types of NLP approaches used to assist software testing, types of required input requirements, and a review of tool support in this area. Some key results we have detected are: (1) only four of the 38 tools (11%) presented in the papers are available for download; (2) a larger ratio of the papers (30 of 67) provided a shallow exposure to the NLP aspects (almost no details). This paper would benefit both practitioners and researchers by serving as an “index” to the body of knowledge in this area. The results could help practitioners utilizing the existing NLP-based techniques; this in turn reduces the cost of test-case design and decreases the amount of human resources spent on test activities. After sharing this review with some of our industrial collaborators, initial insights show that this review can indeed be useful and beneficial to practitioners. Context: To reduce manual effort of extracting test cases from natural-language requirements, many approaches based on Natural Language Processing (NLP) have been proposed in the literature. Given the large amount of approaches in this area, and since many practitioners are eager to utilize such techniques, it is important to synthesize and provide an overview of the state-of-the-art in this area. Objective: Our objective is to summarize the state-of-the-art in NLP-assisted software testing which could benefit practitioners to potentially utilize those NLP-based techniques. Moreover, this can benefit researchers in providing an overview of the research landscape. Method: To address the above need, we conducted a survey in the form of a systematic literature mapping (classification). After compiling an initial pool of 95 papers, we conducted a systematic voting, and our final pool included 67 technical papers. Results: This review paper provides an overview of the contribution types presented in the papers, types of NLP approaches used to assist software testing, types of required input requirements, and a review of tool support in this area. Some key results we have detected are: (1) only four of the 38 tools (11%) presented in the papers are available for download; (2) a larger ratio of the papers (30 of 67) provided a shallow exposure to the NLP aspects (almost no details). Conclusion: This paper would benefit both practitioners and researchers by serving as an “index” to the body of knowledge in this area. The results could help practitioners utilizing the existing NLP-based techniques; this in turn reduces the cost of test-case design and decreases the amount of human resources spent on test activities. After sharing this review with some of our industrial collaborators, initial insights show that this review can indeed be useful and beneficial to practitioners. © 2020 Elsevier B.V.
ArticleNumber	106321
Author	Bauer, Sara Felderer, Michael Garousi, Vahid
Author_xml	– sequence: 1 givenname: Vahid orcidid: 0000-0001-6590-7576 surname: Garousi fullname: Garousi, Vahid email: v.garousi@qub.ac.uk organization: Queen's University Belfast, Northern Ireland, United Kingdom – sequence: 2 givenname: Sara surname: Bauer fullname: Bauer, Sara email: sara.bauer@uibk.ac.at organization: University of Innsbruck, Austria – sequence: 3 givenname: Michael surname: Felderer fullname: Felderer, Michael email: michael.felderer@uibk.ac.at organization: University of Innsbruck, Austria
BackLink	https://urn.kb.se/resolve?urn=urn:nbn:se:bth-19524$$DView record from Swedish Publication Index (Blekinge Tekniska Högskola)
BookMark	eNqFkLtOwzAYhS1UJErhDRj8AKTYcezEHZCqUi5SBQzAajm-tK7aOLJdqr49qYIYGGD6pfOf7wzfORg0vjEAXGE0xgizm_XYNTZ6O85RfowYyfEJGOKqJBlDOR2AIeIUZbQq-Bk4j3GNEC4RQUMwf168ZjJGF5PRsNtIexkMTCYm1ywncArjoXttZXIKbmXbdin0FqaVgRuXTJBpF8wFOLVyE83l9x2B9_v52-wxW7w8PM2mi0yRiqcsZ8ToUiuFNau4wlYSqWlBdMUpY7asNa5QRZXluC7ygnEmMdZYa0pKKwtJRuC634170-5q0Qa3leEgvHTizn1MhQ9LUaeVwJzmRVcv-roKPsZg7A-AkTiaE2vRmxNHc6I312GTX5hyqRPgmxSk2_wH3_aw6Tx8OhNEVM40ymgXjEpCe_f3wBe8Oo7r
CitedBy_id	crossref_primary_10_1080_09537287_2024_2436464 crossref_primary_10_1016_j_infsof_2025_107805 crossref_primary_10_1016_j_micpro_2023_104967 crossref_primary_10_1109_ACCESS_2021_3070606 crossref_primary_10_2196_29871 crossref_primary_10_1080_23311916_2023_2276495 crossref_primary_10_1007_s00766_022_00390_8 crossref_primary_10_1515_auto_2021_0138 crossref_primary_10_1016_j_jksuci_2021_09_018 crossref_primary_10_1007_s10836_023_06086_3 crossref_primary_10_1109_ACCESS_2023_3346767 crossref_primary_10_1016_j_engappai_2024_108378 crossref_primary_10_1145_3731756 crossref_primary_10_1007_s10664_023_10333_8 crossref_primary_10_1016_j_procs_2024_06_186 crossref_primary_10_1007_s40572_023_00418_3 crossref_primary_10_1145_3573074_3573101 crossref_primary_10_1016_j_jksuci_2022_01_013 crossref_primary_10_1002_smr_2606 crossref_primary_10_1145_3444689 crossref_primary_10_3390_electronics12092109 crossref_primary_10_1109_ACCESS_2024_3407753 crossref_primary_10_1145_3617946_3617957 crossref_primary_10_1007_s11219_025_09722_7 crossref_primary_10_1016_j_nlp_2025_100128 crossref_primary_10_1145_3616372 crossref_primary_10_23939_acps2025_01_095 crossref_primary_10_1016_j_jss_2023_111901 crossref_primary_10_1016_j_jss_2022_111549 crossref_primary_10_3390_app122110773
Cites_doi	10.1016/j.infsof.2015.03.007 10.1007/s11192-017-2419-6 10.1002/jrsm.1106 10.1192/bja.2017.3 10.1145/2601248.2601268 10.1016/j.infsof.2013.03.004 10.1038/nj7584-117a 10.1016/j.joi.2006.06.001 10.1109/MS.2017.101 10.1109/TSE.2010.62 10.1016/j.infsof.2013.02.006 10.1016/j.infsof.2016.04.015 10.1007/978-3-031-01855-8 10.1016/j.jss.2011.03.051 10.1186/s13643-015-0125-0 10.1186/s13643-018-0724-7 10.1186/s13643-016-0337-y 10.1145/1831407.1831415 10.1016/j.infsof.2018.12.003 10.1016/j.jss.2012.12.051 10.1038/467753a 10.1109/MS.2017.34 10.1016/j.jss.2014.09.042 10.1371/journal.pone.0138237
ContentType	Journal Article
Copyright	2020 Elsevier B.V.
Copyright_xml	– notice: 2020 Elsevier B.V.
DBID	AAYXX CITATION ADTPV AOWAS DF3
DOI	10.1016/j.infsof.2020.106321
DatabaseName	CrossRef SwePub SwePub Articles SWEPUB Blekinge Tekniska Högskola
DatabaseTitle	CrossRef
DatabaseTitleList
DeliveryMethod	fulltext_linktorsrc
Discipline	Business
EISSN	1873-6025
ExternalDocumentID	oai_DiVA_org_bth_19524 10_1016_j_infsof_2020_106321 S0950584920300744
GroupedDBID	--K --M -~X .DC .~1 0R~ 1B1 1~. 1~5 29I 4.4 457 4G. 5GY 5VS 7-5 71M 77K 8P~ 9JN AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN AAYOK ABBOA ABFNM ABFRF ABJNI ABMAC ABTAH ABXDB ABYKQ ACDAQ ACGFO ACGFS ACGOD ACNNM ACRLP ACZNC ADBBV ADEZE ADJOM ADMUD AEBSH AEFWE AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BKOJK BKOMP BLXMC CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q G8K GBLVA GBOLZ HLZ HVGLF HZ~ IHE J1W KOM LG9 M41 MO0 MS~ N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. PQQKQ Q38 R2- RIG ROL RPZ SBC SDF SDG SDP SES SEW SPC SPCBC SSV SSZ T5K TWZ UHS UNMZH WH7 WUQ XFK ZY4 ~G- 77I 9DU AATTM AAXKI AAYWO AAYXX ABDPE ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD ADTPV AOWAS DF3
ID	FETCH-LOGICAL-c389t-263ed7dcc1d689c1fa3ad543d89566f7bd18085cf91b424696a11d1dd537fa4a3
ISICitedReferencesCount	48
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000573271800010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0950-5849 1873-6025
IngestDate	Tue Nov 04 15:52:15 EST 2025 Sat Nov 29 07:06:14 EST 2025 Tue Nov 18 20:52:13 EST 2025 Fri Feb 23 02:47:56 EST 2024
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Keywords	Software testing Systematic literature mapping Natural Language Processing (NLP) Systematic literature review
Language	English
LinkModel	OpenURL
MergedId	FETCHMERGED-LOGICAL-c389t-263ed7dcc1d689c1fa3ad543d89566f7bd18085cf91b424696a11d1dd537fa4a3
ORCID	0000-0001-6590-7576
OpenAccessLink	http://www.scopus.com/inward/record.url?scp=85084949953&partnerID=8YFLogxK
ParticipantIDs	swepub_primary_oai_DiVA_org_bth_19524 crossref_primary_10_1016_j_infsof_2020_106321 crossref_citationtrail_10_1016_j_infsof_2020_106321 elsevier_sciencedirect_doi_10_1016_j_infsof_2020_106321
PublicationCentury	2000
PublicationDate	2020-10-01
PublicationDateYYYYMMDD	2020-10-01
PublicationDate_xml	– month: 10 year: 2020 text: 2020-10-01 day: 01
PublicationDecade	2020
PublicationTitle	Information and software technology
PublicationYear	2020
Publisher	Elsevier B.V
Publisher_xml	– name: Elsevier B.V
References	Lee (bib0009) 2008 Escalona, Gutierrez, Mejías, Aragón, Ramos, Torres, Domínguez (bib0006) 2011; 84 Liddy (bib0008) 2001 Bird, Loper (bib0030) 2004 De Marneffe, MacCartney, Manning (bib0055) 2006; 6 Haddaway, Collins, Coughlin, Kirk (bib0031) 2015; 10 Wohlin, Runeson, Höst, Ohlsson, Regnell, Wesslén (bib0056) 2000 Zhi, Garousi, Sun, Garousi, Shahnewaz, Ruhe (bib0034) 2015; 99 Miles, Huberman, Saldana (bib0048) 2014 McCafferty (bib0052) 2010; 53 Manning, Surdeanu, Bauer, Finkel, Bethard, Mcclosky (bib0013) 2014 Garousi, Felderer, Kılıçaslan (bib0046) 2019; 108 Leidner (bib0012) 2003 Mathew, Agrawal, Menzies (bib0043) 2018 Garousi, Felderer, Karapıçak, Yılmaz (bib0021) 2018 Atkinson, Cipriani (bib0033) 2018; 24 Tsafnat, Glasziou, Karystianis, Coiera (bib0032) 2018; 7 Gewin (bib0026) 2016; 529 Garousi, Mäntylä (bib0040) 2016; 76 Shahbaz, McMinn, Stevenson (bib0016) 2012 Petersen, Vakkalanka, Kuzniarz (bib0022) 2015; 64 Gutiérrez, Escalona, Mejías, Ramos, Gómez (bib0042) 2008 Graham, Fewster (bib0014) 2012 Chen, Xie, Maslov, Redner (bib0036) 2007; 1 Cook (bib0050) 1989 Koehn (bib0010) 2009 Zhang, Yue, Ali, Zhang, Wu (bib0017) 2014 Denger, Mora (bib0018) 2003 Petersen, Feldt, Mujtaba, Mattsson (bib0024) 2008 Kitchenham, Charters (bib0025) 2007 Berti-Equille, Borge-Holthoefer (bib0011) 2015 Jia, Harman (bib0047) 2011; 37 Ammann, Offutt (bib0003) 2008 Garousi, Zhi (bib0001) 2013; 86 V.Garousi, S.Bauer, and M.Felderer, "Dataset for the study: nLP-assisted software testing: a survey," Barnes (bib0053) 2010; 467 De Marneffe, MacCartney, Manning (bib0029) 2006; 6 Garousi, Felderer (bib0049) 2017 M.Baker, "1,500 scientists lift the lid on reproducibility," Obrenovic (bib0005) 2017; 34 Ahsan, Butt, Ahmed, Anwar (bib0007) 2017 Garousi, Elberzhager (bib0015) 2017; 34 Last accessed: Mar. 28,2018. Goffi, Gorla, Ernst, Pezzè (bib0004) 2016 C.Wohlin, "Guidelines for snowballing in systematic literature studies and a replication in software engineering," presented at the Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, London, England, United Kingdom, 2014. Banerjee, Nguyen, Garousi, Memon (bib0045) 2013; 55 Adams, Hillier-Brown, Moore, Lake, Araujo-Soares, White, Summerbell (bib0039) 2016; 5 Garousi, Mesbah, Betin-Can, Mirshokraie (bib0044) 2013; 55 Gutiérrez Rodríguez, Escalona Cuaresma, Mejías Risoto, Torres Valderrama (bib0019) 2006 EnagoAcademy, "The Importance of Scientific Transparency and Reproducibility," Nazir, Butt, Anwar, Khan Khattak (bib0020) 2017 Godin, Stapleton, Kirkpatrick, Hanning, Leatherdale (bib0037) 2015; 4 Mahood, Van Eerd, Irvin (bib0038) 2014; 5 (bib0051) 2018 Sinha, Mihalcea (bib0054) 2007 Indurkhya, Damerau (bib0002) 2010 Last accessed: Mar. 28, 2018. Garousi, Fernandes (bib0035) 2017; 112 10.1016/j.infsof.2020.106321_bib0027 10.1016/j.infsof.2020.106321_bib0028 Koehn (10.1016/j.infsof.2020.106321_bib0010) 2009 Atkinson (10.1016/j.infsof.2020.106321_bib0033) 2018; 24 De Marneffe (10.1016/j.infsof.2020.106321_bib0029) 2006; 6 Graham (10.1016/j.infsof.2020.106321_bib0014) 2012 Manning (10.1016/j.infsof.2020.106321_bib0013) 2014 Garousi (10.1016/j.infsof.2020.106321_bib0044) 2013; 55 Zhi (10.1016/j.infsof.2020.106321_bib0034) 2015; 99 (10.1016/j.infsof.2020.106321_bib0051) 2018 10.1016/j.infsof.2020.106321_bib0023 Banerjee (10.1016/j.infsof.2020.106321_bib0045) 2013; 55 Adams (10.1016/j.infsof.2020.106321_bib0039) 2016; 5 Godin (10.1016/j.infsof.2020.106321_bib0037) 2015; 4 Garousi (10.1016/j.infsof.2020.106321_bib0015) 2017; 34 Petersen (10.1016/j.infsof.2020.106321_bib0024) 2008 Garousi (10.1016/j.infsof.2020.106321_bib0001) 2013; 86 Lee (10.1016/j.infsof.2020.106321_bib0009) 2008 Ammann (10.1016/j.infsof.2020.106321_bib0003) 2008 Leidner (10.1016/j.infsof.2020.106321_bib0012) 2003 Escalona (10.1016/j.infsof.2020.106321_bib0006) 2011; 84 Zhang (10.1016/j.infsof.2020.106321_bib0017) 2014 Petersen (10.1016/j.infsof.2020.106321_bib0022) 2015; 64 Liddy (10.1016/j.infsof.2020.106321_bib0008) 2001 Sinha (10.1016/j.infsof.2020.106321_bib0054) 2007 Wohlin (10.1016/j.infsof.2020.106321_bib0056) 2000 De Marneffe (10.1016/j.infsof.2020.106321_bib0055) 2006; 6 Gewin (10.1016/j.infsof.2020.106321_bib0026) 2016; 529 Goffi (10.1016/j.infsof.2020.106321_bib0004) 2016 10.1016/j.infsof.2020.106321_bib0041 Gutiérrez (10.1016/j.infsof.2020.106321_bib0042) 2008 Garousi (10.1016/j.infsof.2020.106321_bib0021) 2018 Indurkhya (10.1016/j.infsof.2020.106321_bib0002) 2010 Gutiérrez Rodríguez (10.1016/j.infsof.2020.106321_bib0019) 2006 Bird (10.1016/j.infsof.2020.106321_bib0030) 2004 Miles (10.1016/j.infsof.2020.106321_bib0048) 2014 Shahbaz (10.1016/j.infsof.2020.106321_bib0016) 2012 Tsafnat (10.1016/j.infsof.2020.106321_bib0032) 2018; 7 Garousi (10.1016/j.infsof.2020.106321_bib0046) 2019; 108 Ahsan (10.1016/j.infsof.2020.106321_bib0007) 2017 Mahood (10.1016/j.infsof.2020.106321_bib0038) 2014; 5 Berti-Equille (10.1016/j.infsof.2020.106321_bib0011) 2015 Haddaway (10.1016/j.infsof.2020.106321_bib0031) 2015; 10 Garousi (10.1016/j.infsof.2020.106321_bib0049) 2017 Barnes (10.1016/j.infsof.2020.106321_bib0053) 2010; 467 Denger (10.1016/j.infsof.2020.106321_bib0018) 2003 Nazir (10.1016/j.infsof.2020.106321_bib0020) 2017 Mathew (10.1016/j.infsof.2020.106321_bib0043) 2018 Chen (10.1016/j.infsof.2020.106321_bib0036) 2007; 1 Kitchenham (10.1016/j.infsof.2020.106321_bib0025) 2007 Jia (10.1016/j.infsof.2020.106321_bib0047) 2011; 37 Cook (10.1016/j.infsof.2020.106321_bib0050) 1989 Garousi (10.1016/j.infsof.2020.106321_bib0040) 2016; 76 McCafferty (10.1016/j.infsof.2020.106321_bib0052) 2010; 53 Garousi (10.1016/j.infsof.2020.106321_bib0035) 2017; 112 Obrenovic (10.1016/j.infsof.2020.106321_bib0005) 2017; 34
References_xml	– year: 2003 ident: bib0018 article-title: Test case derived from requirement specification publication-title: Fraunhofer IESE Tech. Rep., – reference: V.Garousi, S.Bauer, and M.Felderer, "Dataset for the study: nLP-assisted software testing: a survey," – volume: 467 start-page: 753 year: 2010 ident: bib0053 article-title: Publish your computer code: it is good enough publication-title: Nature News – volume: 55 start-page: 1679 year: 2013 end-page: 1694 ident: bib0045 article-title: Graphical User Interface (GUI) Testing: systematic Mapping and Repository publication-title: Inf. Softw. Technol. – volume: 108 start-page: 35 year: 2019 end-page: 64 ident: bib0046 article-title: A survey on software testability publication-title: Inf. Softw. Technol. – volume: 37 start-page: 649 year: 2011 end-page: 678 ident: bib0047 article-title: An Analysis and Survey of the Development of Mutation Testing publication-title: IEEE Trans. Softw. Eng. – volume: 76 start-page: 92 year: 2016 end-page: 117 ident: bib0040 article-title: When and what to automate in software testing? A multivocal literature review publication-title: Inf. Softw. Technol. – year: 2015 ident: bib0011 article-title: Veracity of Data: from Truth Discovery Computation Algorithms to Models of Misinformation Dynamics publication-title: Synthesis Lect. Data Manage. – reference: M.Baker, "1,500 scientists lift the lid on reproducibility," – volume: 10 year: 2015 ident: bib0031 article-title: The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching publication-title: PLoS ONE – year: 1989 ident: bib0050 article-title: Case GrammarTheory – volume: 99 start-page: 175 year: 2015 end-page: 198 ident: bib0034 article-title: Cost, benefits and quality of software development documentation: a systematic mapping publication-title: J. Syst. Softw. – year: 2001 ident: bib0008 article-title: Natural language processing publication-title: Encyclopedia of Library and Information Science – volume: 5 start-page: 221 year: 2014 end-page: 234 ident: bib0038 article-title: Searching for grey literature for systematic reviews: challenges and benefits publication-title: Res. Synth. Methods – volume: 7 start-page: 64 year: 2018 ident: bib0032 article-title: Automated screening of research studies for systematic reviews using study characteristics publication-title: Syst. Rev. – year: 2009 ident: bib0010 article-title: Statistical Machine Translation – start-page: 79 year: 2012 end-page: 88 ident: bib0016 article-title: Automated discovery of valid test strings from the web using dynamic regular expressions collation and natural language processing publication-title: International Conference on Quality Software – year: 2008 ident: bib0042 article-title: Automatic application of the category-partition method with loops support over functional requirements publication-title: Proceedings of the Euro American Conference on Telematics and Information Systems – start-page: 363 year: 2007 end-page: 369 ident: bib0054 article-title: Unsupervised graph-basedword sense disambiguation using measures of word semantic similarity publication-title: International Conference on Semantic Computing – reference: EnagoAcademy, "The Importance of Scientific Transparency and Reproducibility," – reference: C.Wohlin, "Guidelines for snowballing in systematic literature studies and a replication in software engineering," presented at the Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, London, England, United Kingdom, 2014. – volume: 64 start-page: 1 year: 2015 end-page: 18 ident: bib0022 article-title: Guidelines for conducting systematic mapping studies in software engineering: an update publication-title: Inf. Softw. Technol. – volume: 34 start-page: 90 year: 2017 end-page: 96 ident: bib0015 article-title: Test automation: not just for test execution publication-title: IEEE Softw. – volume: 6 start-page: 449 year: 2006 end-page: 454 ident: bib0055 article-title: Generating typed dependency parses from phrase structure parses publication-title: Proceedings of The International Conference on Language Resources and Evaluation – reference: , Last accessed: Mar. 28,2018. – year: 2018 ident: bib0043 article-title: Finding Trends in Software Research publication-title: IEEE Trans. Softw. Eng. – year: 2008 ident: bib0024 article-title: Systematic mapping studies in software engineering publication-title: presented at the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE) – volume: 53 start-page: 16 year: 2010 end-page: 17 ident: bib0052 article-title: Should code be released? publication-title: Commun. ACM – volume: 112 start-page: 963 year: 2017 end-page: 1006 ident: bib0035 article-title: Quantity versus impact of software engineering papers: a quantitative study publication-title: Scientometrics J. Article – year: 2014 ident: bib0013 article-title: The stanford core-NLP natural language processing toolkit publication-title: ACL System Demonstrations – volume: 529 start-page: 117 year: 2016 end-page: 119 ident: bib0026 article-title: Data sharing: an open mind on open data publication-title: Nature – volume: 4 start-page: 138 year: 2015 end-page: 148 ident: bib0037 article-title: Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada publication-title: Syst. Rev. – year: 2008 ident: bib0003 article-title: Introduction to Software Testing – start-page: 86 year: 2008 end-page: 99 ident: bib0009 article-title: Corpora and discourse analysis publication-title: Advances in discourse studies – volume: 6 start-page: 449 year: 2006 end-page: 454 ident: bib0029 article-title: Generating typed dependency parses from phrase structure parses publication-title: Proceedings of International Conference on Language Resources and Evaluation – volume: 84 start-page: 1379 year: 2011 end-page: 1393 ident: bib0006 article-title: An overview on test generation from functional requirements publication-title: J. Syst. Softw. – start-page: 142 year: 2014 end-page: 157 ident: bib0017 article-title: A systematic approach to automatically derive test cases from use cases specified in restricted natural languages publication-title: International Conference on System Analysis and Modeling – year: 2007 ident: bib0025 article-title: Guidelines for Performing Systematic Literature Reviews in Software engineering publication-title: EBSE Technical Report – year: 2012 ident: bib0014 article-title: Experiences of Test Automation: Case Studies of Software Test Automation – year: 2014 ident: bib0048 article-title: Qualitative Data Analysis: A Methods Sourcebook – year: 2006 ident: bib0019 article-title: Generation of test cases from functional requirements. A survey publication-title: Workshop on System Testing and Validation – volume: 5 start-page: 164 year: 2016 ident: bib0039 article-title: Searching and synthesising ‘grey literature’ and ‘grey information’ in public health: critical reflections on three case studies publication-title: Syst. Rev. – volume: 55 start-page: 1374 year: 2013 end-page: 1396 ident: bib0044 article-title: A systematic mapping study of web application testing publication-title: Elsevier J. Inform. Softw. Tech. – volume: 24 start-page: 74 year: 2018 end-page: 82 ident: bib0033 article-title: How to carry out a literature search for a systematic review: a practical guide publication-title: BJPsych Adv. – start-page: 45 year: 2003 end-page: 50 ident: bib0012 article-title: Current issues in software engineering for natural language processing publication-title: Proceedings of workshop on Software engineering and architecture of language technology systems – year: 2018 ident: bib0051 article-title: Why are papers without code but with results accepted? publication-title: Various online users – year: 2000 ident: bib0056 article-title: Experimentation in Software Engineering: An Introduction – volume: 1 start-page: 8 year: 2007 end-page: 15 ident: bib0036 article-title: Finding scientific gems with Google’s PageRank algorithm publication-title: J. Informetr. – volume: 34 start-page: 71 year: 2017 end-page: 78 ident: bib0005 article-title: Insights from the Past: the IEEE Software History Experiment publication-title: IEEE Softw. – year: 2010 ident: bib0002 article-title: Handbook of NaturalLanguageProcessing – start-page: 213 year: 2016 end-page: 224 ident: bib0004 article-title: Automatic generation of oracles for exceptional behaviors publication-title: Proceedings of International Symposium on Software Testing and Analysis – start-page: 485 year: 2017 end-page: 493 ident: bib0020 article-title: The Applications of Natural Language Processing (NLP) for Software Requirement Engineering - A Systematic Literature Review publication-title: Information Science and Applications 2017 – start-page: 170 year: 2017 end-page: 179 ident: bib0049 article-title: Experience-based guidelines for effective and efficient data extraction in systematic reviews in software engineering publication-title: International Conference on Evaluation and Assessment in Software Engineering – year: 2018 ident: bib0021 article-title: Testing Embedded Software: A Survey of the Literature – volume: 86 start-page: 1354 year: 2013 end-page: 1376 ident: bib0001 article-title: A survey of software testing practices in Canada publication-title: J. Syst. Softw. – start-page: 31 year: 2004 ident: bib0030 article-title: NLTK: the natural language toolkit publication-title: Proceedings of the Annual Meeting of the Association for Computational Linguistics – reference: , Last accessed: Mar. 28, 2018. – start-page: 1 year: 2017 end-page: 10 ident: bib0007 article-title: A comprehensive investigation of natural language processing techniques and tools to generate automated test cases publication-title: Proceedings of the International Conference on Internet of things and Cloud Computing – start-page: 31 year: 2004 ident: 10.1016/j.infsof.2020.106321_bib0030 article-title: NLTK: the natural language toolkit – volume: 64 start-page: 1 issue: 0 year: 2015 ident: 10.1016/j.infsof.2020.106321_bib0022 article-title: Guidelines for conducting systematic mapping studies in software engineering: an update publication-title: Inf. Softw. Technol. doi: 10.1016/j.infsof.2015.03.007 – year: 2018 ident: 10.1016/j.infsof.2020.106321_bib0043 article-title: Finding Trends in Software Research publication-title: IEEE Trans. Softw. Eng. – volume: 112 start-page: 963 issue: 2 year: 2017 ident: 10.1016/j.infsof.2020.106321_bib0035 article-title: Quantity versus impact of software engineering papers: a quantitative study publication-title: Scientometrics J. Article doi: 10.1007/s11192-017-2419-6 – volume: 5 start-page: 221 issue: 3 year: 2014 ident: 10.1016/j.infsof.2020.106321_bib0038 article-title: Searching for grey literature for systematic reviews: challenges and benefits publication-title: Res. Synth. Methods doi: 10.1002/jrsm.1106 – volume: 24 start-page: 74 issue: 2 year: 2018 ident: 10.1016/j.infsof.2020.106321_bib0033 article-title: How to carry out a literature search for a systematic review: a practical guide publication-title: BJPsych Adv. doi: 10.1192/bja.2017.3 – year: 2008 ident: 10.1016/j.infsof.2020.106321_bib0003 – ident: 10.1016/j.infsof.2020.106321_bib0023 doi: 10.1145/2601248.2601268 – start-page: 485 year: 2017 ident: 10.1016/j.infsof.2020.106321_bib0020 article-title: The Applications of Natural Language Processing (NLP) for Software Requirement Engineering - A Systematic Literature Review – year: 2008 ident: 10.1016/j.infsof.2020.106321_bib0024 article-title: Systematic mapping studies in software engineering – ident: 10.1016/j.infsof.2020.106321_bib0028 – start-page: 1 year: 2017 ident: 10.1016/j.infsof.2020.106321_bib0007 article-title: A comprehensive investigation of natural language processing techniques and tools to generate automated test cases – start-page: 45 year: 2003 ident: 10.1016/j.infsof.2020.106321_bib0012 article-title: Current issues in software engineering for natural language processing – volume: 55 start-page: 1679 issue: 10 year: 2013 ident: 10.1016/j.infsof.2020.106321_bib0045 article-title: Graphical User Interface (GUI) Testing: systematic Mapping and Repository publication-title: Inf. Softw. Technol. doi: 10.1016/j.infsof.2013.03.004 – volume: 529 start-page: 117 issue: 7584 year: 2016 ident: 10.1016/j.infsof.2020.106321_bib0026 article-title: Data sharing: an open mind on open data publication-title: Nature doi: 10.1038/nj7584-117a – volume: 1 start-page: 8 issue: 1 year: 2007 ident: 10.1016/j.infsof.2020.106321_bib0036 article-title: Finding scientific gems with Google’s PageRank algorithm publication-title: J. Informetr. doi: 10.1016/j.joi.2006.06.001 – volume: 34 start-page: 71 issue: 4 year: 2017 ident: 10.1016/j.infsof.2020.106321_bib0005 article-title: Insights from the Past: the IEEE Software History Experiment publication-title: IEEE Softw. doi: 10.1109/MS.2017.101 – volume: 37 start-page: 649 issue: 5 year: 2011 ident: 10.1016/j.infsof.2020.106321_bib0047 article-title: An Analysis and Survey of the Development of Mutation Testing publication-title: IEEE Trans. Softw. Eng. doi: 10.1109/TSE.2010.62 – volume: 55 start-page: 1374 issue: 8 year: 2013 ident: 10.1016/j.infsof.2020.106321_bib0044 article-title: A systematic mapping study of web application testing publication-title: Elsevier J. Inform. Softw. Tech. doi: 10.1016/j.infsof.2013.02.006 – volume: 76 start-page: 92 year: 2016 ident: 10.1016/j.infsof.2020.106321_bib0040 article-title: When and what to automate in software testing? A multivocal literature review publication-title: Inf. Softw. Technol. doi: 10.1016/j.infsof.2016.04.015 – year: 2015 ident: 10.1016/j.infsof.2020.106321_bib0011 article-title: Veracity of Data: from Truth Discovery Computation Algorithms to Models of Misinformation Dynamics publication-title: Synthesis Lect. Data Manage. doi: 10.1007/978-3-031-01855-8 – year: 2001 ident: 10.1016/j.infsof.2020.106321_bib0008 article-title: Natural language processing – volume: 84 start-page: 1379 issue: 8 year: 2011 ident: 10.1016/j.infsof.2020.106321_bib0006 article-title: An overview on test generation from functional requirements publication-title: J. Syst. Softw. doi: 10.1016/j.jss.2011.03.051 – year: 2000 ident: 10.1016/j.infsof.2020.106321_bib0056 – year: 2009 ident: 10.1016/j.infsof.2020.106321_bib0010 – volume: 4 start-page: 138 year: 2015 ident: 10.1016/j.infsof.2020.106321_bib0037 article-title: Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada publication-title: Syst. Rev. doi: 10.1186/s13643-015-0125-0 – year: 2003 ident: 10.1016/j.infsof.2020.106321_bib0018 article-title: Test case derived from requirement specification publication-title: Fraunhofer IESE Tech. Rep., – year: 2010 ident: 10.1016/j.infsof.2020.106321_bib0002 – volume: 7 start-page: 64 issue: 1 year: 2018 ident: 10.1016/j.infsof.2020.106321_bib0032 article-title: Automated screening of research studies for systematic reviews using study characteristics publication-title: Syst. Rev. doi: 10.1186/s13643-018-0724-7 – volume: 5 start-page: 164 issue: 1 year: 2016 ident: 10.1016/j.infsof.2020.106321_bib0039 article-title: Searching and synthesising ‘grey literature’ and ‘grey information’ in public health: critical reflections on three case studies publication-title: Syst. Rev. doi: 10.1186/s13643-016-0337-y – year: 2018 ident: 10.1016/j.infsof.2020.106321_bib0021 – year: 2008 ident: 10.1016/j.infsof.2020.106321_bib0042 article-title: Automatic application of the category-partition method with loops support over functional requirements – volume: 53 start-page: 16 issue: 10 year: 2010 ident: 10.1016/j.infsof.2020.106321_bib0052 article-title: Should code be released? publication-title: Commun. ACM doi: 10.1145/1831407.1831415 – volume: 108 start-page: 35 year: 2019 ident: 10.1016/j.infsof.2020.106321_bib0046 article-title: A survey on software testability publication-title: Inf. Softw. Technol. doi: 10.1016/j.infsof.2018.12.003 – start-page: 170 year: 2017 ident: 10.1016/j.infsof.2020.106321_bib0049 article-title: Experience-based guidelines for effective and efficient data extraction in systematic reviews in software engineering – volume: 86 start-page: 1354 issue: 5 year: 2013 ident: 10.1016/j.infsof.2020.106321_bib0001 article-title: A survey of software testing practices in Canada publication-title: J. Syst. Softw. doi: 10.1016/j.jss.2012.12.051 – year: 2007 ident: 10.1016/j.infsof.2020.106321_bib0025 article-title: Guidelines for Performing Systematic Literature Reviews in Software engineering – volume: 467 start-page: 753 issue: 7317 year: 2010 ident: 10.1016/j.infsof.2020.106321_bib0053 article-title: Publish your computer code: it is good enough publication-title: Nature News doi: 10.1038/467753a – volume: 34 start-page: 90 issue: 2 year: 2017 ident: 10.1016/j.infsof.2020.106321_bib0015 article-title: Test automation: not just for test execution publication-title: IEEE Softw. doi: 10.1109/MS.2017.34 – volume: 99 start-page: 175 year: 2015 ident: 10.1016/j.infsof.2020.106321_bib0034 article-title: Cost, benefits and quality of software development documentation: a systematic mapping publication-title: J. Syst. Softw. doi: 10.1016/j.jss.2014.09.042 – ident: 10.1016/j.infsof.2020.106321_bib0041 – start-page: 86 year: 2008 ident: 10.1016/j.infsof.2020.106321_bib0009 article-title: Corpora and discourse analysis – start-page: 363 year: 2007 ident: 10.1016/j.infsof.2020.106321_bib0054 article-title: Unsupervised graph-basedword sense disambiguation using measures of word semantic similarity – year: 2006 ident: 10.1016/j.infsof.2020.106321_bib0019 article-title: Generation of test cases from functional requirements. A survey – year: 2014 ident: 10.1016/j.infsof.2020.106321_bib0013 article-title: The stanford core-NLP natural language processing toolkit – start-page: 142 year: 2014 ident: 10.1016/j.infsof.2020.106321_bib0017 article-title: A systematic approach to automatically derive test cases from use cases specified in restricted natural languages – year: 2014 ident: 10.1016/j.infsof.2020.106321_bib0048 – volume: 6 start-page: 449 year: 2006 ident: 10.1016/j.infsof.2020.106321_bib0029 article-title: Generating typed dependency parses from phrase structure parses – year: 2012 ident: 10.1016/j.infsof.2020.106321_bib0014 – start-page: 79 year: 2012 ident: 10.1016/j.infsof.2020.106321_bib0016 article-title: Automated discovery of valid test strings from the web using dynamic regular expressions collation and natural language processing – volume: 10 issue: 9 year: 2015 ident: 10.1016/j.infsof.2020.106321_bib0031 article-title: The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching publication-title: PLoS ONE doi: 10.1371/journal.pone.0138237 – start-page: 213 year: 2016 ident: 10.1016/j.infsof.2020.106321_bib0004 article-title: Automatic generation of oracles for exceptional behaviors – year: 1989 ident: 10.1016/j.infsof.2020.106321_bib0050 – ident: 10.1016/j.infsof.2020.106321_bib0027 – volume: 6 start-page: 449 year: 2006 ident: 10.1016/j.infsof.2020.106321_bib0055 article-title: Generating typed dependency parses from phrase structure parses – year: 2018 ident: 10.1016/j.infsof.2020.106321_bib0051 article-title: Why are papers without code but with results accepted?
SSID	ssj0017030
Score	2.5261838
SecondaryResourceType	review_article
Snippet	To reduce manual effort of extracting test cases from natural-language requirements, many approaches based on Natural Language Processing (NLP) have been... Context: To reduce manual effort of extracting test cases from natural-language requirements, many approaches based on Natural Language Processing (NLP) have...
SourceID	swepub crossref elsevier
SourceType	Open Access Repository Enrichment Source Index Database Publisher
StartPage	106321
SubjectTerms	Body of knowledge Industrial collaborators Large amounts Mapping NAtural language processing Natural Language Processing (NLP) Natural language processing systems Natural language requirements Review papers Software testing State of the art Systematic literature mapping Systematic literature review Systematic mapping
Title	NLP-assisted software testing: A systematic mapping of the literature
URI	https://dx.doi.org/10.1016/j.infsof.2020.106321 https://urn.kb.se/resolve?urn=urn:nbn:se:bth-19524
Volume	126
WOSCitedRecordID	wos000573271800010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-6025 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0017030 issn: 0950-5849 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LT9wwELbaBVW9oLYUlVe1h_aEXK3jxI_jCpY-hFYc6GpvVmI7wKrKomy28PMZx3bYFhD00EsUWc5rPmv8zWQeCH0aEKsFdxUANE9wmmqKC9iXMGMSNiNdZLbtDTg54eOxmE7laejNuWjbCfCqEjc38uq_Qg1jALZLnf0HuLubwgCcA-hwBNjh-CzgxyenGBixg8-5xMvm2gV3Na6aRtilZjF6vctcPPDu827u3-72r3k9Xy7aH_-T_CIGwTv_5zI07srrTr8fW9f524-vRuUH3wIYkjFK7c5JOMBAUeQf-tKnuN_Tvd4NMHMGA7zwF3c_GGTUZ0A_UNX66HIyVPP6XBXNhSIyS9KXaC3hmQTttDb8Ppr-6P4HOb3kqyb693FmtOAUM6BsMSGyjdq7__BHCcdqZdiWTZy9QRvBDOgPPXxv0QtbvUOvYhbCJtpfRbEfkekHFN-jn8ejs8NvOHSywBoIYYMTRq3hRmtimJCalDnNTZZSI8A8ZSUvDBHAfXUpSZEmKZMsJ8QQYzLKyzzN6RbqVfPKfkD9pOSDQcFLTbhJhU6ktpwUEmikENaIZBvR-LlKhzLvrtvILxXj-WbKC0k5ISkvpG2Eu6uufJmTJ-bzKEkVqJqnYApWwxNXfvaC757z8FLYeea8XfT6buXuoV5TL-0-Wte_m8tF_TGso1tKmGoL
linkProvider	Elsevier
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=NLP-assisted+software+testing&rft.jtitle=Information+and+software+technology&rft.au=Garousi%2C+Vahid&rft.au=Bauer%2C+Sara&rft.au=Felderer%2C+Michael&rft.date=2020-10-01&rft.issn=0950-5849&rft.volume=126&rft_id=info:doi/10.1016%2Fj.infsof.2020.106321&rft.externalDocID=oai_DiVA_org_bth_19524
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0950-5849&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0950-5849&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0950-5849&client=summon