LTM: Scalable and Black-Box Similarity-Based Test Suite Minimization Based on Language Models

Test suites tend to grow when software evolves, making it often infeasible to execute all test cases with the allocated testing budgets, especially for large software systems. Test suite minimization (TSM) is employed to improve the efficiency of software testing by removing redundant test cases, th...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on software engineering Ročník 50; číslo 11; s. 3053 - 3070
Hlavní autoři:	Pan, Rongqi, Ghaleb, Taher A., Briand, Lionel C.
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York IEEE 01.11.2024 IEEE Computer Society
Témata:	Black boxes black-box testing Closed box Codes Euclidean geometry Fault detection genetic algorithm Genetic algorithms Large language models Minimization Optimization pre-trained language models Scalability Similarity Similarity measures Software testing Source coding Test suite minimization test suite reduction Testing time Time measurement Unified modeling language Vectors
ISSN:	0098-5589, 1939-3520
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	Test suites tend to grow when software evolves, making it often infeasible to execute all test cases with the allocated testing budgets, especially for large software systems. Test suite minimization (TSM) is employed to improve the efficiency of software testing by removing redundant test cases, thus reducing testing time and resources while maintaining the fault detection capability of the test suite. Most existing TSM approaches rely on code coverage (white-box) or model-based features, which are not always available to test engineers. Recent TSM approaches that rely only on test code (black-box) have been proposed, such as ATM and FAST-R. The former yields higher fault detection rates ( FDR ) while the latter is faster. To address scalability while retaining a high FDR , we propose LTM ( L anguage model-based T est suite M inimization), a novel, scalable, and black-box similarity-based TSM approach based on large language models (LLMs), which is the first application of LLMs in the context of TSM. To support similarity measurement using test method embeddings, we investigate five different pre-trained language models: CodeBERT, GraphCodeBERT, UniXcoder, StarEncoder, and CodeLlama, on which we compute two similarity measures: Cosine Similarity and Euclidean Distance. Our goal is to find similarity measures that are not only computationally more efficient but can also better guide a Genetic Algorithm (GA), which is used to search for optimal minimized test suites, thus reducing the overall search time. Experimental results show that the best configuration of LTM (UniXcoder/Cosine) outperforms ATM in three aspects: (a) achieving a slightly greater saving rate of testing time (<inline-formula><tex-math notation="LaTeX">41.72\%</tex-math> <mml:math display="inline"><mml:mn>41.72</mml:mn><mml:mi mathvariant="normal">%</mml:mi></mml:math><inline-graphic xlink:href="pan-ieq1-3469582.gif"/> </inline-formula> versus <inline-formula><tex-math notation="LaTeX">41.02\%</tex-math> <mml:math display="inline"><mml:mn>41.02</mml:mn><mml:mi mathvariant="normal">%</mml:mi></mml:math><inline-graphic xlink:href="pan-ieq2-3469582.gif"/> </inline-formula>, on average); (b) attaining a significantly higher fault detection rate (<inline-formula><tex-math notation="LaTeX">0.84</tex-math> <mml:math display="inline"><mml:mn>0.84</mml:mn></mml:math><inline-graphic xlink:href="pan-ieq3-3469582.gif"/> </inline-formula> versus <inline-formula><tex-math notation="LaTeX">0.81</tex-math> <mml:math display="inline"><mml:mn>0.81</mml:mn></mml:math><inline-graphic xlink:href="pan-ieq4-3469582.gif"/> </inline-formula>, on average); and, most importantly, (c) minimizing test suites nearly five times faster on average, with higher gains for larger test suites and systems, thus achieving much higher scalability.
AbstractList	Test suites tend to grow when software evolves, making it often infeasible to execute all test cases with the allocated testing budgets, especially for large software systems. Test suite minimization (TSM) is employed to improve the efficiency of software testing by removing redundant test cases, thus reducing testing time and resources while maintaining the fault detection capability of the test suite. Most existing TSM approaches rely on code coverage (white-box) or model-based features, which are not always available to test engineers. Recent TSM approaches that rely only on test code (black-box) have been proposed, such as ATM and FAST-R. The former yields higher fault detection rates ( FDR ) while the latter is faster. To address scalability while retaining a high FDR , we propose LTM ( L anguage model-based T est suite M inimization), a novel, scalable, and black-box similarity-based TSM approach based on large language models (LLMs), which is the first application of LLMs in the context of TSM. To support similarity measurement using test method embeddings, we investigate five different pre-trained language models: CodeBERT, GraphCodeBERT, UniXcoder, StarEncoder, and CodeLlama, on which we compute two similarity measures: Cosine Similarity and Euclidean Distance. Our goal is to find similarity measures that are not only computationally more efficient but can also better guide a Genetic Algorithm (GA), which is used to search for optimal minimized test suites, thus reducing the overall search time. Experimental results show that the best configuration of LTM (UniXcoder/Cosine) outperforms ATM in three aspects: (a) achieving a slightly greater saving rate of testing time ([Formula Omitted] versus [Formula Omitted], on average); (b) attaining a significantly higher fault detection rate ([Formula Omitted] versus [Formula Omitted], on average); and, most importantly, (c) minimizing test suites nearly five times faster on average, with higher gains for larger test suites and systems, thus achieving much higher scalability. Test suites tend to grow when software evolves, making it often infeasible to execute all test cases with the allocated testing budgets, especially for large software systems. Test suite minimization (TSM) is employed to improve the efficiency of software testing by removing redundant test cases, thus reducing testing time and resources while maintaining the fault detection capability of the test suite. Most existing TSM approaches rely on code coverage (white-box) or model-based features, which are not always available to test engineers. Recent TSM approaches that rely only on test code (black-box) have been proposed, such as ATM and FAST-R. The former yields higher fault detection rates ( FDR ) while the latter is faster. To address scalability while retaining a high FDR , we propose LTM ( L anguage model-based T est suite M inimization), a novel, scalable, and black-box similarity-based TSM approach based on large language models (LLMs), which is the first application of LLMs in the context of TSM. To support similarity measurement using test method embeddings, we investigate five different pre-trained language models: CodeBERT, GraphCodeBERT, UniXcoder, StarEncoder, and CodeLlama, on which we compute two similarity measures: Cosine Similarity and Euclidean Distance. Our goal is to find similarity measures that are not only computationally more efficient but can also better guide a Genetic Algorithm (GA), which is used to search for optimal minimized test suites, thus reducing the overall search time. Experimental results show that the best configuration of LTM (UniXcoder/Cosine) outperforms ATM in three aspects: (a) achieving a slightly greater saving rate of testing time (<inline-formula><tex-math notation="LaTeX">41.72\%</tex-math> <mml:math display="inline"><mml:mn>41.72</mml:mn><mml:mi mathvariant="normal">%</mml:mi></mml:math><inline-graphic xlink:href="pan-ieq1-3469582.gif"/> </inline-formula> versus <inline-formula><tex-math notation="LaTeX">41.02\%</tex-math> <mml:math display="inline"><mml:mn>41.02</mml:mn><mml:mi mathvariant="normal">%</mml:mi></mml:math><inline-graphic xlink:href="pan-ieq2-3469582.gif"/> </inline-formula>, on average); (b) attaining a significantly higher fault detection rate (<inline-formula><tex-math notation="LaTeX">0.84</tex-math> <mml:math display="inline"><mml:mn>0.84</mml:mn></mml:math><inline-graphic xlink:href="pan-ieq3-3469582.gif"/> </inline-formula> versus <inline-formula><tex-math notation="LaTeX">0.81</tex-math> <mml:math display="inline"><mml:mn>0.81</mml:mn></mml:math><inline-graphic xlink:href="pan-ieq4-3469582.gif"/> </inline-formula>, on average); and, most importantly, (c) minimizing test suites nearly five times faster on average, with higher gains for larger test suites and systems, thus achieving much higher scalability.
Author	Ghaleb, Taher A. Pan, Rongqi Briand, Lionel C.
Author_xml	– sequence: 1 givenname: Rongqi orcidid: 0000-0002-9096-6241 surname: Pan fullname: Pan, Rongqi email: rpan099@uottawa.ca organization: School of EECS, University of Ottawa, Ottawa, ON, Canada – sequence: 2 givenname: Taher A. orcidid: 0000-0001-9336-7298 surname: Ghaleb fullname: Ghaleb, Taher A. email: taherghaleb@trentu.ca organization: Computer Science Department, Trent University, Peterborough, ON, Canada – sequence: 3 givenname: Lionel C. orcidid: 0000-0002-1393-1010 surname: Briand fullname: Briand, Lionel C. email: lbriand@uottawa.ca organization: Lero SFI Centre for Software Research, University of Limerick, Limerick, Ireland
BookMark	eNp9UD1PwzAUtFCRKIWdgcESc4o_msRmo1X5kFIxJIzI8lcqlzQpcSJRfj0u6YAYmN6T7u7duzsHo7qpLQBXGE0xRvy2yJdTgshsSmcJjxk5AWPMKY9oTNAIjBHiLIpjxs_AufcbhFCcpvEYvGXF6g7mWlZSVRbK2sB5JfV7NG8-Ye62rpKt6_bRXHprYGF9B_PedRauXB3QL9m5poYDGpZM1utergPcGFv5C3Baysrby-OcgNeHZbF4irKXx-fFfRZpwkkXWWXKxDKtCMJSKs1MaS03yBiUkBjPOCqNUoThUmnMFJYJZYTSlDNOqSYlnYCb4e6ubT768KTYNH1bB0tBMUlTHPKmgYUGlm4b71tbil3rtrLdC4zEoUQRShSHEsWxxCBJ_ki0634yd6101X_C60HorLW_fBKecoroN9xPgI8
CODEN	IESEDJ
CitedBy_id	crossref_primary_10_1145_3721128
Cites_doi	10.18653/v1/2020.findings-emnlp.139 10.1145/3644388 10.1109/ICSE48619.2023.00146 10.1145/3194718.3194731 10.18653/v1/2021.emnlp-main.552 10.1109/TPWRD.2019.2962275 10.1109/TSE.2022.3170272 10.1109/ICSE.2019.00054 10.1016/j.jss.2019.03.011 10.1145/3540250.3549089 10.18653/v1/d18-2029 10.1109/ACCESS.2018.2809600 10.1007/978-3-642-00234-2_1 10.1109/TSE.2022.3201209 10.1109/ICSE.2019.00055 10.1090/conm/026/737400 10.1111/j.1558-5646.1995.tb04456.x 10.1109/iThings/CPSCom.2011.135 10.1145/2430536.2430540 10.48550/arXiv.1310.4546 10.1109/4235.996017 10.1109/ACCESS.2020.2990567 10.18653/v1/p16-1162 10.18653/v1/D19-1410 10.1007/978-3-030-35510-4_4 10.1613/jair.2934 10.1007/978-3-642-16573-3_6 10.1002/stv.430 10.1108/ws.2000.07949fab.004 10.1016/j.jss.2016.06.058 10.18653/v1/2022.acl-long.499 10.48550/ARXIV.1706.03762 10.1016/j.infsof.2019.06.009 10.1007/s10664-021-10066-6 10.1109/SANER.2018.8330200
ContentType	Journal Article
Copyright	Copyright IEEE Computer Society 2024
Copyright_xml	– notice: Copyright IEEE Computer Society 2024
DBID	97E ESBDL RIA RIE AAYXX CITATION JQ2 K9.
DOI	10.1109/TSE.2024.3469582
DatabaseName	IEEE Xplore (IEEE) IEEE Xplore Open Access Journals IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef ProQuest Computer Science Collection ProQuest Health & Medical Complete (Alumni)
DatabaseTitle	CrossRef ProQuest Health & Medical Complete (Alumni) ProQuest Computer Science Collection
DatabaseTitleList	ProQuest Health & Medical Complete (Alumni)
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	1939-3520
EndPage	3070
ExternalDocumentID	10_1109_TSE_2024_3469582 10697930
Genre	orig-research
GrantInformation_xml	– fundername: Science Foundation Ireland grantid: 13/RC/2094-2 funderid: 10.13039/501100001602 – fundername: Mitacs Accelerate Program – fundername: Huawei Technologies Canada Company, Ltd. – fundername: Natural Sciences and Engineering Research Council of Canada (NSERC) funderid: 10.13039/501100000038 – fundername: Digital Research Alliance of Canada
GroupedDBID	--Z -DZ -~X .4S .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 7WY 7X7 85S 88E 88I 8FE 8FG 8FI 8FJ 8FL 8G5 8R4 8R5 97E 9M8 AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABJCF ABPPZ ABQJQ ABUWG ABVLG ACGFO ACGOD ACIWK ACNCT ADBBV AENEX AETIX AFKRA AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ARAPS ARCSS ASUFR ATWAV AZQEC BEFXN BENPR BEZIV BFFAM BGLVJ BGNUA BKEBE BKOMP BPEOZ BPHCQ BVXVI CCPQU CS3 DU5 DWQXO E.L EBS EDO EJD ESBDL FRNLG FYUFA GNUQQ GROUPED_ABI_INFORM_RESEARCH GUQSH HCIFZ HMCUK HZ~ H~9 I-F IBMZZ ICLAB IEDLZ IFIPE IFJZH IPLJI ITG ITH JAVBF K60 K6V K6~ K7- L6V LAI M0C M1P M1Q M2O M2P M43 M7S MS~ O9- OCL OHT P2P P62 PHGZM PHGZT PJZUB PPXIY PQBIZ PQBZA PQGLB PQQKQ PROAC PSQYO PTHSS PUEGO Q2X RIA RIE RNI RNS RXW RZB S10 TAE TN5 TWZ UHB UKHRP UPT UQL VH1 WH7 XOL YYP YZZ ZCG AAYXX AFFHD CITATION JQ2 K9.
ID	FETCH-LOGICAL-c292t-ebdf6e8cb201aabc8dfee9d0dd06251490fdbb281fbc18b1a638233798933c2f3
IEDL.DBID	RIE
ISICitedReferencesCount	3
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001369099900008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0098-5589
IngestDate	Fri Oct 03 04:01:47 EDT 2025 Sat Nov 29 03:10:28 EST 2025 Tue Nov 18 22:08:49 EST 2025 Wed Aug 27 03:06:46 EDT 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	11
Language	English
License	https://creativecommons.org/licenses/by-nc-nd/4.0
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c292t-ebdf6e8cb201aabc8dfee9d0dd06251490fdbb281fbc18b1a638233798933c2f3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0001-9336-7298 0000-0002-9096-6241 0000-0002-1393-1010
OpenAccessLink	https://ieeexplore.ieee.org/document/10697930
PQID	3127710007
PQPubID	21418
PageCount	18
ParticipantIDs	proquest_journals_3127710007 ieee_primary_10697930 crossref_citationtrail_10_1109_TSE_2024_3469582 crossref_primary_10_1109_TSE_2024_3469582
PublicationCentury	2000
PublicationDate	2024-11-01
PublicationDateYYYYMMDD	2024-11-01
PublicationDate_xml	– month: 11 year: 2024 text: 2024-11-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on software engineering
PublicationTitleAbbrev	TSE
PublicationYear	2024
Publisher	IEEE IEEE Computer Society
Publisher_xml	– name: IEEE – name: IEEE Computer Society
References	ref13 ref12 ref15 ref10 ref17 ref16 ref19 ref18 Kusner (ref27) 2015 Kocetkov (ref37) 2022 ref50 Lu (ref34) 2021 ref46 Roziè (ref14) 2023 ref44 Guo (ref11) 2020 ref43 ref49 ref8 ref9 ref4 ref3 ref6 ref5 ref40 ref35 ref31 ref30 ref32 ref2 Vijaymeena (ref33) 2016; 3 ref1 ref38 Joachims (ref26) 1997; 97 Clark (ref39) 2020 Devlin (ref23) 2018 ref24 ref25 ref20 ref22 ref21 Husain (ref36) 2019 Fan (ref48) 2023 ref28 Herzig (ref7) 2018 ref29 Pan (ref45) 2023 Ma (ref41) 2019 Gomaa (ref42) 2013; 68 Xia (ref47) 2015; 126
References_xml	– ident: ref10 doi: 10.18653/v1/2020.findings-emnlp.139 – ident: ref16 doi: 10.1145/3644388 – volume: 126 start-page: 5614 issue: 24 volume-title: Optik year: 2015 ident: ref47 article-title: Effectiveness of the euclidean distance in high dimensional spaces – ident: ref8 doi: 10.1109/ICSE48619.2023.00146 – start-page: 957 volume-title: Proc. Int. Conf. Mach. Learn. year: 2015 ident: ref27 article-title: From word embeddings to document distances – start-page: 38 volume-title: Proc. 11th Int. Workshop Search-Based Softw. Testing year: 2018 ident: ref7 article-title: Testing and continuous integration at scale: Limits, costs, and expectations doi: 10.1145/3194718.3194731 – volume: 68 start-page: 13 issue: 13 year: 2013 ident: ref42 article-title: A survey of text similarity approaches publication-title: Int. J. Comput. Appl. – ident: ref40 doi: 10.18653/v1/2021.emnlp-main.552 – ident: ref49 doi: 10.1109/TPWRD.2019.2962275 – ident: ref21 doi: 10.1109/TSE.2022.3170272 – ident: ref28 doi: 10.1109/ICSE.2019.00054 – ident: ref20 doi: 10.1016/j.jss.2019.03.011 – year: 2020 ident: ref11 article-title: GraphCodeBERT: Pre-training code representations with data flow – volume: 97 start-page: 143 volume-title: Proc. Int. Conf. Mach. Learning year: 1997 ident: ref26 article-title: A probabilistic analysis of the Rocchio algorithm with TFIDF for text categorization – year: 2018 ident: ref23 article-title: BERT: Pre-training of deep bidirectional transformers for language understanding – ident: ref30 doi: 10.1145/3540250.3549089 – year: 2023 ident: ref45 article-title: LTM: Scalable and black-box similarity-based test suite minimization based on language models (replication package) – ident: ref25 doi: 10.18653/v1/d18-2029 – ident: ref1 doi: 10.1109/ACCESS.2018.2809600 – volume: 3 start-page: 19 issue: 2 year: 2016 ident: ref33 article-title: A survey on similarity measures in text mining publication-title: Mach. Learn. Appl. Int. J. – ident: ref43 doi: 10.1007/978-3-642-00234-2_1 – ident: ref50 doi: 10.1109/TSE.2022.3201209 – ident: ref6 doi: 10.1109/ICSE.2019.00055 – ident: ref32 doi: 10.1090/conm/026/737400 – ident: ref46 doi: 10.1111/j.1558-5646.1995.tb04456.x – ident: ref19 doi: 10.1109/iThings/CPSCom.2011.135 – ident: ref2 doi: 10.1145/2430536.2430540 – ident: ref22 doi: 10.48550/arXiv.1310.4546 – ident: ref29 doi: 10.1109/4235.996017 – ident: ref44 doi: 10.1109/ACCESS.2020.2990567 – year: 2022 ident: ref37 article-title: The stack: 3 TB of permissively licensed source code – ident: ref35 doi: 10.18653/v1/p16-1162 – ident: ref24 doi: 10.18653/v1/D19-1410 – ident: ref9 doi: 10.1007/978-3-030-35510-4_4 – year: 2019 ident: ref36 article-title: CodeSearchNet challenge: Evaluating the state of semantic code search – year: 2020 ident: ref39 article-title: ELECTRA: Pre-training text encoders as discriminators rather than generators – ident: ref31 doi: 10.1613/jair.2934 – year: 2019 ident: ref41 article-title: Universal text representation from BERT: An empirical study – ident: ref15 doi: 10.1007/978-3-642-16573-3_6 – ident: ref3 doi: 10.1002/stv.430 – ident: ref13 doi: 10.1108/ws.2000.07949fab.004 – ident: ref17 doi: 10.1016/j.jss.2016.06.058 – ident: ref12 doi: 10.18653/v1/2022.acl-long.499 – ident: ref38 doi: 10.48550/ARXIV.1706.03762 – ident: ref5 doi: 10.1016/j.infsof.2019.06.009 – ident: ref4 doi: 10.1007/s10664-021-10066-6 – year: 2021 ident: ref34 article-title: CodeXGLUE: A machine learning benchmark dataset for code understanding and generation – volume-title: Survey and open problems year: 2023 ident: ref48 article-title: Large language models for software engineering: – ident: ref18 doi: 10.1109/SANER.2018.8330200 – year: 2023 ident: ref14 article-title: Code Llama: Open foundation models for code
SSID	ssj0005775 ssib053395008
Score	2.4768522
Snippet	Test suites tend to grow when software evolves, making it often infeasible to execute all test cases with the allocated testing budgets, especially for large...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	3053
SubjectTerms	Black boxes black-box testing Closed box Codes Euclidean geometry Fault detection genetic algorithm Genetic algorithms Large language models Minimization Optimization pre-trained language models Scalability Similarity Similarity measures Software testing Source coding Test suite minimization test suite reduction Testing time Time measurement Unified modeling language Vectors
Title	LTM: Scalable and Black-Box Similarity-Based Test Suite Minimization Based on Language Models
URI	https://ieeexplore.ieee.org/document/10697930 https://www.proquest.com/docview/3127710007
Volume	50
WOSCitedRecordID	wos001369099900008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1939-3520 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: RIE dateStart: 19750101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aPHixPipWq-TgxcO2-07izUrFQ1uErtCLLHlCoW6lD_HnO8nu-kAUvGU3yRL2SzLzTTIzCF0aFSe-UtwLpVEeaOCwD7Io8IwQqSEJh-0hdskmyHhMp1P2UDmrO18YrbW7fKa7tujO8tVCbqypDFZ4ymA-AUPfJiQtnbU-73MQktQBMpOEsvpM0me9bDIAJhjG3QjIYELDbzLIJVX5sRM78XLX_OfA9tFepUfimxL4A7Sli0PUrHM04GrJHqGnYTa6hkc-tz5SmBcKO5ud11-84cnseQbMFhRxrw_STOEMRoYnG1BD8WhWQG3ppInLWigMK_MmtjnU5qsWerwbZLf3XpVSwZMhC9eeFsqkmkoBcp9zIakyWjMFaPlAhIAt-UYJEdLACBlQEfDUnhNGhAGAkQxNdIwaxaLQJwgTHsuIcHgrdUxkKji14cIibqimgHEb9eqfnMsq3rhNezHPHe_wWQ6w5BaWvIKlja4-eryUsTb-aNuyMHxpVyLQRp0ayLxajas8CkJioxj55PSXbmdo1369dDLsoMZ6udHnaEe-rmer5YWbaO9aZNDj
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JSwMxFA6igl7cxWrVHLx4GJ09iTcVpWJbhI7gRYasUKhTsa34833JZFwQBW-ZScKE-ZK8JXnvQ-jIqDQLleJBLI0KQAOHfZAlUWCEyA3JOGwPqSObIP0-fXhgdz5Y3cXCaK3d5TN9YovuLF-N5cy6ymCF5wzmE1joC5Y6y4drfd7oICRrUmRmGWXNqWTITovBFdiCcXqSgDmY0fibFHK0Kj_2Yidgrlf_ObQ1tOI1SXxeQ7-O5nS1gVYblgbsF-0meuwWvTN45CMbJYV5pbDz2gUX4zc8GD4NwbYFVTy4AHmmcAEjw4MZKKK4N6ygtg7TxHUtFLrewYkti9posoXur6-Ky07gSRUCGbN4GmihTK6pFCD5OReSKqM1U4BXCKYQ2EuhUULENDJCRlREPLcnhQlhAGEiY5Nso_lqXOkdhAlPZUI4vJU6JTIXnNqEYQk3VFNAuYVOm59cSp9x3BJfjEpneYSsBFhKC0vpYWmh448ez3W2jT_ablkYvrSrEWihdgNk6dfjpEyimNg8RiHZ_aXbIVrqFL1u2b3p3-6hZfulOuSwjeanLzO9jxbl63Q4eTlwk-4dZrfULA
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=LTM%3A+Scalable+and+Black-Box+Similarity-Based+Test+Suite+Minimization+Based+on+Language+Models&rft.jtitle=IEEE+transactions+on+software+engineering&rft.au=Pan%2C+Rongqi&rft.au=Ghaleb%2C+Taher+A.&rft.au=Briand%2C+Lionel+C.&rft.date=2024-11-01&rft.issn=0098-5589&rft.eissn=1939-3520&rft.volume=50&rft.issue=11&rft.spage=3053&rft.epage=3070&rft_id=info:doi/10.1109%2FTSE.2024.3469582&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TSE_2024_3469582
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0098-5589&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0098-5589&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0098-5589&client=summon