Understandable Test Generation Through Capture/Replay and LLMs

Automatic unit test generators, particularly search-based software testing (SBST) tools such as EvoSuite, efficiently generate unit test suites with acceptable coverage. Although this removes the burden of writing unit tests from developers, these generated tests often pose challenges in terms of co...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE/ACM International Conference on Software Engineering Companion. Online) pp. 261 - 263
Main Author: Deljouyi, Amirhossein
Format: Conference Proceeding
Language:English
Published: ACM 14.04.2024
Subjects:
Automatic Test Generation
Carving and Replaying
Generators
Large Language Models
Medical services
Readability
Software engineering
Software testing
Statistical analysis
Test pattern generators
Understandability
Unit Testing
Writing
ISSN:2574-1934
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Automatic unit test generators, particularly search-based software testing (SBST) tools such as EvoSuite, efficiently generate unit test suites with acceptable coverage. Although this removes the burden of writing unit tests from developers, these generated tests often pose challenges in terms of comprehension for developers. In my doctoral research, I aim to investigate strategies to address the issue of comprehensibility in generated test cases and improve the test suite in terms of effectiveness. To achieve this, I introduce four projects leveraging Capture/Replay and Large Language Model (LLM) techniques. Capture/Replay carves information from End-to-End (E2E) tests, enabling the generation of unit tests containing meaningful test scenarios and actual test data. Moreover, the growing capabilities of large language models (LLMs) in language analysis and transformation play a significant role in improving readability in general. Our proposed approach involves leveraging E2E test scenario extraction alongside an LLM-guided approach to enhance test case understandability, augment coverage, and establish comprehensive mock and test oracles. In this research, we endeavor to conduct both a quantitative analysis and a user evaluation of the quality of the generated tests in terms of executability, coverage, and understandability.
AbstractList Automatic unit test generators, particularly search-based software testing (SBST) tools such as EvoSuite, efficiently generate unit test suites with acceptable coverage. Although this removes the burden of writing unit tests from developers, these generated tests often pose challenges in terms of comprehension for developers. In my doctoral research, I aim to investigate strategies to address the issue of comprehensibility in generated test cases and improve the test suite in terms of effectiveness. To achieve this, I introduce four projects leveraging Capture/Replay and Large Language Model (LLM) techniques. Capture/Replay carves information from End-to-End (E2E) tests, enabling the generation of unit tests containing meaningful test scenarios and actual test data. Moreover, the growing capabilities of large language models (LLMs) in language analysis and transformation play a significant role in improving readability in general. Our proposed approach involves leveraging E2E test scenario extraction alongside an LLM-guided approach to enhance test case understandability, augment coverage, and establish comprehensive mock and test oracles. In this research, we endeavor to conduct both a quantitative analysis and a user evaluation of the quality of the generated tests in terms of executability, coverage, and understandability.
Author Deljouyi, Amirhossein
Author_xml – sequence: 1
  givenname: Amirhossein
  surname: Deljouyi
  fullname: Deljouyi, Amirhossein
  email: a.deljouyi@tudelft.nl
  organization: Delft University of Technology,Delft,Netherlands
BookMark eNotj81Kw0AURkdRsNas3biYF0h75y-T2QgSbBUigqTrctPc2EichJl00bc3oquzOXyH75Zd-cETY_cCVkJos1aZctrmq1_a3F2wxFmXawALBqS4ZAtprE6FU_qGJTF-AYCSoLSzC_a48w2FOKFvsO6JVxQnviVPAadu8Lw6huH0eeQFjtMp0PqDxh7PfNZ5Wb7FO3bdYh8p-eeS7TbPVfGSlu_b1-KpTFEJO6WitXNQUeuMzWp3QC3BCgWyIRSIiowVtQCU1JI2kJF0BzdrdW2Ug0yqJXv42-2IaD-G7hvDeS_AmPkFqB8LBkn4
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
ESBDL
RIE
RIL
DOI 10.1145/3639478.3639789
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Xplore Open Access Journals
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9798400705021
EISSN 2574-1934
EndPage 263
ExternalDocumentID 10554970
Genre orig-research
GroupedDBID 6IE
6IF
6IL
6IN
AAJGR
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
ESBDL
IEGSK
OCL
RIE
RIL
ID FETCH-LOGICAL-a317t-1f72033ef9576b9ca42071302dea1aa3e571b10a2efe4506e29c99cabb5390623
IEDL.DBID RIE
ISICitedReferencesCount 2
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001465567400063&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Thu May 08 06:04:16 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a317t-1f72033ef9576b9ca42071302dea1aa3e571b10a2efe4506e29c99cabb5390623
OpenAccessLink https://ieeexplore.ieee.org/document/10554970
PageCount 3
ParticipantIDs ieee_primary_10554970
PublicationCentury 2000
PublicationDate 2024-April-14
PublicationDateYYYYMMDD 2024-04-14
PublicationDate_xml – month: 04
  year: 2024
  text: 2024-April-14
  day: 14
PublicationDecade 2020
PublicationTitle Proceedings (IEEE/ACM International Conference on Software Engineering Companion. Online)
PublicationTitleAbbrev ICSE-COMPANION
PublicationYear 2024
Publisher ACM
Publisher_xml – name: ACM
SSID ssj0003203497
ssib055574197
Score 2.2776864
Snippet Automatic unit test generators, particularly search-based software testing (SBST) tools such as EvoSuite, efficiently generate unit test suites with acceptable...
SourceID ieee
SourceType Publisher
StartPage 261
SubjectTerms Automatic Test Generation
Carving and Replaying
Generators
Large Language Models
Medical services
Readability
Software engineering
Software testing
Statistical analysis
Test pattern generators
Understandability
Unit Testing
Writing
Title Understandable Test Generation Through Capture/Replay and LLMs
URI https://ieeexplore.ieee.org/document/10554970
WOSCitedRecordID wos001465567400063&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELWgYmACRBHf8sCaNnbsOF5YKiqGUnVopW7V2b1ISKit-oHEv-cuSQsLA1OiyIMvH353Ob_3hHiKBeUBGuj7RlUmlBHrxBc-TYByWZp4LBHSymzCDYfFdOpHDVm94sIgYrX5DDt8WvXy58u4419lXTZzNN5RhX7sXF6TtfYvj7WWwLFpifEynGmWXnGNnI8ytpsRGhtXdPjo2Nf9l59KBSf9s39O5Fy0f4h5cnSAnAtxhItL8Tw5EFSYByXHtNDLWk6a77oc11Y8sgcrbhd0Kef-gC9Jw-Vg8LZpi0n_Zdx7TRpbhAQI7LeJKrl1mmHpqVYIPoLRXGqmeo6gADK0TgWVgsYSjU1z1D56GhaCzViVOLsSrcVygddCsnlkMEhlQ866MQC5nocY2fmBogtwI9oc_GxVK1_M9nHf_nH9TpxqAn3utihzL1rb9Q4fxEn83L5v1o_V8_oGkUuT5g
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELVQQYIJEEV844E1bezESbywVFRFpFWHVOpWnd2LhITaqh9I_Hvu0rSwMDAlijz48uF7l_N7T4gnnxEO0EDfN6oyIESsA5vZMADCsjRxXyKEldlEOhhk47Ed1mT1iguDiNXmM2zxadXLn879hn-VtdnMMbYpVeiHbJ1V07V2r48xhtJj3RTjhTjSLL6S1oI-KjbtiPJxnGYtPqbs7P7LUaVKKN3Tf07lTDR_qHlyuE865-IAZxfiebSnqDATSha01MutoDTfd1lszXhkBxbcMGgT6v6AL0nDZZ73V00x6r4UnV5QGyMEQOl-HaiSm6cRlpaqBWc9xJqLzVBPERRAhCZVToWgscTYhAlq6y0Nc85ErEscXYrGbD7DKyHZPtLFSIVDwsoxAImeOu_Z-4Gic3Atmhz8ZLHVvpjs4r754_qjOO4V_XySvw7ebsWJJgjAvRcV34nGernBe3HkP9fvq-VD9ey-AXtily8
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE%2FACM+International+Conference+on+Software+Engineering+Companion.+Online%29&rft.atitle=Understandable+Test+Generation+Through+Capture%2FReplay+and+LLMs&rft.au=Deljouyi%2C+Amirhossein&rft.date=2024-04-14&rft.pub=ACM&rft.eissn=2574-1934&rft.spage=261&rft.epage=263&rft_id=info:doi/10.1145%2F3639478.3639789&rft.externalDocID=10554970