Late Breaking Results: Fine-Tuning LLMs for Test Stimuli Generation

The understanding and reasoning capabilities of large language models (LLMs) with text data have made them widely used for test stimuli generation. Existing studies have primarily focused on methods such as prompt engineering or providing feedback to the LLMs' generated outputs to improve test...

Full description

Saved in:
Bibliographic Details
Published in:2025 62nd ACM/IEEE Design Automation Conference (DAC) pp. 1 - 2
Main Authors: Park, Hyeonwoo, Park, Seonghyeon, Kang, Seokhyeong
Format: Conference Proceeding
Language:English
Published: IEEE 22.06.2025
Subjects:
Codes
Cognition
Design automation
Large language models
Prompt engineering
Reinforcement learning
Syntactics
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract The understanding and reasoning capabilities of large language models (LLMs) with text data have made them widely used for test stimuli generation. Existing studies have primarily focused on methods such as prompt engineering or providing feedback to the LLMs' generated outputs to improve test stimuli generation. However, these approaches have not been successful in enhancing the LLMs' domain-specific performance in generating test stimuli. In this paper, we introduce a framework for finetuning LLMs for test stimuli generation through dataset generation and reinforcement learning (RL). Our dataset generation approach creates a table-shaped test stimuli dataset, which helps ensure that the LLM produces consistent outputs. Additionally, our two-stage fine-tuning process involves training the LLMs on domain-specific data and using RL to provide feedback on the generated outputs, further enhancing the LLMs' performance in test stimuli generation. Experimental results confirm that our framework improves syntax correctness and code coverage of test stimuli, outperforming commercial models.
AbstractList The understanding and reasoning capabilities of large language models (LLMs) with text data have made them widely used for test stimuli generation. Existing studies have primarily focused on methods such as prompt engineering or providing feedback to the LLMs' generated outputs to improve test stimuli generation. However, these approaches have not been successful in enhancing the LLMs' domain-specific performance in generating test stimuli. In this paper, we introduce a framework for finetuning LLMs for test stimuli generation through dataset generation and reinforcement learning (RL). Our dataset generation approach creates a table-shaped test stimuli dataset, which helps ensure that the LLM produces consistent outputs. Additionally, our two-stage fine-tuning process involves training the LLMs on domain-specific data and using RL to provide feedback on the generated outputs, further enhancing the LLMs' performance in test stimuli generation. Experimental results confirm that our framework improves syntax correctness and code coverage of test stimuli, outperforming commercial models.
Author Park, Seonghyeon
Park, Hyeonwoo
Kang, Seokhyeong
Author_xml – sequence: 1
  givenname: Hyeonwoo
  surname: Park
  fullname: Park, Hyeonwoo
  email: jimmy0709@postech.ac.kr
  organization: Pohang University of Science and Technology,Pohang,Republic of Korea
– sequence: 2
  givenname: Seonghyeon
  surname: Park
  fullname: Park, Seonghyeon
  email: seonghyeon98@postech.ac.kr
  organization: Pohang University of Science and Technology,Pohang,Republic of Korea
– sequence: 3
  givenname: Seokhyeong
  surname: Kang
  fullname: Kang, Seokhyeong
  email: shkang@postech.ac.kr
  organization: Pohang University of Science and Technology,Pohang,Republic of Korea
BookMark eNo1j81KxDAURiPoQsd5A5G8QMf8NGmuu7E6o1ARtK6H2_RWgp1U2nTh2zuDujpwDnzwXbDTOERi7FqKlZQCbu7XpdUuh5USyhyU1ApsccKWUIDTWhqhRe7OWVlhIn43En6G-MFfaZr7NN3yTYiU1XM8yqp6nng3jLymKfG3FPZzH_iWIo2YwhAv2VmH_UTLPy7Y--ahLh-z6mX7VK6rDGUBKfPGOo2tEWBBKSWMRHBW5OqQG5SdR6vI-dzZpmjyzrcIoEXrHRXGNN7qBbv63Q1EtPsawx7H793_N_0DRzpHnw
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/DAC63849.2025.11132967
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9798331503048
EndPage 2
ExternalDocumentID 11132967
Genre orig-research
GroupedDBID 6IE
6IH
CBEJK
RIE
RIO
ID FETCH-LOGICAL-a179t-c5683ad50969222051a986042a17ba1fca62e8c486b7b4fcda9930dc8e755bc63
IEDL.DBID RIE
IngestDate Wed Oct 01 07:05:15 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a179t-c5683ad50969222051a986042a17ba1fca62e8c486b7b4fcda9930dc8e755bc63
PageCount 2
ParticipantIDs ieee_primary_11132967
PublicationCentury 2000
PublicationDate 2025-June-22
PublicationDateYYYYMMDD 2025-06-22
PublicationDate_xml – month: 06
  year: 2025
  text: 2025-June-22
  day: 22
PublicationDecade 2020
PublicationTitle 2025 62nd ACM/IEEE Design Automation Conference (DAC)
PublicationTitleAbbrev DAC
PublicationYear 2025
Publisher IEEE
Publisher_xml – name: IEEE
Score 2.2950602
Snippet The understanding and reasoning capabilities of large language models (LLMs) with text data have made them widely used for test stimuli generation. Existing...
SourceID ieee
SourceType Publisher
StartPage 1
SubjectTerms Codes
Cognition
Design automation
Large language models
Prompt engineering
Reinforcement learning
Syntactics
Training
Title Late Breaking Results: Fine-Tuning LLMs for Test Stimuli Generation
URI https://ieeexplore.ieee.org/document/11132967
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA62ePCkYkWtSg5e0-5uNy9vWi0eailaobeSzM5CQVrp7vr7ney2igcP3kISEjKTx8dkvhnGbgbaRZjGmciVcSJVoIXPnRJSQyxRxWAzqJNN6MnEzOd2uiWr11wYRKydz7AXivVffraGKpjK-k1adKVbrKW1ashaW9ZvHNn-w92QdlMa6CeJ7O06_0qbUr8ao8N_znfEOj_8Oz79flmO2R6uTthwTLCQ3xPIC9Zt_oJF9V4Wt3xEOFHMqmDf4OPxc8EJhvIZXfb8tVwG5yfehJYOGuiwt9HjbPgktikQhKOTUgqQygxcFmK02CRwYmNnjaKDRs3exTk4laCB1CivfZpD5ghvRBkY1FJ6UINT1l6tV3jGuFMpSuNtZH1EGMg7oNGQRpWpJ23BOesECSw-migXi93iL_6o77KDIOfgNpUkl6xdbiq8YvvwWS6LzXWtmy8Y4JBW
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB60CnpSseLbHLym3d3mtd60WipuS9EVvJVkNgsFaaW76-832W0VDx68hSQkZCaPj8l8MwDXPakDy8KM5kJpygRKanItKJcYcitCjDOsk03I8Vi9vcWTFVm95sJYa2vnM9vxxfovP1tg5U1l3SYtupCbsMUZi4KGrrXi_YZB3L2_7bv9xDwBJeKddfdfiVPqd2Ow988Z96H9w8Ajk--35QA27PwQ-okDhuTOwTxv3ybPtqjey-KGDBxSpGnlLRwkSUYFcUCUpO66Jy_lzLs_kSa4tNdBG14HD2l_SFdJEKh2Z6WkyIXq6cxHaYkjz4oNdayEO2qu2egwRy0iq5ApYaRhOWbaIY4gQ2Ul5wZF7wha88XcHgPRglmuTBzEJnAoyGh0o1k3KmfG6QtPoO0lMP1o4lxM14s__aP-CnaG6SiZJo_jpzPY9TL3TlRRdA6tclnZC9jGz3JWLC9rPX0BzhuTnQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2025+62nd+ACM%2FIEEE+Design+Automation+Conference+%28DAC%29&rft.atitle=Late+Breaking+Results%3A+Fine-Tuning+LLMs+for+Test+Stimuli+Generation&rft.au=Park%2C+Hyeonwoo&rft.au=Park%2C+Seonghyeon&rft.au=Kang%2C+Seokhyeong&rft.date=2025-06-22&rft.pub=IEEE&rft.spage=1&rft.epage=2&rft_id=info:doi/10.1109%2FDAC63849.2025.11132967&rft.externalDocID=11132967