Late Breaking Results: Fine-Tuning LLMs for Test Stimuli Generation
The understanding and reasoning capabilities of large language models (LLMs) with text data have made them widely used for test stimuli generation. Existing studies have primarily focused on methods such as prompt engineering or providing feedback to the LLMs' generated outputs to improve test...
Uloženo v:
| Vydáno v: | 2025 62nd ACM/IEEE Design Automation Conference (DAC) s. 1 - 2 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
22.06.2025
|
| Témata: | |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | The understanding and reasoning capabilities of large language models (LLMs) with text data have made them widely used for test stimuli generation. Existing studies have primarily focused on methods such as prompt engineering or providing feedback to the LLMs' generated outputs to improve test stimuli generation. However, these approaches have not been successful in enhancing the LLMs' domain-specific performance in generating test stimuli. In this paper, we introduce a framework for finetuning LLMs for test stimuli generation through dataset generation and reinforcement learning (RL). Our dataset generation approach creates a table-shaped test stimuli dataset, which helps ensure that the LLM produces consistent outputs. Additionally, our two-stage fine-tuning process involves training the LLMs on domain-specific data and using RL to provide feedback on the generated outputs, further enhancing the LLMs' performance in test stimuli generation. Experimental results confirm that our framework improves syntax correctness and code coverage of test stimuli, outperforming commercial models. |
|---|---|
| AbstractList | The understanding and reasoning capabilities of large language models (LLMs) with text data have made them widely used for test stimuli generation. Existing studies have primarily focused on methods such as prompt engineering or providing feedback to the LLMs' generated outputs to improve test stimuli generation. However, these approaches have not been successful in enhancing the LLMs' domain-specific performance in generating test stimuli. In this paper, we introduce a framework for finetuning LLMs for test stimuli generation through dataset generation and reinforcement learning (RL). Our dataset generation approach creates a table-shaped test stimuli dataset, which helps ensure that the LLM produces consistent outputs. Additionally, our two-stage fine-tuning process involves training the LLMs on domain-specific data and using RL to provide feedback on the generated outputs, further enhancing the LLMs' performance in test stimuli generation. Experimental results confirm that our framework improves syntax correctness and code coverage of test stimuli, outperforming commercial models. |
| Author | Park, Seonghyeon Park, Hyeonwoo Kang, Seokhyeong |
| Author_xml | – sequence: 1 givenname: Hyeonwoo surname: Park fullname: Park, Hyeonwoo email: jimmy0709@postech.ac.kr organization: Pohang University of Science and Technology,Pohang,Republic of Korea – sequence: 2 givenname: Seonghyeon surname: Park fullname: Park, Seonghyeon email: seonghyeon98@postech.ac.kr organization: Pohang University of Science and Technology,Pohang,Republic of Korea – sequence: 3 givenname: Seokhyeong surname: Kang fullname: Kang, Seokhyeong email: shkang@postech.ac.kr organization: Pohang University of Science and Technology,Pohang,Republic of Korea |
| BookMark | eNo1j81KxDAURiPoQsd5A5G8QMf8NGmuu7E6o1ARtK6H2_RWgp1U2nTh2zuDujpwDnzwXbDTOERi7FqKlZQCbu7XpdUuh5USyhyU1ApsccKWUIDTWhqhRe7OWVlhIn43En6G-MFfaZr7NN3yTYiU1XM8yqp6nng3jLymKfG3FPZzH_iWIo2YwhAv2VmH_UTLPy7Y--ahLh-z6mX7VK6rDGUBKfPGOo2tEWBBKSWMRHBW5OqQG5SdR6vI-dzZpmjyzrcIoEXrHRXGNN7qBbv63Q1EtPsawx7H793_N_0DRzpHnw |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/DAC63849.2025.11132967 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Xplore IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9798331503048 |
| EndPage | 2 |
| ExternalDocumentID | 11132967 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH CBEJK RIE RIO |
| ID | FETCH-LOGICAL-a179t-c5683ad50969222051a986042a17ba1fca62e8c486b7b4fcda9930dc8e755bc63 |
| IEDL.DBID | RIE |
| IngestDate | Wed Oct 01 07:05:15 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a179t-c5683ad50969222051a986042a17ba1fca62e8c486b7b4fcda9930dc8e755bc63 |
| PageCount | 2 |
| ParticipantIDs | ieee_primary_11132967 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-June-22 |
| PublicationDateYYYYMMDD | 2025-06-22 |
| PublicationDate_xml | – month: 06 year: 2025 text: 2025-June-22 day: 22 |
| PublicationDecade | 2020 |
| PublicationTitle | 2025 62nd ACM/IEEE Design Automation Conference (DAC) |
| PublicationTitleAbbrev | DAC |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| Score | 2.2954834 |
| Snippet | The understanding and reasoning capabilities of large language models (LLMs) with text data have made them widely used for test stimuli generation. Existing... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | Codes Cognition Design automation Large language models Prompt engineering Reinforcement learning Syntactics Training |
| Title | Late Breaking Results: Fine-Tuning LLMs for Test Stimuli Generation |
| URI | https://ieeexplore.ieee.org/document/11132967 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NSwMxEA22ePCkYkWtSg5e0-6m2Xx409XioZaiK_RWstkJFKSV7q6_38luq3jw4C0kgZCZZPKYzJsh5EapqOCGW6al8AwNHjAjfcykyo3VoHSUi6bYhJpO9XxuZluyesOFAYAm-AwGodn85RdrVwdX2bAtiy5Vh3SUki1Za8v6jSMzfLhL8TSJQD_hyWA3-VfZlObVGB_-c70j0vvh39HZ98tyTPZgdULSCcJCeo8gL3i36QuU9XtV3tIx4kSW1cG_QSeT55IiDKUZGnv6Wi1D8BNtU0sHDfTI2_gxS5_YtgQCs3hTKuYSqUe2CDlaDA-c2NgaLfGi4XBuY--s5KCd0DJXufCusIg3osKhlJMkd3J0Srqr9QrOCPXhhyVyaP2UFWaEU7z3AEYYr8DG8pz0ggQWH22Wi8Vu8xd_9PfJQZBzCJvi_JJ0q00NV2TffVbLcnPd6OYLo2CQEA |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aBT2pWPFtDl7T7qbZPLxptVTclqIr9Fay2QkUyla6u_5-k91W8eDBW0gCITPJ5GMy3wxCt0IEGVVUE8mZJc7gAVHchoSLVGkJQgYpq4tNiPFYTqdqsiar11wYAKiDz6Djm_VffrY0lXeVdZuy6Fxso52IMRo0dK017zcMVPfxvu_OE_MEFBp1NtN_FU6p343BwT9XPETtHwYenny_LUdoC_Jj1I8dMMQPDuZ5_zZ-haJalMUdHjikSJLKezhwHI8K7IAoTpy5x2_l3Ic_4Sa5tNdBG70PnpL-kKyLIBDt7kpJTMRlT2c-S4uinhUbaiW5u2puONWhNZpTkIZJnoqUWZNphziCzDg5R1FqeO8EtfJlDqcIW__HEhhn_4RmquemWGsBFFNWgA75GWp7Ccw-mjwXs83mz__ov0F7w2QUz-Ln8csF2vcy90FUlF6iVrmq4Artms9yXqyuaz19AVOyk1c |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2025+62nd+ACM%2FIEEE+Design+Automation+Conference+%28DAC%29&rft.atitle=Late+Breaking+Results%3A+Fine-Tuning+LLMs+for+Test+Stimuli+Generation&rft.au=Park%2C+Hyeonwoo&rft.au=Park%2C+Seonghyeon&rft.au=Kang%2C+Seokhyeong&rft.date=2025-06-22&rft.pub=IEEE&rft.spage=1&rft.epage=2&rft_id=info:doi/10.1109%2FDAC63849.2025.11132967&rft.externalDocID=11132967 |