ITG: Trace Generation via Iterative Interaction between LLM Query and Trace Checking

Due to the complexity of linear temporal logic (LTL) trace generation (PSPACE-Complete), existing neural network-based approaches will fail as the formula sizes increase. Recently, large language models (LLMs) have demonstrated remarkable reasoning capabilities, benefiting from efficient training on...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Technologies Results (Online) s. 11 - 15
Hlavní autoři: Luo, Weilin, Fang, Weiyuan, Qiu, Junming, Wan, Hai, Liu, Yanan, Ye, Rongzhen
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: ACM 14.04.2024
Témata:
Accuracy
Chatbots
Cognition
Complexity theory
Iterative methods
large language model
Large language models
linear temporal logic
Logic
satisfiability checking
Software engineering
Synthetic data
trace checking
trace generation
Training
ISSN:2832-7632
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract Due to the complexity of linear temporal logic (LTL) trace generation (PSPACE-Complete), existing neural network-based approaches will fail as the formula sizes increase. Recently, large language models (LLMs) have demonstrated remarkable reasoning capabilities, benefiting from efficient training on hyper-scale data. Inspired by this, we propose an iterative interaction framework for applying LLMs, exemplified by ChatGPT, to generate a trace satisfying a given LTL formula. The key insight behind it is to transfer the powerful reasoning capabilities of LLM to LTL trace generation via iterative interaction between LLM reasoning and logical reasoning. Preliminary results show that compared with the state-of-the-art approach, the accuracy is relatively improved by 9.7%-23.4%. Besides, we show that our framework is able to produce heuristics for new tasks, which provides a reference for other reasoning-heavy tasks requiring heuristics.
AbstractList Due to the complexity of linear temporal logic (LTL) trace generation (PSPACE-Complete), existing neural network-based approaches will fail as the formula sizes increase. Recently, large language models (LLMs) have demonstrated remarkable reasoning capabilities, benefiting from efficient training on hyper-scale data. Inspired by this, we propose an iterative interaction framework for applying LLMs, exemplified by ChatGPT, to generate a trace satisfying a given LTL formula. The key insight behind it is to transfer the powerful reasoning capabilities of LLM to LTL trace generation via iterative interaction between LLM reasoning and logical reasoning. Preliminary results show that compared with the state-of-the-art approach, the accuracy is relatively improved by 9.7%-23.4%. Besides, we show that our framework is able to produce heuristics for new tasks, which provides a reference for other reasoning-heavy tasks requiring heuristics.
Author Fang, Weiyuan
Ye, Rongzhen
Luo, Weilin
Wan, Hai
Qiu, Junming
Liu, Yanan
Author_xml – sequence: 1
  givenname: Weilin
  surname: Luo
  fullname: Luo, Weilin
  email: luowlin5@mail.sysu.edu.cn
  organization: School of Computer Science and Engineering, Sun Yat-sen University,Guangzhou,China
– sequence: 2
  givenname: Weiyuan
  surname: Fang
  fullname: Fang, Weiyuan
  email: fangwy3@mail2.sysu.edu.cn
  organization: School of Computer Science and Engineering, Sun Yat-sen University,Guangzhou,China
– sequence: 3
  givenname: Junming
  surname: Qiu
  fullname: Qiu, Junming
  email: qiujm9@mail2.sysu.edu.cn
  organization: School of Computer Science and Engineering, Sun Yat-sen University,Guangzhou,China
– sequence: 4
  givenname: Hai
  surname: Wan
  fullname: Wan, Hai
  email: wanhai@mail.sysu.edu.cn
  organization: School of Computer Science and Engineering, Sun Yat-sen University,Guangzhou,China
– sequence: 5
  givenname: Yanan
  surname: Liu
  fullname: Liu, Yanan
  email: liuyn56@mail2.sysu.edu.cn
  organization: School of Computer Science and Engineering, Sun Yat-sen University,Guangzhou,China
– sequence: 6
  givenname: Rongzhen
  surname: Ye
  fullname: Ye, Rongzhen
  email: yerzh@mail2.sysu.edu.cn
  organization: School of Computer Science and Engineering, Sun Yat-sen University,Guangzhou,China
BookMark eNotjEtLw0AUhUdRsNas3biYP5A674c7KRoDERHiutxJbnRQp5LESv9909rNeXAO3yU5S-uEhFxztuBc6VtppFfWLPZurT8hmbfeKcYs05OckplwUuTWSHFBsmGIgWmtreJezUhd1sUdrXtokBaYsIcxrhPdRKDleGgbpGXax-awBBz_EBOtqmf6-ov9lkJqj4DlBzafMb1fkfMOvgbMjj4nb48P9fIpr16Kcnlf5SCcH3PdGatCCKqB4DoHQlptoe0Y050GAyF4IyauM1yAkWhw-jAGvlGyFa2Xc3Lzz42IuPrp4zf02xVnVhgvvdwBspJSaw
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1145/3639476.3639779
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE/IET Electronic Library
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9798400705007
EISSN 2832-7632
EndPage 15
ExternalDocumentID 10726939
Genre orig-research
GrantInformation_xml – fundername: Fundamental Research Funds for the Central Universities, Sun Yat-sen University
  grantid: 23ptpy31
  funderid: 10.13039/100007844
– fundername: Humanities and Social Science Research Project of Ministry of Education
  grantid: 18YJCZH006
  funderid: 10.13039/100009950
– fundername: National Natural Science Foundation of China
  grantid: 62276284,61976232,51978675
  funderid: 10.13039/501100001809
GroupedDBID 6IE
6IL
6IN
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
OCL
RIE
RIL
ID FETCH-LOGICAL-a289t-5f674bbb4cab8f8a23757adf005f5a6abb962ace8612a63e6e8a200a9c43d2d93
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001234856600003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 03:00:32 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a289t-5f674bbb4cab8f8a23757adf005f5a6abb962ace8612a63e6e8a200a9c43d2d93
OpenAccessLink https://dl.acm.org/doi/pdf/10.1145/3639476.3639779
PageCount 5
ParticipantIDs ieee_primary_10726939
PublicationCentury 2000
PublicationDate 2024-April-14
PublicationDateYYYYMMDD 2024-04-14
PublicationDate_xml – month: 04
  year: 2024
  text: 2024-April-14
  day: 14
PublicationDecade 2020
PublicationTitle IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Technologies Results (Online)
PublicationTitleAbbrev ICSE-NIER
PublicationYear 2024
Publisher ACM
Publisher_xml – name: ACM
SSID ssib055574194
ssj0003211718
Score 2.2567515
Snippet Due to the complexity of linear temporal logic (LTL) trace generation (PSPACE-Complete), existing neural network-based approaches will fail as the formula...
SourceID ieee
SourceType Publisher
StartPage 11
SubjectTerms Accuracy
Chatbots
Cognition
Complexity theory
Iterative methods
large language model
Large language models
linear temporal logic
Logic
satisfiability checking
Software engineering
Synthetic data
trace checking
trace generation
Training
Title ITG: Trace Generation via Iterative Interaction between LLM Query and Trace Checking
URI https://ieeexplore.ieee.org/document/10726939
WOSCitedRecordID wos001234856600003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NTwIxEG2EePCkRozf6cFrEfpdr0SUBAkmmHAjbbeNXBaDLIn_3ml3kZMHT9vsbppm2s5rp_NeEboXmnHnuSEANppwaxzR0WkindWa9x2lPuvMjtVkoudzM23I6pkLE0LIyWehm4r5LL9Y-SqFymCGKyoNMy3UUkrWZK3d4BFCADg2gyu5YQZbG3C8jZxPn4sHBmjMleymp8q5W_v7VDKcDI__2ZAT1NkT8_D0F3JO0UEoz9BsNHt-xAA68LFWkU7GxtulxaOsmQwODefIX01iwE1uFh6PX_FbFdbf2JZFU8HgI_gUPe-g9-HTbPBCmssSiIU904aIKBV3znFvnY7aUqaEskWEWRaFldY5IynUo2FJYyULMsA_vZ41nrOCFoado3a5KsMFwlRG5xk1UTHNhdPGBWUUC4XX1homLlEnmWTxWethLHbWuPrj_TU6orAUSGcwfX6D2pt1FW7Rod9ull_ru9yLP-JknFo
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT8IwFG4UTfSkRoy_7cHrEPq7XokIcRBMZsKNtF0XuQyCQOJ_72s35OTB05ptaZrX9n3t6_u-IvTIFWXWMZ0A2KiEGW0TVViVCGuUYh1LiIs6s6kcjdRkosc1WT1yYbz3MfnMt0IxnuXnc7cOoTKY4ZIITfU-OuCMkXZF19oOH845wGM9vIIjprC5AddbC_p0GH-igMdMilZ4ypi9tbtRJQJK7-SfTTlFzR01D49_QecM7fnyHGWD7PUZA-zAx0pHOpgbb2YGD6JqMrg0HGN_FY0B19lZOE2H-H3tl9_YlHldQffTuxA_b6KP3kvW7Sf1dQmJgV3TKuGFkMxay5yxqlCGUMmlyQuYZwU3wlirBYF6FCxqjKBeePin3TbaMZqTXNML1Cjnpb9EmIjCOkp0Iali3CptvdSS-twpYzTlV6gZTDJdVIoY0601rv94_4CO-tkwnaaD0dsNOiawMAgnMh12ixqr5drfoUO3Wc2-lvexR38ANeefoQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=IEEE%2FACM+International+Conference+on+Software+Engineering%3A+New+Ideas+and+Emerging+Technologies+Results+%28Online%29&rft.atitle=ITG%3A+Trace+Generation+via+Iterative+Interaction+between+LLM+Query+and+Trace+Checking&rft.au=Luo%2C+Weilin&rft.au=Fang%2C+Weiyuan&rft.au=Qiu%2C+Junming&rft.au=Wan%2C+Hai&rft.date=2024-04-14&rft.pub=ACM&rft.eissn=2832-7632&rft.spage=11&rft.epage=15&rft_id=info:doi/10.1145%2F3639476.3639779&rft.externalDocID=10726939