Software Model Evolution with Large Language Models: Experiments on Simulated, Public, and Industrial Datasets

Modeling structure and behavior of software systems plays a crucial role in the industrial practice of software engineering. As with other software engineering artifacts, software models are subject to evolution. Supporting modelers in evolving software models with recommendations for model completi...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Proceedings / International Conference on Software Engineering s. 950 - 962
Hlavní autori: Tinnes, Christof, Welter, Alisa, Apel, Sven
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 26.04.2025
Predmet:
ai4se
Biological system modeling
copilots
Data models
generative ai
intelligent modeling assistant
Large language models
model completion
model-driven engineering
Real-time systems
Retrieval augmented generation
Semantics
Software
Software product lines
Software systems
Source coding
ISSN:1558-1225
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract Modeling structure and behavior of software systems plays a crucial role in the industrial practice of software engineering. As with other software engineering artifacts, software models are subject to evolution. Supporting modelers in evolving software models with recommendations for model completions is still an open problem, though. In this paper, we explore the potential of large language models for this task. In particular, we propose an approach, RAMC, leveraging large language models, model histories, and retrieval-augmented generation for model completion. Through experiments on three datasets, including an industrial application, one public open-source community dataset, and one controlled collection of simulated model repositories, we evaluate the potential of large language models for model completion with RAMC. We found that large language models are indeed a promising technology for supporting software model evolution (62.30% semantically correct completions on real-world industrial data and up to 86.19% type-correct completions). The general inference capabilities of large language models are particularly useful when dealing with concepts for which there are few, noisy, or no examples at all.
AbstractList Modeling structure and behavior of software systems plays a crucial role in the industrial practice of software engineering. As with other software engineering artifacts, software models are subject to evolution. Supporting modelers in evolving software models with recommendations for model completions is still an open problem, though. In this paper, we explore the potential of large language models for this task. In particular, we propose an approach, RAMC, leveraging large language models, model histories, and retrieval-augmented generation for model completion. Through experiments on three datasets, including an industrial application, one public open-source community dataset, and one controlled collection of simulated model repositories, we evaluate the potential of large language models for model completion with RAMC. We found that large language models are indeed a promising technology for supporting software model evolution (62.30% semantically correct completions on real-world industrial data and up to 86.19% type-correct completions). The general inference capabilities of large language models are particularly useful when dealing with concepts for which there are few, noisy, or no examples at all.
Author Welter, Alisa
Tinnes, Christof
Apel, Sven
Author_xml – sequence: 1
  givenname: Christof
  surname: Tinnes
  fullname: Tinnes, Christof
  email: christof.tinnes@siemens.com
  organization: Siemens AG,Garching bei München,Germany
– sequence: 2
  givenname: Alisa
  surname: Welter
  fullname: Welter, Alisa
  email: welter@cs.uni-saarland.de
  organization: Saarland University,Saarbrücken,Germany
– sequence: 3
  givenname: Sven
  surname: Apel
  fullname: Apel, Sven
  email: apel@cs.uni-saarland.de
  organization: Saarland University,Saarbrücken,Germany
BookMark eNotkNFKwzAYhaMo6ObeYBd5gHUmf5Y28U5m1cFEYXo90uTvjHTpaFKnb2_B3Zxz8_FxOCNyEdqAhEw5m3PO9O1quSmlFItiDgzknDHO4YxMdKGVEFwymWt-Tq65lCrjAPKKjGL8YozlC62vSdi0dTqaDulL67Ch5Xfb9Mm3gR59-qRr0-1wyLDrze7ExDta_hyw83sMKdIB3fh935iEbkbf-qrxdkZNcHQVXB9T501DH0wyEVO8IZe1aSJOTj0mH4_l-_I5W78-rZb368xAzlJmlbAcoZZQKSFNLmprhXBOL5AJtA5yy3juWCUF1mAL7ngB0lQoQbDCghiT6b_XI-L2MGw13e92OAy0Ukr8AQcWXMM
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/ICSE55347.2025.00112
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE/IET Electronic Library
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9798331505691
EISSN 1558-1225
EndPage 962
ExternalDocumentID 11029888
Genre orig-research
GroupedDBID -~X
.4S
.DC
29O
5VS
6IE
6IF
6IH
6IK
6IL
6IM
6IN
8US
AAJGR
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
ARCSS
AVWKF
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
EDO
FEDTE
I-F
IEGSK
IJVOP
IPLJI
M43
OCL
RIE
RIL
RIO
ID FETCH-LOGICAL-a260t-c83c1e2f52b835a63fcc33dd94e03ecd26c016d0b53ef2c71d1725abe52307c23
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001538318100074&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 01:40:13 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a260t-c83c1e2f52b835a63fcc33dd94e03ecd26c016d0b53ef2c71d1725abe52307c23
PageCount 13
ParticipantIDs ieee_primary_11029888
PublicationCentury 2000
PublicationDate 2025-April-26
PublicationDateYYYYMMDD 2025-04-26
PublicationDate_xml – month: 04
  year: 2025
  text: 2025-April-26
  day: 26
PublicationDecade 2020
PublicationTitle Proceedings / International Conference on Software Engineering
PublicationTitleAbbrev ICSE
PublicationYear 2025
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0006499
Score 2.290509
Snippet Modeling structure and behavior of software systems plays a crucial role in the industrial practice of software engineering. As with other software engineering...
SourceID ieee
SourceType Publisher
StartPage 950
SubjectTerms ai4se
Biological system modeling
copilots
Data models
generative ai
intelligent modeling assistant
Large language models
model completion
model-driven engineering
Real-time systems
Retrieval augmented generation
Semantics
Software
Software product lines
Software systems
Source coding
Title Software Model Evolution with Large Language Models: Experiments on Simulated, Public, and Industrial Datasets
URI https://ieeexplore.ieee.org/document/11029888
WOSCitedRecordID wos001538318100074&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3Pa8IwFA5TdtjJ_XDsNznsaKcmTdPu6iobiAhuw5ukL68gbHXY6v79vbRV2WGHQQmlBAJJmve97-W9j7F7At20axA8FaL0_NSVvAWrPJH4fqpSDGxZjuF9pMfjcDaLJnWyepkLg4jl5TN8cK9lLN8uYe2osi6ZKhGRy9ZgDa11lay1O3YDwu51bly_F3VfBtNYKelr8gGFKgMO4peCSmlAhq1_Dn3M2vtUPD7ZGZkTdoDZKWtttRh4_WuesWxK5-m3WSF36mYfPN7Ue4o7ppWP3IVvaitysuqTP_J4V98_59R1uvh0al5oO7yi8zrcZJbv5T34kynI7BV5m70N49fBs1drKXiGPJbCg1BCH0WqREKYywQyBZDS2sjHnkSwIgACf7aXKImpAN23hGyUSdCxxhqEPGfNbJnhBePCN1qibwDIlzKoTWh74ENCDxBaCC5Z283f_KsqlzHfTt3VH9-v2ZFbIheiEcENaxarNd6yQ9gUi3x1Vy7yD6wOqiw
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1La8JAEF5aW2hP9mHpu3vo0bS6j0R7tYpSK4K2eJPN7ASENhYT7d_vbBKVHnoohCWEhYXdzc433-zMx9g9gW7aNQiebqD0VORK3oLVngiVinSEvs3KMbz3g8GgMZk0h0WyepYLg4jZ5TN8cK9ZLN_OYemoskcyVaJJLtsu29NKiXqerrU5eH1C70V2XL3WfOy1Rm2tpQrICxQ6CzmIXxoqmQnplP85-BGrbJPx-HBjZo7ZDsYnrLxWY-DFz3nK4hGdqN9mgdzpm33w9qrYVdxxrbzvrnxTm9OTeZ_kibc3Ff4TTl1Hs0-n54W2ynNCr8pNbPlW4IM_m5QMX5pU2FunPW51vUJNwTPks6QeNCTUUURahIS6jC8jACmtbSqsSQQrfCD4Z2uhlhgJCOqWsI02ITreOAAhz1gpnsd4zrhQJpCoDAB5UwYD07A1UBDSA4QX_AtWcfM3_coLZkzXU3f5x_c7dtAdv_an_d7g5YoduuVyARvhX7NSuljiDduHVTpLFrfZgv8AsNqtcw
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%2F+International+Conference+on+Software+Engineering&rft.atitle=Software+Model+Evolution+with+Large+Language+Models%3A+Experiments+on+Simulated%2C+Public%2C+and+Industrial+Datasets&rft.au=Tinnes%2C+Christof&rft.au=Welter%2C+Alisa&rft.au=Apel%2C+Sven&rft.date=2025-04-26&rft.pub=IEEE&rft.eissn=1558-1225&rft.spage=950&rft.epage=962&rft_id=info:doi/10.1109%2FICSE55347.2025.00112&rft.externalDocID=11029888