On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML

Most experts agree that large language models (LLMs), such as those used by Copilot and ChatGPT, are expected to revolutionize the way in which software is developed. Many papers are currently devoted to analyzing the potential advantages and limitations of these generative AI models for writing cod...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Software and systems modeling Ročník 22; číslo 3; s. 781 - 793
Hlavní autoři: Cámara, Javier, Troya, Javier, Burgueño, Lola, Vallecillo, Antonio
Médium: Journal Article
Jazyk:angličtina
Vydáno: Berlin/Heidelberg Springer Berlin Heidelberg 01.06.2023
Springer Nature B.V
Témata:
Artificial intelligence
Chatbots
Compilers
Computer Science
Experiments
Expert Voice
Generative artificial intelligence
Information Systems Applications (incl.Internet)
Interpreters
IT in Business
Large language models
Modelling
Programming Languages
Programming Techniques
Software
Software Engineering
Software Engineering/Programming and Operating Systems
Large language models
Software models
ChatGPT
UML
Modeling languages
ISSN:1619-1366, 1619-1374
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Most experts agree that large language models (LLMs), such as those used by Copilot and ChatGPT, are expected to revolutionize the way in which software is developed. Many papers are currently devoted to analyzing the potential advantages and limitations of these generative AI models for writing code. However, the analysis of the current state of LLMs with respect to software modeling has received little attention. In this paper, we investigate the current capabilities of ChatGPT to perform modeling tasks and to assist modelers, while also trying to identify its main shortcomings. Our findings show that, in contrast to code generation, the performance of the current version of ChatGPT for software modeling is limited, with various syntactic and semantic deficiencies, lack of consistency in responses and scalability issues. We also outline our views on how we perceive the role that LLMs can play in the software modeling discipline in the short term, and how the modeling community can help to improve the current capabilities of ChatGPT and the coming LLMs for software modeling.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1619-1366
1619-1374
DOI:10.1007/s10270-023-01105-5