On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML

Most experts agree that large language models (LLMs), such as those used by Copilot and ChatGPT, are expected to revolutionize the way in which software is developed. Many papers are currently devoted to analyzing the potential advantages and limitations of these generative AI models for writing cod...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Software and systems modeling Jg. 22; H. 3; S. 781 - 793
Hauptverfasser: Cámara, Javier, Troya, Javier, Burgueño, Lola, Vallecillo, Antonio
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Berlin/Heidelberg Springer Berlin Heidelberg 01.06.2023
Springer Nature B.V
Schlagworte:
Artificial intelligence
Chatbots
Compilers
Computer Science
Experiments
Expert Voice
Generative artificial intelligence
Information Systems Applications (incl.Internet)
Interpreters
IT in Business
Large language models
Modelling
Programming Languages
Programming Techniques
Software
Software Engineering
Software Engineering/Programming and Operating Systems
Large language models
Software models
ChatGPT
UML
Modeling languages
ISSN:1619-1366, 1619-1374
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Most experts agree that large language models (LLMs), such as those used by Copilot and ChatGPT, are expected to revolutionize the way in which software is developed. Many papers are currently devoted to analyzing the potential advantages and limitations of these generative AI models for writing code. However, the analysis of the current state of LLMs with respect to software modeling has received little attention. In this paper, we investigate the current capabilities of ChatGPT to perform modeling tasks and to assist modelers, while also trying to identify its main shortcomings. Our findings show that, in contrast to code generation, the performance of the current version of ChatGPT for software modeling is limited, with various syntactic and semantic deficiencies, lack of consistency in responses and scalability issues. We also outline our views on how we perceive the role that LLMs can play in the software modeling discipline in the short term, and how the modeling community can help to improve the current capabilities of ChatGPT and the coming LLMs for software modeling.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1619-1366
1619-1374
DOI:10.1007/s10270-023-01105-5