An assessment of large language models for OpenMP-based code parallelization: a user perspective

Large language models have sparked a lot of attention in the research community in recent days, especially with the introduction of practical tools such as ChatGPT and Github Copilot. Their ability to solve complex programming tasks was also shown in several studies and commercial solutions increasi...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of big data Ročník 11; číslo 1; s. 161 - 23
Hlavní autoři: Mišić, Marko, Dodović, Matija
Médium: Journal Article
Jazyk:angličtina
Vydáno: Cham Springer International Publishing 01.12.2024
Springer Nature B.V
SpringerOpen
Témata:
Artificial intelligence
Big Data
Big Data and Artificial Intelligence in Emerging Engineering Fields
Chatbots
ChatGPT
Communications Engineering
Community research
Computation
Computational Science and Engineering
Computer Science
Data Mining and Knowledge Discovery
Database Management
Github Copilot
High performance computing
Human-computer interaction
Information Storage and Retrieval
Language modeling
Large language models
Mathematical Applications in Computer Science
Microprocessors
Networks
OpenMP
Parallel processing
Parallel programming
Performance evaluation
Software
Software development
Task complexity
Visual programming languages
Workloads
Parallel programming
High performance computing
Large language models
ChatGPT
OpenMP
Github Copilot
ISSN:2196-1115, 2196-1115
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Large language models have sparked a lot of attention in the research community in recent days, especially with the introduction of practical tools such as ChatGPT and Github Copilot. Their ability to solve complex programming tasks was also shown in several studies and commercial solutions increasing the interest in using them for software development in different fields. High performance computing is one of such fields, where parallel programming techniques have been extensively used to utilize raw computing power available in contemporary multicore and manycore processors. In this paper, we perform an evaluation of the ChatGPT and Github Copilot tools for OpenMP-based code parallelization using a proposed methodology. We used nine different benchmark applications which represent typical parallel programming workloads and compared their OpenMP-based parallel solutions produced manually and using ChatGPT and Github Copilot in terms of obtained speedup, applied optimizations, and quality of the solution. ChatGPT 3.5 and Github Copilot installed with Visual Studio Code 1.88 were used. We concluded that both tools can produce correct parallel code in most cases. However, performance-wise, ChatGPT can match manually produced and optimized parallel code only in simpler cases, as it lacks a deeper understanding of the code and the context. The results are much better with Github Copilot, where much less effort is needed to obtain correct and performant parallel solution.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2196-1115
2196-1115
DOI:10.1186/s40537-024-01019-z