Enhancing GPT-3.5's Proficiency in Netlogo Through Few-Shot Prompting and Retrieval-Augmented Generation

Recognizing the limited research on Large Language Models (LLMs) capabilities with low-resource languages, this study evaluates and increases the proficiency of the LLM GPT-3.5 in generating interface and procedural code elements for NetLogo, a multi-agent programming language and modeling environme...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings - Winter Simulation Conference s. 666 - 677
Hlavní autoři: Martinez, Joseph, Llinas, Brian, Botello, Jhon G., Padilla, Jose J., Frydenlund, Erika
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 15.12.2024
Témata:
Codes
Computer languages
Hands
Large language models
Retrieval augmented generation
ISSN:1558-4305
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Recognizing the limited research on Large Language Models (LLMs) capabilities with low-resource languages, this study evaluates and increases the proficiency of the LLM GPT-3.5 in generating interface and procedural code elements for NetLogo, a multi-agent programming language and modeling environment. To achieve this, we employed "few-shot" prompting and Retrieval-Augmented Generation (RAG) methodologies using two manually created datasets, NetLogoEvalCode and NetLogoEvalInterface. The results demonstrate that GPT-3.5 can generate NetLogo elements and code procedures more effectively when provided with additional examples to learn from, highlighting the potential of LLMs in aiding the development of agent-based models (ABMs). On the other hand, the RAG model obtained a poor performance. We listed possible reasons for this result, which were aligned with RAG's common challenges identified by the state-of-the-art. We propose future research directions for leveraging LLMs for simulation development and instructional purposes in the context of ABMs.
ISSN:1558-4305
DOI:10.1109/WSC63780.2024.10838967