On the Impact of Requirements Smells in Prompts: The Case of Automated Traceability

Large language models (LLMs) are increasingly used to generate software artifacts, such as source code, tests, and trace links. Requirements play a central role in shaping the input prompts that guide LLMs, as they are often used as part of the prompts to synthesize the artifacts. However, the impac...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Technologies Results (Online) s. 51 - 55
Hlavní autoři: Vogelsang, Andreas, Korn, Alexander, Broccia, Giovanna, Ferrari, Alessio, Fischbach, Jannik, Arora, Chetan
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 27.04.2025
Témata:
Codes
Focusing
Guidelines
Large language models
LLMs
Requirements Eng
Smells
Software
Software engineering
Source coding
Traceability
ISSN:2832-7632
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Large language models (LLMs) are increasingly used to generate software artifacts, such as source code, tests, and trace links. Requirements play a central role in shaping the input prompts that guide LLMs, as they are often used as part of the prompts to synthesize the artifacts. However, the impact of requirements formulation on LLM performance remains unclear. In this paper, we investigate the role of requirements smells- indicators of potential issues like ambiguity and inconsistency- when used in prompts for LLMs. We conducted experiments using two LLMs focusing on automated trace link generation between requirements and code. Our results show mixed outcomes: while requirements smells had a small but significant effect when predicting whether a requirement was implemented in a piece of code (i.e., a trace link exists), no significant effect was observed when tracing the requirements with the associated lines of code. These findings suggest that requirements smells can affect LLM performance in certain SE tasks but may not uniformly impact all tasks. We highlight the need for further research to understand these nuances and propose future work toward developing guidelines for mitigating the negative effects of requirements smells in AI-driven SE processes.
ISSN:2832-7632
DOI:10.1109/ICSE-NIER66352.2025.00016