HILL: A Hallucination Identifier for Large Language Models
Gespeichert in:
| Titel: | HILL: A Hallucination Identifier for Large Language Models |
|---|---|
| Autoren: | Leiser, Florian, Eckhardt, Sven, Leuthe, Valentin, Knaeble, Merlin, Mädche, Alexander, Schwabe, Gerhard, Sunyaev, Ali |
| Weitere Verfasser: | University of Zurich |
| Quelle: | Proceedings of the CHI Conference on Human Factors in Computing Systems. :1-13 |
| Publication Status: | Preprint |
| Verlagsinformationen: | ACM, 2024. |
| Publikationsjahr: | 2024 |
| Schlagwörter: | FOS: Computer and information sciences, Interaction, 10009 Department of Informatics, ddc:330, Economics, Computer Science - Human-Computer Interaction, (cs.HC), 000 Computer science, knowledge & systems, 1704 Computer Graphics and Computer-Aided Design, Human-Computer Interaction (cs.HC), 1712 Software, 1709 Human-Computer Interaction, Human-Computer |
| Beschreibung: | Large language models (LLMs) are prone to hallucinations, i.e., nonsensical, unfaithful, and undesirable text. Users tend to overrely on LLMs and corresponding hallucinations which can lead to misinterpretations and errors. To tackle the problem of overreliance, we propose HILL, the "Hallucination Identifier for Large Language Models". First, we identified design features for HILL with a Wizard of Oz approach with nine participants. Subsequently, we implemented HILL based on the identified design features and evaluated HILL's interface design by surveying 17 participants. Further, we investigated HILL's functionality to identify hallucinations based on an existing question-answering dataset and five user interviews. We find that HILL can correctly identify and highlight hallucinations in LLM responses which enables users to handle LLM responses with more caution. With that, we propose an easy-to-implement adaptation to existing LLMs and demonstrate the relevance of user-centered designs of AI artifacts. |
| Publikationsart: | Article Report Other literature type Conference object Book |
| Dateibeschreibung: | application/pdf; 3613904.3642428.pdf - application/pdf |
| DOI: | 10.1145/3613904.3642428 |
| DOI: | 10.5445/ir/1000170638 |
| DOI: | 10.5445/ir/1000169738 |
| DOI: | 10.48550/arxiv.2403.06710 |
| DOI: | 10.5167/uzh-264258 |
| Zugangs-URL: | http://arxiv.org/abs/2403.06710 https://publikationen.bibliothek.kit.edu/1000170638 https://doi.org/10.5445/IR/1000170638 https://publikationen.bibliothek.kit.edu/1000170638/152854120 https://publikationen.bibliothek.kit.edu/1000169738 https://doi.org/10.5445/IR/1000169738 https://publikationen.bibliothek.kit.edu/1000169738/152585658 https://www.zora.uzh.ch/id/eprint/264258/ https://doi.org/10.5167/uzh-264258 |
| Rights: | CC BY SA CC BY |
| Dokumentencode: | edsair.doi.dedup.....e77ca671f686afaf19cb77ce6343a5a6 |
| Datenbank: | OpenAIRE |
Nájsť tento článok vo Web of Science | Abstract: | Large language models (LLMs) are prone to hallucinations, i.e., nonsensical, unfaithful, and undesirable text. Users tend to overrely on LLMs and corresponding hallucinations which can lead to misinterpretations and errors. To tackle the problem of overreliance, we propose HILL, the "Hallucination Identifier for Large Language Models". First, we identified design features for HILL with a Wizard of Oz approach with nine participants. Subsequently, we implemented HILL based on the identified design features and evaluated HILL's interface design by surveying 17 participants. Further, we investigated HILL's functionality to identify hallucinations based on an existing question-answering dataset and five user interviews. We find that HILL can correctly identify and highlight hallucinations in LLM responses which enables users to handle LLM responses with more caution. With that, we propose an easy-to-implement adaptation to existing LLMs and demonstrate the relevance of user-centered designs of AI artifacts. |
|---|---|
| DOI: | 10.1145/3613904.3642428 |