The Impact of Access to Clinical Guidelines on LLM-Based Treatment Recommendations for Chronic Hepatitis B.
Uloženo v:
| Název: | The Impact of Access to Clinical Guidelines on LLM-Based Treatment Recommendations for Chronic Hepatitis B. |
|---|---|
| Autoři: | Siepmann R; Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany., Schneider CV; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany., von der Stueck MS; Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany., Amygdalos I; Department of Surgery and Transplantation, University Hospital RWTH Aachen, Aachen, Germany., Große K; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany., Schneider KM; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany., Pollmanns MR; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany., Murad M; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany., Joy J; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany., Kabak E; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany., May MR; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany., Clusmann J; Department of Gastroenterology, Metabolic Diseases and Intensive Care, University Hospital RWTH Aachen, Aachen, Germany.; Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany., Kuhl C; Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany., Nebelung S; Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany., Kather JN; Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany.; Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.; Department of Medicine 1, University Hospital and Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany., Truhn D; Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. |
| Zdroj: | Liver international : official journal of the International Association for the Study of the Liver [Liver Int] 2025 Oct; Vol. 45 (10), pp. e70324. |
| Způsob vydávání: | Journal Article |
| Jazyk: | English |
| Informace o časopise: | Publisher: Wiley-Blackwell Country of Publication: United States NLM ID: 101160857 Publication Model: Print Cited Medium: Internet ISSN: 1478-3231 (Electronic) Linking ISSN: 14783223 NLM ISO Abbreviation: Liver Int Subsets: MEDLINE |
| Imprint Name(s): | Publication: Malden, MA : Wiley-Blackwell Original Publication: Oxford, UK : Blackwell Munksgaard, c2003- |
| Výrazy ze slovníku MeSH: | Hepatitis B, Chronic*/drug therapy , Hepatitis B, Chronic*/therapy , Practice Guidelines as Topic* , Antiviral Agents*/therapeutic use , Language* , Access to Information*, Humans ; World Health Organization ; Female ; Male ; Adult ; Middle Aged |
| Abstrakt: | Background and Aims: Large language models (LLMs) can potentially support clinicians in their daily routine by providing easy access to information. Yet, they are plagued by stating incorrect facts and hallucinating when queried. Increasing the context by providing external databases while prompting LLMs may decrease the risk of misinformation. This study compares the influence of increased context on the coherence of LLM-based treatment recommendations with the recently updated WHO guidelines for the treatment of chronic hepatitis B (CHB). Methods: GPT-4 was queried with five clinical case vignettes in two configurations: with and without additional context. The clinical vignettes were explicitly constructed so that treatment recommendations differed between the formerly applicable 2015 WHO guidelines and the updated 2024 ones. GPT-4 with context was provided access to the updated guidelines, while GPT-4 without context had to rely on its internal knowledge. GPT-4 was accessed only a few days after the release of the new WHO guidelines. Treatment recommendations were compared regarding guideline coherence, information inclusion, textual errors, wording clarity and preciseness by seven physicians. Results: Using GPT-4 with context increased the coherence of the treatment recommendations with the new 2024 guidelines from 51% to 91% compared to GPT-4 without context. Similar trends were observed for all other categories, leading to an increase of 54% in preciseness and clarity, 24% in completeness of incorporating the case vignette information, and 12% in textual correctness. Conclusions: If LLMs are consulted by clinicians for medical advice, they should be given access to external data sources to increase the chance of providing factually correct advice. (© 2025 The Author(s). Liver International published by John Wiley & Sons Ltd.) |
| References: | J Pathol. 2024 Mar;262(3):310-319. (PMID: 38098169) JAMA. 2024 Jan 2;331(1):65-69. (PMID: 38032660) Prostate Cancer Prostatic Dis. 2025 Mar;28(1):229-231. (PMID: 38228809) Nature. 2023 Mar;615(7954):773. (PMID: 36928404) Rofo. 2024 Nov;196(11):1166-1170. (PMID: 38408477) Liver Int. 2024 Sep;44(9):2114-2124. (PMID: 38819632) NPJ Digit Med. 2023 Nov 16;6(1):210. (PMID: 37973919) J Am Med Inform Assoc. 2024 May 20;31(6):1356-1366. (PMID: 38447590) JAMA. 2025 Jan 28;333(4):319-328. (PMID: 39405325) Eur Arch Otorhinolaryngol. 2024 Apr;281(4):2023-2030. (PMID: 38345613) Liver Int. 2025 Oct;45(10):e70324. (PMID: 40891225) Nat Med. 2023 Aug;29(8):1930-1940. (PMID: 37460753) Eur Radiol. 2024 Oct;34(10):6652-6666. (PMID: 38627289) J Med Internet Res. 2023 Jun 28;25:e48568. (PMID: 37379067) Arch Med Res. 2023 Jul;54(5):102835. (PMID: 37248157) NPJ Digit Med. 2024 Apr 23;7(1):102. (PMID: 38654102) Nat Med. 2023 Dec;29(12):2983-2984. (PMID: 37853138) J Nucl Med Technol. 2023 Dec 5;51(4):307-313. (PMID: 37699647) Hepatology. 2024 Nov 1;80(5):1158-1168. (PMID: 38451962) Eur Arch Otorhinolaryngol. 2024 Apr;281(4):2081-2086. (PMID: 37405455) Lancet Digit Health. 2023 Apr;5(4):e179-e181. (PMID: 36894409) Sci Rep. 2023 Nov 17;13(1):20159. (PMID: 37978240) Neuropsychopharmacology. 2024 Aug;49(9):1412-1416. (PMID: 38480911) Medicina (Kaunas). 2024 Mar 08;60(3):. (PMID: 38541171) Clin Gastroenterol Hepatol. 2024 Apr;22(4):886-889.e5. (PMID: 37716618) |
| Grant Information: | 031L0312C German Federal Ministry of Research, Technology and Space; 01KD2420B German Federal Ministry of Research, Technology and Space; TR 1700/7-1 Deutsche Forschungsgemeinschaft; NE2136/7-1 Deutsche Forschungsgemeinschaft; NE2136/3-1 Deutsche Forschungsgemeinschaft; GA101057091 European Union Research and Innovation Programme |
| Contributed Indexing: | Keywords: GPT; LLM; chronic hepatitis B; guideline coherence |
| Substance Nomenclature: | 0 (Antiviral Agents) |
| Entry Date(s): | Date Created: 20250902 Date Completed: 20250902 Latest Revision: 20250905 |
| Update Code: | 20250905 |
| PubMed Central ID: | PMC12402858 |
| DOI: | 10.1111/liv.70324 |
| PMID: | 40891225 |
| Databáze: | MEDLINE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstrakt: | Background and Aims: Large language models (LLMs) can potentially support clinicians in their daily routine by providing easy access to information. Yet, they are plagued by stating incorrect facts and hallucinating when queried. Increasing the context by providing external databases while prompting LLMs may decrease the risk of misinformation. This study compares the influence of increased context on the coherence of LLM-based treatment recommendations with the recently updated WHO guidelines for the treatment of chronic hepatitis B (CHB).<br />Methods: GPT-4 was queried with five clinical case vignettes in two configurations: with and without additional context. The clinical vignettes were explicitly constructed so that treatment recommendations differed between the formerly applicable 2015 WHO guidelines and the updated 2024 ones. GPT-4 with context was provided access to the updated guidelines, while GPT-4 without context had to rely on its internal knowledge. GPT-4 was accessed only a few days after the release of the new WHO guidelines. Treatment recommendations were compared regarding guideline coherence, information inclusion, textual errors, wording clarity and preciseness by seven physicians.<br />Results: Using GPT-4 with context increased the coherence of the treatment recommendations with the new 2024 guidelines from 51% to 91% compared to GPT-4 without context. Similar trends were observed for all other categories, leading to an increase of 54% in preciseness and clarity, 24% in completeness of incorporating the case vignette information, and 12% in textual correctness.<br />Conclusions: If LLMs are consulted by clinicians for medical advice, they should be given access to external data sources to increase the chance of providing factually correct advice.<br /> (© 2025 The Author(s). Liver International published by John Wiley & Sons Ltd.) |
|---|---|
| ISSN: | 1478-3231 |
| DOI: | 10.1111/liv.70324 |