Lack of methodological rigor and limited coverage of generative artificial intelligence in existing artificial intelligence reporting guidelines: a scoping review.
Gespeichert in:
| Titel: | Lack of methodological rigor and limited coverage of generative artificial intelligence in existing artificial intelligence reporting guidelines: a scoping review. |
|---|---|
| Autoren: | Luo X; Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; World Health Organization Collaboration Center for Guideline Implementation and Knowledge Translation, Lanzhou, China; Institute of Health Data Science, Lanzhou University, Lanzhou, China; Key Laboratory of Evidence Based Medicine of Gansu Province, Lanzhou University, Lanzhou, China., Wang B; Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; World Health Organization Collaboration Center for Guideline Implementation and Knowledge Translation, Lanzhou, China; Institute of Health Data Science, Lanzhou University, Lanzhou, China; Key Laboratory of Evidence Based Medicine of Gansu Province, Lanzhou University, Lanzhou, China., Shi Q; The First School of Clinical Medicine, Lanzhou University, Lanzhou, China., Wang Z; Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; World Health Organization Collaboration Center for Guideline Implementation and Knowledge Translation, Lanzhou, China; Institute of Health Data Science, Lanzhou University, Lanzhou, China; Key Laboratory of Evidence Based Medicine of Gansu Province, Lanzhou University, Lanzhou, China., Lai H; Department of Health Policy and Health Management, School of Public Health, Lanzhou University, Lanzhou, China; Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China., Liu H; Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; World Health Organization Collaboration Center for Guideline Implementation and Knowledge Translation, Lanzhou, China; Institute of Health Data Science, Lanzhou University, Lanzhou, China; Key Laboratory of Evidence Based Medicine of Gansu Province, Lanzhou University, Lanzhou, China., Qin Y; Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; World Health Organization Collaboration Center for Guideline Implementation and Knowledge Translation, Lanzhou, China; Institute of Health Data Science, Lanzhou University, Lanzhou, China; Key Laboratory of Evidence Based Medicine of Gansu Province, Lanzhou University, Lanzhou, China., Chen F; School of Information Science & Engineering, Lanzhou University, Lanzhou, China., Song X; Department of Health Policy and Health Management, School of Public Health, Lanzhou University, Lanzhou, China; Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China., Ge L; Department of Health Policy and Health Management, School of Public Health, Lanzhou University, Lanzhou, China; Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China., Zhang L; Department of Computer Science, Hong Kong Baptist University, Hongkong, China., Bian Z; Vincent V.C. Woo Chinese Medicine Clinical Research Institute, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong, China; Chinese EQUATOR Centre, Hong Kong, China., Chen Y; Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou, China; World Health Organization Collaboration Center for Guideline Implementation and Knowledge Translation, Lanzhou, China; Institute of Health Data Science, Lanzhou University, Lanzhou, China; Key Laboratory of Evidence Based Medicine of Gansu Province, Lanzhou University, Lanzhou, China. Electronic address: chevidence@lzu.edu.cn. |
| Körperschaften: | ADVANCED working group |
| Quelle: | Journal of clinical epidemiology [J Clin Epidemiol] 2025 Oct; Vol. 186, pp. 111903. Date of Electronic Publication: 2025 Jul 18. |
| Publikationsart: | Journal Article; Scoping Review |
| Sprache: | English |
| Info zur Zeitschrift: | Publisher: Elsevier Country of Publication: United States NLM ID: 8801383 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1878-5921 (Electronic) Linking ISSN: 08954356 NLM ISO Abbreviation: J Clin Epidemiol Subsets: MEDLINE |
| Imprint Name(s): | Publication: New York : Elsevier Original Publication: Oxford ; New York : Pergamon Press, c1988- |
| MeSH-Schlagworte: | Artificial Intelligence*/standards , Guidelines as Topic*/standards , Research Design*/standards, Humans ; Generative Artificial Intelligence |
| Abstract: | Competing Interests: Declaration of competing interest There are no competing interests for any authors. Objectives: This study aimed to systematically map the development methods, scope, and limitations of existing artificial intelligence (AI) reporting guidelines in medicine and to explore their applicability to generative AI (GAI) tools, such as large language models (LLMs). Study Design and Setting: We reported a scoping review adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews. Five information sources were searched, including MEDLINE (via PubMed), Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network, China National Knowledge Infrastructure, FAIRsharing, and Google Scholar, from inception to December 31, 2024. Two reviewers independently screened records and extracted data using a predefined Excel template. Data included guideline characteristics (eg, development methods, target audience, AI domain), adherence to EQUATOR Network recommendations, and consensus methodologies. Discrepancies were resolved by a third reviewer. Results: Sixty-eight AI reporting guidelines were included; 48.5% focused on general AI, whereas only 7.4% addressed GAI/LLMs. Methodological rigor was limited; 39.7% described development processes, 42.6% involved multidisciplinary experts, and 33.8% followed EQUATOR recommendations. Significant overlap existed, particularly in medical imaging (20.6% of guidelines). GAI-specific guidelines (14.7%) lacked comprehensive coverage and methodological transparency. Conclusion: Existing AI reporting guidelines in medicine have suboptimal methodological rigor, redundancy, and insufficient coverage of GAI applications. Future and updated guidelines should prioritize standardized development processes, multidisciplinary collaboration, and expanded focus on emerging AI technologies like LLMs. (Copyright © 2025 Elsevier Inc. All rights reserved.) |
| Contributed Indexing: | Investigator: H He; Y Wang; H Li; H Zhang; D Zhu; Y Yao; D Peng; Z Li; J Zhang; Y Qin; F Wang; Z Tang; Y Li; H Liu; J Zhao Keywords: Artificial intelligence; Generative artificial intelligence; Large language models; Methodological quality; Reporting guidelines; Scoping review |
| Entry Date(s): | Date Created: 20250720 Date Completed: 20251017 Latest Revision: 20251017 |
| Update Code: | 20251018 |
| DOI: | 10.1016/j.jclinepi.2025.111903 |
| PMID: | 40684889 |
| Datenbank: | MEDLINE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstract: | Competing Interests: Declaration of competing interest There are no competing interests for any authors.<br />Objectives: This study aimed to systematically map the development methods, scope, and limitations of existing artificial intelligence (AI) reporting guidelines in medicine and to explore their applicability to generative AI (GAI) tools, such as large language models (LLMs).<br />Study Design and Setting: We reported a scoping review adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews. Five information sources were searched, including MEDLINE (via PubMed), Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network, China National Knowledge Infrastructure, FAIRsharing, and Google Scholar, from inception to December 31, 2024. Two reviewers independently screened records and extracted data using a predefined Excel template. Data included guideline characteristics (eg, development methods, target audience, AI domain), adherence to EQUATOR Network recommendations, and consensus methodologies. Discrepancies were resolved by a third reviewer.<br />Results: Sixty-eight AI reporting guidelines were included; 48.5% focused on general AI, whereas only 7.4% addressed GAI/LLMs. Methodological rigor was limited; 39.7% described development processes, 42.6% involved multidisciplinary experts, and 33.8% followed EQUATOR recommendations. Significant overlap existed, particularly in medical imaging (20.6% of guidelines). GAI-specific guidelines (14.7%) lacked comprehensive coverage and methodological transparency.<br />Conclusion: Existing AI reporting guidelines in medicine have suboptimal methodological rigor, redundancy, and insufficient coverage of GAI applications. Future and updated guidelines should prioritize standardized development processes, multidisciplinary collaboration, and expanded focus on emerging AI technologies like LLMs.<br /> (Copyright © 2025 Elsevier Inc. All rights reserved.) |
|---|---|
| ISSN: | 1878-5921 |
| DOI: | 10.1016/j.jclinepi.2025.111903 |