Improving large language models for miRNA information extraction via prompt engineering.
Saved in:
| Title: | Improving large language models for miRNA information extraction via prompt engineering. |
|---|---|
| Authors: | Wu R; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China; Operation Management Department, The First Affiliated Hospital of Soochow University, Suzhou, China., Zong H; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China., Wu E; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China; Department of Neurosurgery, the First Affiliated Hospital of Xinjiang Medical University, Urumqi, China., Li J; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China., Zhou Y; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China., Zhang C; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China; Joint Laboratory of Artificial Intelligence for Critical Care Medicine, Department of Critical Care Medicine, West China Hospital, Sichuan University, Chengdu, China., Zhang Y; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China; Tropical Crops Genetic Resources Institute, Chinese Academy of Tropical Agricultural Sciences, Haikou, China., Wang J; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China; Department of Computer Science and Information Technologies, Elviña Campus, University of A Coruña, A Coruña, Spain., Tang T; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China; Department of Computer Science and Information Technologies, Elviña Campus, University of A Coruña, A Coruña, Spain., Shen B; Department of Urology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China; West China Tianfu Hospital, Sichuan University, Chengdu, Sichuan, China. Electronic address: bairong.shen@scu.edu.cn. |
| Source: | Computer methods and programs in biomedicine [Comput Methods Programs Biomed] 2025 Nov; Vol. 271, pp. 109033. Date of Electronic Publication: 2025 Aug 19. |
| Publication Type: | Journal Article |
| Language: | English |
| Journal Info: | Publisher: Elsevier Scientific Publishers Country of Publication: Ireland NLM ID: 8506513 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1872-7565 (Electronic) Linking ISSN: 01692607 NLM ISO Abbreviation: Comput Methods Programs Biomed Subsets: MEDLINE |
| Imprint Name(s): | Publication: Limerick : Elsevier Scientific Publishers Original Publication: Amsterdam : Elsevier Science Publishers, c1984- |
| MeSH Terms: | MicroRNAs*/genetics , Language* , Natural Language Processing*, Humans ; Computational Biology/methods ; Reproducibility of Results ; Algorithms ; Large Language Models |
| Abstract: | Competing Interests: Declaration of competing interest The authors declare that they have no competing interests. Objective: Large language models (LLMs) demonstrate significant potential in biomedical knowledge discovery, yet their performance in extracting fine-grained biological information, such as miRNA, remains insufficiently explored. Accurate extraction of miRNA-related information is essential for understanding disease mechanisms and identifying biomarkers. This study aims to comprehensively evaluate the capabilities of LLMs in miRNA information extraction through diverse prompt learning strategies. Methods: Three high-quality miRNA information extraction datasets were constructed to support the benchmarking and training of generative LLMs, specifically Re-Tex, Re-miR and miR-Cancer. These datasets encompass three types of entities: miRNAs, genes, and diseases, along with their relationships. The accuracy and reliability of three LLMs, including GPT-4o, Gemini, and Claude, were evaluated and compared with traditional models. Different prompt engineering strategies were implemented to enhance the LLMs' performance, including baseline prompts, 5-shot Chain of Thought prompts, and generated knowledge prompts. Results: The combination of optimized prompt strategies significantly improved overall entity extraction performance across both trained and untrained datasets. Generated knowledge prompting achieved the highest performance, with maximum F1 scores of 76.6 % for entity extraction and 54.8 % for relationship extraction. Comparative analysis indicated GPT-4o exhibited superior performance to Gemini, while Claude showed the lowest performance levels. Extraction accuracy varied considerably across entity types, with miRNA recognition achieving the highest performance and gene/protein identification demonstrating the lowest accuracy levels. Furthermore, binary relationship extraction accuracy was significantly lower than entity extraction performance. The three evaluated LLMs showed similarly limited capability in relationship extraction tasks, with no statistically significant differences observed between models. Finally, comparison with conventional computational methods revealed LLMs have not yet exceeded traditional methods in this specialized domain. Conclusion: This study established high-quality miRNA datasets to support information extraction and knowledge discovery. The overall performance of LLMs in this study proved limited, and challenges remain in processing miRNA-related information extraction. However, optimized prompt combinations can substantially improve performance. Future work should focus on further refinement of LLMs to accelerate the discovery and application of potential diagnostic and therapeutic targets. (Copyright © 2025 The Authors. Published by Elsevier B.V. All rights reserved.) |
| Contributed Indexing: | Keywords: Cancer; Datasets; Information extraction; Large language models; MicroRNA; Prompt engineering |
| Substance Nomenclature: | 0 (MicroRNAs) |
| Entry Date(s): | Date Created: 20250828 Date Completed: 20250906 Latest Revision: 20250906 |
| Update Code: | 20250907 |
| DOI: | 10.1016/j.cmpb.2025.109033 |
| PMID: | 40876082 |
| Database: | MEDLINE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstract: | Competing Interests: Declaration of competing interest The authors declare that they have no competing interests.<br />Objective: Large language models (LLMs) demonstrate significant potential in biomedical knowledge discovery, yet their performance in extracting fine-grained biological information, such as miRNA, remains insufficiently explored. Accurate extraction of miRNA-related information is essential for understanding disease mechanisms and identifying biomarkers. This study aims to comprehensively evaluate the capabilities of LLMs in miRNA information extraction through diverse prompt learning strategies.<br />Methods: Three high-quality miRNA information extraction datasets were constructed to support the benchmarking and training of generative LLMs, specifically Re-Tex, Re-miR and miR-Cancer. These datasets encompass three types of entities: miRNAs, genes, and diseases, along with their relationships. The accuracy and reliability of three LLMs, including GPT-4o, Gemini, and Claude, were evaluated and compared with traditional models. Different prompt engineering strategies were implemented to enhance the LLMs' performance, including baseline prompts, 5-shot Chain of Thought prompts, and generated knowledge prompts.<br />Results: The combination of optimized prompt strategies significantly improved overall entity extraction performance across both trained and untrained datasets. Generated knowledge prompting achieved the highest performance, with maximum F1 scores of 76.6 % for entity extraction and 54.8 % for relationship extraction. Comparative analysis indicated GPT-4o exhibited superior performance to Gemini, while Claude showed the lowest performance levels. Extraction accuracy varied considerably across entity types, with miRNA recognition achieving the highest performance and gene/protein identification demonstrating the lowest accuracy levels. Furthermore, binary relationship extraction accuracy was significantly lower than entity extraction performance. The three evaluated LLMs showed similarly limited capability in relationship extraction tasks, with no statistically significant differences observed between models. Finally, comparison with conventional computational methods revealed LLMs have not yet exceeded traditional methods in this specialized domain.<br />Conclusion: This study established high-quality miRNA datasets to support information extraction and knowledge discovery. The overall performance of LLMs in this study proved limited, and challenges remain in processing miRNA-related information extraction. However, optimized prompt combinations can substantially improve performance. Future work should focus on further refinement of LLMs to accelerate the discovery and application of potential diagnostic and therapeutic targets.<br /> (Copyright © 2025 The Authors. Published by Elsevier B.V. All rights reserved.) |
|---|---|
| ISSN: | 1872-7565 |
| DOI: | 10.1016/j.cmpb.2025.109033 |