GPT meets PubMed: a novel approach to literature review using a large language model to crowdsource migraine medication reviews

Objective To evaluate the potential of two large language models (LLMs), GPT-4 (OpenAI) and PaLM2 (Google), in automating migraine literature analysis by conducting sentiment analysis of migraine medications in clinical trial abstracts. Background Migraine affects over one billion individuals worldw...

Full description

Saved in:
Bibliographic Details
Published in:BMC neurology Vol. 25; no. 1; pp. 69 - 8
Main Authors: Mackenzie, Elyse, Cheng, Roger, Zhang, Pengfei
Format: Journal Article
Language:English
Published: London BioMed Central 19.02.2025
BioMed Central Ltd
Springer Nature B.V
BMC
Subjects:
Application programming interface
Artificial intelligence
Clinical trials
Crowdsourcing
Crowdsourcing - methods
Drug therapy
FDA approval
Headache
Headaches
Humans
Large language model
Large Language Models
Literature review
Literature reviews
Medical Subject Headings-MeSH
Medicine
Medicine & Public Health
Migraine
Migraine Disorders - drug therapy
Neurochemistry
Neurology
Neurosurgery
Pharmaceuticals
PubMed
Python
Quality of life
Sentiment analysis
Technology application
United States
Headaches
Literature review
Large language model
Artificial intelligence
Migraine
ISSN:1471-2377, 1471-2377
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Objective To evaluate the potential of two large language models (LLMs), GPT-4 (OpenAI) and PaLM2 (Google), in automating migraine literature analysis by conducting sentiment analysis of migraine medications in clinical trial abstracts. Background Migraine affects over one billion individuals worldwide, significantly impacting their quality of life. A vast amount of scientific literature on novel migraine therapeutics continues to emerge, but an efficient method by which to perform ongoing analysis and integration of this information poses a challenge. Methods “Sentiment analysis” is a data science technique used to ascertain whether a text has positive, negative, or neutral emotional tone. Migraine medication names were extracted from lists of licensed biological products from the FDA, and relevant abstracts were identified using the MeSH term “migraine disorders” on PubMed and filtered for clinical trials. Standardized prompts were provided to the APIs of both GPT-4 and PaLM2 to request an article sentiment as to the efficacy of each medication found in the abstract text. The resulting sentiment outputs were classified using both a binary and a distribution-based model to determine the efficacy of a given medication. Results In both the binary and distribution-based models, the most favorable migraine medications identified by GPT-4 and PaLM2 aligned with evidence-based guidelines for migraine treatment. Conclusions LLMs have potential as complementary tools in migraine literature analysis. Despite some inconsistencies in output and methodological limitations, the results highlight the utility of LLMs in enhancing the efficiency of literature review through sentiment analysis.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1471-2377
1471-2377
DOI:10.1186/s12883-025-04071-1