Learning on knowledge graph dynamics provides an early warning of impactful research

The scientific ecosystem relies on citation-based metrics that provide only imperfect, inconsistent and easily manipulated measures of research quality. Here we describe DELPHI (Dynamic Early-warning by Learning to Predict High Impact), a framework that provides an early-warning signal for ‘impactfu...

Full description

Saved in:
Bibliographic Details
Published in:Nature biotechnology Vol. 39; no. 10; pp. 1300 - 1307
Main Authors: Weis, James W., Jacobson, Joseph M.
Format: Journal Article
Language:English
Published: New York Nature Publishing Group US 01.10.2021
Nature Publishing Group
Subjects:
631/114/1305
631/114/2406
706/648/1496
Agriculture
Analysis
Artificial intelligence
Bibliometrics
Bioinformatics
Biomedical and Life Sciences
Biomedical Engineering/Biotechnology
Biomedicine
Biotechnology
Datasets
Funding
Graph theory
Knowledge representation
Learning algorithms
Life Sciences
Machine learning
Mathematical analysis
Scientific papers
Warning systems
United States
ISSN:1087-0156, 1546-1696, 1546-1696
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The scientific ecosystem relies on citation-based metrics that provide only imperfect, inconsistent and easily manipulated measures of research quality. Here we describe DELPHI (Dynamic Early-warning by Learning to Predict High Impact), a framework that provides an early-warning signal for ‘impactful’ research by autonomously learning high-dimensional relationships among features calculated across time from the scientific literature. We prototype this framework and deduce its performance and scaling properties on time-structured publication graphs from 1980 to 2019 drawn from 42 biotechnology-related journals, including over 7.8 million individual nodes, 201 million relationships and 3.8 billion calculated metrics. We demonstrate the framework’s performance by correctly identifying 19/20 seminal biotechnologies from 1980 to 2014 via a blinded retrospective study and provide 50 research papers from 2018 that DELPHI predicts will be in the top 5% of time-rescaled node centrality in the future. We propose DELPHI as a tool to aid in the construction of diversified, impact-optimized funding portfolios. Biotechnology-related papers predicted to be of long-term impact are identified in a machine learning framework (DELPHI) that analyzes relationships among a range of features from the scientific literature over time.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1087-0156
1546-1696
1546-1696
DOI:10.1038/s41587-021-00907-6