Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial

To determine the accuracy of single-reviewer screening in correctly classifying abstracts as relevant or irrelevant for literature reviews. We conducted a crowd-based, parallel-group randomized controlled trial. Using the Cochrane Crowd platform, we randomly assigned eligible participants to 100 abs...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Journal of clinical epidemiology Ročník 121; s. 20 - 28
Hlavní autori: Gartlehner, Gerald, Affengruber, Lisa, Titscher, Viktoria, Noel-Storr, Anna, Dooley, Gordon, Ballarini, Nicolas, König, Franz
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: United States Elsevier Inc 01.05.2020
Elsevier Limited
Predmet:
Abstracting and Indexing - standards
Abstracting and Indexing - statistics & numerical data
Accuracy
Adult
Bibliographic data bases
Clinical trials
Confidence intervals
Data Accuracy
Decision making
Depression - therapy
Epidemiology
Female
Humans
Internal Medicine
Literature reviews
Literature screening
Male
Peer Review, Research - standards
Public health
Quality standards
Random Allocation
Randomization
Randomized controlled trial
Rapid reviews
Regression Analysis
Sample Size
Screening
Sensitivity
Sensitivity and Specificity
Skills
Standard deviation
Studies
Sugar-Sweetened Beverages
Systematic review
Systematic reviews
Systematic Reviews as Topic
United States > US
Literature screening
Systematic reviews
Rapid reviews
Accuracy
Randomized controlled trial
rapid reviews
accuracy
systematic reviews
literature screening
ISSN:0895-4356, 1878-5921, 1878-5921
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:To determine the accuracy of single-reviewer screening in correctly classifying abstracts as relevant or irrelevant for literature reviews. We conducted a crowd-based, parallel-group randomized controlled trial. Using the Cochrane Crowd platform, we randomly assigned eligible participants to 100 abstracts each of a pharmacological or a public health topic. After completing a training exercise, participants screened abstracts online based on predefined inclusion and exclusion criteria. We calculated sensitivities and specificities of single- and dual-reviewer screening using two published systematic reviews as reference standards. Two hundred and eighty participants made 24,942 screening decisions on 2,000 randomly selected abstracts from the reference standard reviews. On average, each abstract was screened 12 times. Overall, single-reviewer abstract screening missed 13% of relevant studies (sensitivity: 86.6%; 95% confidence interval [CI], 80.6%–91.2%). By comparison, dual-reviewer abstract screening missed 3% of relevant studies (sensitivity: 97.5%; 95% CI, 95.1%–98.8%). The corresponding specificities were 79.2% (95% CI, 77.4%–80.9%) and 68.7% (95% CI, 66.4%–71.0%), respectively. Single-reviewer abstract screening does not appear to fulfill the high methodological standards that decisionmakers expect from systematic reviews. It may be a viable option for rapid reviews, which deliberately lower methodological standards to provide decision makers with accelerated evidence synthesis products.
Bibliografia:ObjectType-Article-2
SourceType-Scholarly Journals-1
content type line 14
ObjectType-Feature-3
ObjectType-Evidence Based Healthcare-1
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0895-4356
1878-5921
1878-5921
DOI:10.1016/j.jclinepi.2020.01.005