Including non-randomized studies of interventions in meta-analyses of randomized controlled trials changed the estimates in more than a third of the studies: evidence from an empirical analysis

There is a growing trend to include nonrandomized studies of interventions (NRSIs) in meta-analyses of randomized controlled trials (RCTs) for health decision-making. The study aimed to quantify the impact of integrating NRSI on the evidence derived from RCTs within the same systematic review. We se...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Journal of clinical epidemiology Ročník 183; s. 111815
Hlavní autori: Yao, Minghong, Mei, Fan, Ma, Yu, Qin, Xuan, Huan, Jiayidaer, Zou, Kang, Li, Ling, Sun, Xin
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: United States Elsevier Inc 01.07.2025
Elsevier Limited
Predmet:
Agreements
Clinical trials
Cohort analysis
Consistency
Decision making
Empirical analysis
Estimates
Heterogeneity
Humans
Internal Medicine
Meta-analysis
Meta-Analysis as Topic
Metaepidemiology
Non-Randomized Controlled Trials as Topic - methods
Non-Randomized Controlled Trials as Topic - statistics & numerical data
Nonrandomized studies of interventions
Qualitative analysis
Randomized controlled trials
Randomized Controlled Trials as Topic - statistics & numerical data
Research Design
Statistical analysis
Statistical heterogeneity
Statistics
Subgroups
Systematic review
Systematic Reviews as Topic
Weight
Metaepidemiology
Statistical heterogeneity
Nonrandomized studies of interventions
Consistency
Meta-analysis
Randomized controlled trials
ISSN:0895-4356, 1878-5921, 1878-5921
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:There is a growing trend to include nonrandomized studies of interventions (NRSIs) in meta-analyses of randomized controlled trials (RCTs) for health decision-making. The study aimed to quantify the impact of integrating NRSI on the evidence derived from RCTs within the same systematic review. We searched PubMed for systematic reviews published between December 9, 2017, and December 9, 2022, that included both RCTs and NRSIs under the same outcome. Using the DerSimonian–Laird random-effects model, we reanalyzed the pooled estimates to compare those derived from RCTs with those from combined RCTs and NRSIs. We examined changes in point estimates, subgroup differences, statistical heterogeneity, and the weight of RCTs in pooled estimates. Results were defined as being in qualitative agreement if both estimates demonstrated statistical significance in the same direction or if neither achieved statistical significance. A total of 220 eligible systematic reviews were identified and 217 meta-analyses were reanalyzed. Qualitative disagreement between RCTs only and pooled estimates combining RCTs and NRSIs was observed in 78 meta-analyses (35.9%), of which 69 (88.5%) gained statistical significance after the inclusion of NRSIs. Point estimates in 58 meta-analyses (26.7%) failed to meet predefined agreement criteria, and statistically significant subgroup differences between RCTs and NRSIs were identified in 32 meta-analyses (14.8%). The incorporation of NRSIs raised the heterogeneity from 21.8% to 36.9%, whereas RCTs accounted for a median weight of 33.9% in the pooled estimates. These findings highlight the need for caution in conducting and interpreting meta-analyses combining RCTs and NRSIs, particularly in scenarios where RCTs yield nonsignificant results whereas the inclusion of NRSIs achieves statistical significance. Although randomized controlled trials (RCTs) remain the gold standard for clinical evidence, they are often insufficient to address complex clinical questions. Nonrandomized studies of interventions (NRSIs), leveraging real-world clinical data, are increasingly used to supplement RCT findings. Despite growing interest in integrating NRSIs into meta-analyses with RCTs, the clinical and statistical implications of this approach remain uncertain. To address this gap, we conducted a systematic evaluation of how NRSI inclusion impacts meta-analytic results by analyzing 220 systematic reviews that combined RCTs and NRSIs under the same outcome. Our analysis revealed that incorporating NRSIs altered effect estimates in over one-third of cases, with 88.5% of meta-analyses achieving statistical significance only after NRSI inclusion–a finding with critical implications for decision-making. In addition, NRSI integration elevated statistical heterogeneity, although RCTs accounted for less than one-third of the weight in pooled estimates. These findings collectively underscore the necessity for robust evaluation and cautious interpretation when merging NRSI data with RCTs in meta-analyses. [Display omitted] •Including NRSIs in meta-analyses of RCTs altered the estimates in more than one-third of the studies.•The inclusion of NRSIs increased statistical heterogeneity of the pooled estimates. •The median weight of the RCTs in the pooled estimates was approximately one-third.
Bibliografia:ObjectType-Article-2
SourceType-Scholarly Journals-1
content type line 14
ObjectType-Feature-3
ObjectType-Evidence Based Healthcare-1
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0895-4356
1878-5921
1878-5921
DOI:10.1016/j.jclinepi.2025.111815