Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature

We have empirically assessed the distribution of published effect sizes and estimated power by analyzing 26,841 statistical records from 3,801 cognitive neuroscience and psychology papers published recently. The reported median effect size was D = 0.93 (interquartile range: 0.64-1.46) for nominally...

Full description

Saved in:
Bibliographic Details
Published in:PLoS biology Vol. 15; no. 3; p. e2000797
Main Authors: Szucs, Denes, Ioannidis, John P. A.
Format: Journal Article
Language:English
Published: United States Public Library of Science 02.03.2017
Public Library of Science (PLoS)
Subjects:
Abbreviations
Bayesian analysis
Biology and Life Sciences
Carp
Circularity
Clinical trials
Cognition
Cognition & reasoning
Cognitive ability
Cognitive Neuroscience - statistics & numerical data
Company structure
Computer programs
Correlation analysis
Crises
Data acquisition
Data collection
Data processing
Decision analysis
Degrees of freedom
Drafting software
Ecology
Editing
Empirical Research
Error analysis
Functional magnetic resonance imaging
Funding
Humans
Hypotheses
Impact factors
Incentives
Management
Medical wastes
Meta
Nervous system
Neuroimaging
Neurosciences
Null hypothesis
Personality
Philosophy
Physical Sciences
Physics
Probability
Psychiatry
Psychology
Psychology - statistics & numerical data
Publications - statistics & numerical data
Reliability analysis
Replication
Research and Analysis Methods
Resonance
Reviews
Science
Science publishing
Social interactions
Social Sciences
Software
Spatial distribution
Statistical analysis
Statistical significance
Statistics
Studies
Surveys and Questionnaires
Systems analysis
ISSN:1545-7885, 1544-9173, 1545-7885
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We have empirically assessed the distribution of published effect sizes and estimated power by analyzing 26,841 statistical records from 3,801 cognitive neuroscience and psychology papers published recently. The reported median effect size was D = 0.93 (interquartile range: 0.64-1.46) for nominally statistically significant results and D = 0.24 (0.11-0.42) for nonsignificant results. Median power to detect small, medium, and large effects was 0.12, 0.44, and 0.73, reflecting no improvement through the past half-century. This is so because sample sizes have remained small. Assuming similar true effect sizes in both disciplines, power was lower in cognitive neuroscience than in psychology. Journal impact factors negatively correlated with power. Assuming a realistic range of prior probabilities for null hypotheses, false report probability is likely to exceed 50% for the whole literature. In light of our findings, the recently reported low replication success in psychology is realistic, and worse performance may be expected for cognitive neuroscience.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
The authors have declared that no competing interests exist.
ISSN:1545-7885
1544-9173
1545-7885
DOI:10.1371/journal.pbio.2000797