View in EDS

The P Value: What It Is and What It Is Not.

Saved in:
Bibliographic Details
Title: The P Value: What It Is and What It Is Not.
Authors: Habibzadeh F; Independent Research Consultant, Shiraz, Iran. Farrokh.Habibzadeh@gmail.com.
Source: Journal of Korean medical science [J Korean Med Sci] 2025 Nov 17; Vol. 40 (44), pp. e321. Date of Electronic Publication: 2025 Nov 17.
Publication Type: Journal Article; Review
Language: English
Journal Info: Publisher: Korean Academy of Medical Science Country of Publication: Korea (South) NLM ID: 8703518 Publication Model: Electronic Cited Medium: Internet ISSN: 1598-6357 (Electronic) Linking ISSN: 10118934 NLM ISO Abbreviation: J Korean Med Sci Subsets: MEDLINE
Imprint Name(s): Original Publication: Seoul, Korea : Korean Academy of Medical Science, [1986-
MeSH Terms: Data Interpretation, Statistical*, Bayes Theorem ; Confidence Intervals ; Likelihood Functions ; Probability ; Reproducibility of Results
Abstract: Competing Interests: The author has no potential conflicts of interest to disclose.
The P value remains one of the most frequently reported statistical measures in biomedical literature, yet it is also one of the most widely misunderstood statistics. Introduced by Fisher as a measure of evidence against the null hypothesis, it was subsequently incorporated into the Neyman-Pearson decision framework, which emphasized long-run error rates and decision thresholds. Over the past decades, reliance on the conventional cut-off of P = 0.05 has fostered misconceptions, including the belief that the P value represents the probability that the null hypothesis is true or that statistical significance implies clinical importance. In this review, I examine the historical evolution of the P value, clarify the conceptual distinctions between evidential and decision-theoretic perspectives, and illustrate their implications through a case study. Common misinterpretations and the limitations of threshold-based inference are discussed, together with the consequences for reproducibility, statistical power, and interpretation of results. Recent recommendations from statistical associations and methodologists to move beyond dichotomous significance testing are highlighted. Complementary approaches, such as estimation of effect sizes with confidence intervals (CIs), likelihood ratios, and Bayesian inference, are briefly considered. I conclude that although the P value may provide useful information when properly interpreted, it should not be used as a sole criterion for inference. Transparent reporting of effect sizes, CIs, and contextual information offers a more reliable foundation for scientific interpretation and decision making.
(© 2025 The Korean Academy of Medical Sciences.)
References: J Pers Soc Psychol. 2011 Mar;100(3):407-25. (PMID: 21280961)
Psychophysiology. 1996 Mar;33(2):175-83. (PMID: 8851245)
Am J Epidemiol. 1993 Mar 1;137(5):485-96; discussion 497-501. (PMID: 8465801)
Biochem Med (Zagreb). 2019 Jun 15;29(2):020101. (PMID: 31015780)
Psychol Sci. 2010 Oct;21(10):1363-8. (PMID: 20855902)
JAMA. 2018 Apr 10;319(14):1429-1430. (PMID: 29566133)
BMJ Evid Based Med. 2021 Apr;26(2):39-42. (PMID: 31732498)
Ann Intern Med. 1999 Jun 15;130(12):995-1004. (PMID: 10383371)
Nat Hum Behav. 2018 Jan;2(1):6-10. (PMID: 30980045)
PLoS One. 2025 Jun 13;20(6):e0325920. (PMID: 40512828)
R Soc Open Sci. 2016 Sep 21;3(9):160384. (PMID: 27703703)
PLoS Med. 2005 Aug;2(8):e124. (PMID: 16060722)
Eur J Epidemiol. 2016 Apr;31(4):337-50. (PMID: 27209009)
J Transl Med. 2024 Jan 4;22(1):16. (PMID: 38178182)
Int J Nurs Stud. 2015 Jan;52(1):5-9. (PMID: 25441757)
Rheumatol Int. 2021 Jan;41(1):43-55. (PMID: 33201265)
Proc Natl Acad Sci U S A. 2013 Nov 26;110(48):19313-7. (PMID: 24218581)
PLoS One. 2024 Jun 14;19(6):e0305575. (PMID: 38875254)
J Korean Med Sci. 2024 Jun 03;39(21):e177. (PMID: 38832479)
JAMA. 2016 Mar 15;315(11):1141-8. (PMID: 26978209)
J Korean Med Sci. 2017 Jul;32(7):1072-1076. (PMID: 28581261)
Contributed Indexing: Keywords: Biostatistics; Confidence Intervals; Likelihood Functions; P Value; Publication Bias; Statistics as Topic
Entry Date(s): Date Created: 20251118 Date Completed: 20251118 Latest Revision: 20251123
Update Code: 20251123
PubMed Central ID: PMC12624209
DOI: 10.3346/jkms.2025.40.e321
PMID: 41250654
Database: MEDLINE
Description
Abstract:Competing Interests: The author has no potential conflicts of interest to disclose.<br />The P value remains one of the most frequently reported statistical measures in biomedical literature, yet it is also one of the most widely misunderstood statistics. Introduced by Fisher as a measure of evidence against the null hypothesis, it was subsequently incorporated into the Neyman-Pearson decision framework, which emphasized long-run error rates and decision thresholds. Over the past decades, reliance on the conventional cut-off of P = 0.05 has fostered misconceptions, including the belief that the P value represents the probability that the null hypothesis is true or that statistical significance implies clinical importance. In this review, I examine the historical evolution of the P value, clarify the conceptual distinctions between evidential and decision-theoretic perspectives, and illustrate their implications through a case study. Common misinterpretations and the limitations of threshold-based inference are discussed, together with the consequences for reproducibility, statistical power, and interpretation of results. Recent recommendations from statistical associations and methodologists to move beyond dichotomous significance testing are highlighted. Complementary approaches, such as estimation of effect sizes with confidence intervals (CIs), likelihood ratios, and Bayesian inference, are briefly considered. I conclude that although the P value may provide useful information when properly interpreted, it should not be used as a sole criterion for inference. Transparent reporting of effect sizes, CIs, and contextual information offers a more reliable foundation for scientific interpretation and decision making.<br /> (© 2025 The Korean Academy of Medical Sciences.)
ISSN:1598-6357
DOI:10.3346/jkms.2025.40.e321