Selective Inference for Hierarchical Clustering

Classical tests for a difference in means control the Type I error rate when the groups are defined a priori. However, when the groups are instead defined via clustering, then applying a classical test yields an extremely inflated Type I error rate. Notably, this problem persists even if two separat...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of the American Statistical Association Jg. 119; H. 545; S. 332 - 342
Hauptverfasser: Gao, Lucy L., Bien, Jacob, Witten, Daniela
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States Taylor & Francis 02.01.2024
Taylor & Francis Ltd
Schlagworte:
Americans
Cluster analysis
Clustering
Data
data collection
Difference in means
Errors
Gene sequencing
Hierarchies
Hypothesis testing
Inference
Null hypothesis
Post-selection inference
sequence analysis
Statistics
testing
Tests
Type I error
post-selection inference
hypothesis testing
difference in means
type I error
ISSN:0162-1459, 1537-274X, 1537-274X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Classical tests for a difference in means control the Type I error rate when the groups are defined a priori. However, when the groups are instead defined via clustering, then applying a classical test yields an extremely inflated Type I error rate. Notably, this problem persists even if two separate and independent datasets are used to define the groups and to test for a difference in their means. To address this problem, in this article, we propose a selective inference approach to test for a difference in means between two clusters. Our procedure controls the selective Type I error rate by accounting for the fact that the choice of null hypothesis was made based on the data. We describe how to efficiently compute exact p-values for clusters obtained using agglomerative hierarchical clustering with many commonly used linkages. We apply our method to simulated data and to single-cell RNA-sequencing data. Supplementary materials for this article are available online.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0162-1459
1537-274X
1537-274X
DOI:10.1080/01621459.2022.2116331