Unbiased variable importance for random forests

The default variable-importance measure in random forests, Gini importance, has been shown to suffer from the bias of the underlying Gini-gain splitting criterion. While the alternative permutation importance is generally accepted as a reliable measure of variable importance, it is also computationa...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Communications in statistics. Theory and methods Jg. 51; H. 5; S. 1413 - 1425
1. Verfasser: Loecher, Markus
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Taylor & Francis 04.03.2022
Schlagworte:
Gini impurity
random forests
trees
Variable importance
ISSN:0361-0926, 1532-415X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The default variable-importance measure in random forests, Gini importance, has been shown to suffer from the bias of the underlying Gini-gain splitting criterion. While the alternative permutation importance is generally accepted as a reliable measure of variable importance, it is also computationally demanding and suffers from other shortcomings. We propose a simple solution to the misleading/untrustworthy Gini importance which can be viewed as an over-fitting problem: we compute the loss reduction on the out-of-bag instead of the in-bag training samples.
ISSN:0361-0926
1532-415X
DOI:10.1080/03610926.2020.1764042