Unbiased variable importance for random forests

The default variable-importance measure in random forests, Gini importance, has been shown to suffer from the bias of the underlying Gini-gain splitting criterion. While the alternative permutation importance is generally accepted as a reliable measure of variable importance, it is also computationa...

Full description

Saved in:
Bibliographic Details
Published in:Communications in statistics. Theory and methods Vol. 51; no. 5; pp. 1413 - 1425
Main Author: Loecher, Markus
Format: Journal Article
Language:English
Published: Taylor & Francis 04.03.2022
Subjects:
Gini impurity
random forests
trees
Variable importance
ISSN:0361-0926, 1532-415X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The default variable-importance measure in random forests, Gini importance, has been shown to suffer from the bias of the underlying Gini-gain splitting criterion. While the alternative permutation importance is generally accepted as a reliable measure of variable importance, it is also computationally demanding and suffers from other shortcomings. We propose a simple solution to the misleading/untrustworthy Gini importance which can be viewed as an over-fitting problem: we compute the loss reduction on the out-of-bag instead of the in-bag training samples.
ISSN:0361-0926
1532-415X
DOI:10.1080/03610926.2020.1764042