Fast linear model trees by PILOT

Linear model trees are regression trees that incorporate linear models in the leaf nodes. This preserves the intuitive interpretation of decision trees and at the same time enables them to better capture linear relationships, which is hard for standard decision trees. But most existing methods for f...

Full description

Saved in:
Bibliographic Details
Published in:Machine learning Vol. 113; no. 9; pp. 6561 - 6610
Main Authors: Raymaekers, Jakob, Rousseeuw, Peter J., Verdonck, Tim, Yao, Ruicong
Format: Journal Article
Language:English
Published: New York Springer US 01.09.2024
Springer Nature B.V
Subjects:
Artificial Intelligence
Computer Science
Control
Datasets
Decision trees
Greedy algorithms
Machine Learning
Mechatronics
Natural Language Processing (NLP)
Nodes
Polynomials
Regression analysis
Regression models
Robotics
Simulation and Modeling
Regression trees
Scalable algorithms
Consistency
Piecewise linear model
ISSN:0885-6125, 1573-0565
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Linear model trees are regression trees that incorporate linear models in the leaf nodes. This preserves the intuitive interpretation of decision trees and at the same time enables them to better capture linear relationships, which is hard for standard decision trees. But most existing methods for fitting linear model trees are time consuming and therefore not scalable to large data sets. In addition, they are more prone to overfitting and extrapolation issues than standard regression trees. In this paper we introduce PILOT, a new algorithm for linear model trees that is fast, regularized, stable and interpretable. PILOT trains in a greedy fashion like classic regression trees, but incorporates an L 2 boosting approach and a model selection rule for fitting linear models in the nodes. The abbreviation PILOT stands for PIecewise Linear Organic Tree, where ‘organic’ refers to the fact that no pruning is carried out. PILOT has the same low time and space complexity as CART without its pruning. An empirical study indicates that PILOT tends to outperform standard decision trees and other linear model trees on a variety of data sets. Moreover, we prove its consistency in an additive model setting under weak assumptions. When the data is generated by a linear model, the convergence rate is polynomial.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-024-06590-3