New optimization models for optimal classification trees

The most efficient state-of-the-art methods to build classification trees are greedy heuristics (e.g. CART) that may fail to find underlying characteristics in datasets. Recently, exact linear formulations that have shown better accuracy were introduced. However they do not scale up to datasets with...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Computers & operations research Jg. 164; S. 106515
Hauptverfasser: Ales, Zacharie, Huré, Valentine, Lambert, Amélie
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Ltd 01.04.2024
Elsevier
Schlagworte:
Artificial Intelligence
Combinatorial optimization
Computer Science
Linearizations
Mixed binary programming
Operations Research
Optimal classification trees
Quadratic programming
Mixed binary programming
Linearizations
Combinatorial optimization
Quadratic programming
Optimal classification trees
combinatorial optimization
optimal classification trees
quadratic programming
linearizations
mixed binary programming
ISSN:0305-0548, 1873-765X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The most efficient state-of-the-art methods to build classification trees are greedy heuristics (e.g. CART) that may fail to find underlying characteristics in datasets. Recently, exact linear formulations that have shown better accuracy were introduced. However they do not scale up to datasets with more than a few thousands data points. In this paper, we introduce four new formulations for building optimal trees. The first one is a quadratic model based on the well-known formulation of Bertsimas et al. We then propose two different linearizations of this new quadratic model. The last model is an extension to real-valued datasets of a flow-formulation limited to binary datasets (Aghaei et al.). Each model is introduced for both axis-aligned and oblique splits. We further prove that our new formulations have stronger continuous relaxations than existing models. Finally, we present computational results on 22 standard datasets with up to thousands of data points. Our exact models have reduced solution times with learning performances as strong or significantly better than state of the art exact approaches. •New quadratic formulation linearized in two different ways.•Extension of a flow formulation to non-binary datasets.•Our 3 linear formulations have stronger linear relaxations than other formulations.•Iterative algorithm to fit our models’ parameters with a reduced number of iterations.•Experimental results showing the efficiency of our formulations and our algorithm.
ISSN:0305-0548
1873-765X
DOI:10.1016/j.cor.2023.106515