A branch & bound algorithm to determine optimal cross-splits for decision tree induction

State-of-the-art decision tree algorithms are top-down induction heuristics which greedily partition the attribute space by iteratively choosing the best split on an individual attribute. Despite their attractive performance in terms of runtime, simple examples, such as the XOR-Problem, point out th...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Annals of mathematics and artificial intelligence Ročník 88; číslo 4; s. 291 - 311
Hlavní autori: Bollwein, Ferdinand, Dahmen, Martin, Westphal, Stephan
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Cham Springer International Publishing 01.04.2020
Springer
Springer Nature B.V
Predmet:
Algorithms
Artificial Intelligence
Bivariate analysis
Branch and bound methods
Combinatorial analysis
Complex Systems
Computer Science
Datasets
Decision trees
Machine learning
Mathematics
Optimization
Solution space
Branch & bound
Cross-splits
90C27
68T05
Decision trees
Classification
ISSN:1012-2443, 1573-7470
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:State-of-the-art decision tree algorithms are top-down induction heuristics which greedily partition the attribute space by iteratively choosing the best split on an individual attribute. Despite their attractive performance in terms of runtime, simple examples, such as the XOR-Problem, point out that these heuristics often fail to find the best classification rules if there are strong interactions between two or more attributes from the given datasets. In this context, we present a branch and bound based decision tree algorithm to identify optimal bivariate axis-aligned splits according to a given impurity measure. In contrast to a univariate split that can be found in linear time, such an optimal cross-split has to consider every combination of values for every possible selection of pairs of attributes which leads to a combinatorial optimization problem that is quadratic in the number of values and attributes. To overcome this complexity, we use a branch and bound procedure, a well known technique from combinatorial optimization, to divide the solution space into several sets and to detect the optimal cross-splits in a short amount of time. These cross splits can either be used directly to construct quaternary decision trees or they can be used to select only the better one of the individual splits. In the latter case, the outcome is a binary decision tree with a certain sense of foresight for correlated attributes. We test both of these variants on various datasets of the UCI Machine Learning Repository and show that cross-splits can consistently produce smaller decision trees than state-of-the-art methods with comparable accuracy. In some cases, our algorithm produces considerably more accurate trees due to the ability of drawing more elaborate decisions than univariate induction algorithms.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1012-2443
1573-7470
DOI:10.1007/s10472-019-09684-0