Ensemble classifiers with improved overfitting

Overfitting has been always considered as a challenging problem in designing and training of ensemble classifiers. Obviously, the use of complex multiple classifiers may increase the success of ensemble classifier in feature space division with intertwined data and also may decrease the training err...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2016 1st Conference on Swarm Intelligence and Evolutionary Computation (CSIEC) S. 93 - 97
Hauptverfasser: Pourtaheri, Zeinab Khatoun, Zahiri, Seyed Hamid
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.03.2016
Schlagworte:
Algorithm design and analysis
Classification algorithms
Classifier selection
Ensemble classifier
Ensemble size
Heuristic algorithms
Linear programming
Multi-objective optimization methods
Optimization
Overfitting
Particle swarm optimization
Training
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Overfitting has been always considered as a challenging problem in designing and training of ensemble classifiers. Obviously, the use of complex multiple classifiers may increase the success of ensemble classifier in feature space division with intertwined data and also may decrease the training error to minimum value. However, this success does not exist on the test data. Ensemble classifiers are more prone to overfitting than single classifiers because ensemble classifiers have been formed of several base classifiers and overfitting occurrence in each base classifier can transfer the problem to the final decision of the ensemble. In this paper, after quantitative and qualitative analysis of overfitting, a solution for improving overfitting is proposed by using heuristic algorithms. In this way, Multi-Objective Inclined Planes Optimization (MOIPO) and Multi-Objective Particle Swarm Optimization (MOPSO) are used and their results are compared with each other. Simulation results show that the simultaneous minimization of ensemble size and error rate in the training phase, can lead to a significant reduction in the amount of overfitting. In fact, with this approach in the training phase, the ensemble classifier is required to minimize the error with the most simple and minimum number of base classifiers and therefore overfitting is prevented. However, previous researches related to overfitting have ignored the ensemble size as an objective function.
DOI:10.1109/CSIEC.2016.7482130