Clustering ensembles of neural network models

We show that large ensembles of (neural network) models, obtained e.g. in bootstrapping or sampling from (Bayesian) probability distributions, can be effectively summarized by a relatively small number of representative models. In some cases this summary may even yield better function estimates. We...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Neural networks Ročník 16; číslo 2; s. 261 - 269
Hlavní autori: Bakker, Bart, Heskes, Tom
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Oxford Elsevier Ltd 01.03.2003
Elsevier Science
Predmet:
Algorithms
Applied sciences
Bias
Bias/variance analysis
Bootstrapping
Cluster Analysis
Clustering
Clustering; Bootstrapping
Deterministic annealing
Electric, optical and optoelectronic circuits
Electronics
Exact sciences and technology
Expectation-maximization algorithm
Learning - physiology
Multilayered perceptron
Multitask learning
Neural networks
Neural Networks (Computer)
Nonlinear model
variance analysis
Bootstrapping
Multitask learning
Expectation-maximization algorithm
Bias/variance analysis
Multilayered perceptron
Clustering
Deterministic annealing
Nonlinear model
Bias
Probability distribution
Cluster model
Parallel system
Qualitative analysis
Sampling
Learning algorithm
Bayes estimation
Annealing
Maximization
Neural network
Variance analysis
Experimental result
Multithread
Deterministic model
Non linear model
Multilayer perceptrons
Numerical simulation
Expectation
ISSN:0893-6080, 1879-2782
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:We show that large ensembles of (neural network) models, obtained e.g. in bootstrapping or sampling from (Bayesian) probability distributions, can be effectively summarized by a relatively small number of representative models. In some cases this summary may even yield better function estimates. We present a method to find representative models through clustering based on the models' outputs on a data set. We apply the method on an ensemble of neural network models obtained from bootstrapping on the Boston housing data, and use the results to discuss bootstrapping in terms of bias and variance. A parallel application is the prediction of newspaper sales, where we learn a series of parallel tasks. The results indicate that it is not necessary to store all samples in the ensembles: a small number of representative models generally matches, or even surpasses, the performance of the full ensemble. The clustered representation of the ensemble obtained thus is much better suitable for qualitative analysis, and will be shown to yield new insights into the data.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0893-6080
1879-2782
DOI:10.1016/S0893-6080(02)00187-9