Local Minimax Learning of Functions With Best Finite Sample Estimation Error Bounds: Applications to Ridge and Lasso Regression, Boosting, Tree Learning, Kernel Machines, and Inverse Problems

Optimal local estimation is formulated in the minimax sense for inverse problems and nonlinear regression. This theory provides best mean squared finite sample error bounds for some popular statistical learning algorithms and also for several optimal improvements of other existing learning algorithm...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on information theory Ročník 55; číslo 12; s. 5700 - 5727
Hlavný autor:	Jones, L.K.
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York, NY IEEE 01.12.2009 Institute of Electrical and Electronics Engineers The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Algorithms Applied sciences Boosting Errors Estimating techniques Estimation error Exact sciences and technology Fusion Information geometry Information theory Information, signal and communications theory inverse problem Inverse problems Kernel Learning Machine learning Mathematical analysis Mathematical models minimax Minimax technique Minimax techniques Parameter optimization Pattern recognition Regression analysis Regression tree analysis reproducing kernel ridge regression Samples Signal processing Smoothing methods Statistical analysis Statistical learning Statistical methods Telecommunications and information theory Validity Query Optimal estimation Modeling Implementation Spline Mean square error Learning Optimal strategy Hilbert space Learning algorithm minimax Estimation error Smoothing methods Non linear regression ridge regression Regression analysis Pattern recognition Quadratic programming reproducing kernel Kernel method Inverse problem Pattern classification Minimax method Metric Fusion Algebraic geometry
ISSN:	0018-9448, 1557-9654
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Popis
Shrnutí:	Optimal local estimation is formulated in the minimax sense for inverse problems and nonlinear regression. This theory provides best mean squared finite sample error bounds for some popular statistical learning algorithms and also for several optimal improvements of other existing learning algorithms such as smoothing splines and kernel regularization. The bounds and improved algorithms are not based on asymptotics or Bayesian assumptions and are truly local for each query, not depending on cross validating estimates at other queries to optimize modeling parameters. Results are given for optimal local learning of approximately linear functions with side information (context) using real algebraic geometry. In particular, finite sample error bounds are given for ridge regression and for a local version of lasso regression. The new regression methods require only quadratic programming with linear or quadratic inequality constraints for implementation. Greedy additive expansions are then combined with local minimax learning via a change in metric. An optimal strategy is presented for fusing the local minimax estimators of a class of experts-providing optimal finite sample prediction error bounds from (random) forests. Local minimax learning is extended to kernel machines. Best local prediction error bounds for finite samples are given for Tikhonov regularization. The geometry of reproducing kernel Hilbert space is used to derive improved estimators with finite sample mean squared error (MSE) bounds for class membership probability in two class pattern classification problems. A purely local, cross validation free algorithm is proposed which uses Fisher information with these bounds to determine best local kernel shape in vector machine learning. Finally, a locally quadratic solution to the finite Fourier moments problem is presented. After reading the first three sections the reader may proceed directly to any of the subsequent applications sections.
Bibliografia:	SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-2 content type line 23
ISSN:	0018-9448 1557-9654
DOI:	10.1109/TIT.2009.2027479