Sets of approximating functions with finite Vapnik–Chervonenkis dimension for nearest-neighbors algorithms

► Reformulation of k-NN algorithm to alpha-NN ∗ algorithm (where alpha is a fraction). ► Establishing sets of functions for alpha-NN ∗, with finite capacity. ► Pointing out degrees of freedom for these sets. ► Proving theorems about dichotomies and VC-dimension for the proposed sets. According to a...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Pattern recognition letters Ročník 32; číslo 14; s. 1882 - 1893
Hlavní autori: KLCSK, P, KORZEN, M
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Amsterdam Elsevier B.V 15.10.2011
Elsevier
Predmet:
Applied sciences
Complexity selection
Exact sciences and technology
Generalization
Information, signal and communications theory
k-Nearest neighbors
Signal and communications theory
Signal representation. Spectral analysis
Signal, noise
Statistical learning theory
Structural risk minimization
Telecommunications and information theory
Vapnik–Chervonenkis dimension
Statistical learning theory
k-Nearest neighbors
Vapnik–Chervonenkis dimension
Generalization
Complexity selection
Structural risk minimization
Learning
Nearest neighbour
Vapnik-Chervonenkis dimension
Cross validation
Algorithm
Signal analysis
ISSN:0167-8655, 1872-7344
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:► Reformulation of k-NN algorithm to alpha-NN ∗ algorithm (where alpha is a fraction). ► Establishing sets of functions for alpha-NN ∗, with finite capacity. ► Pointing out degrees of freedom for these sets. ► Proving theorems about dichotomies and VC-dimension for the proposed sets. According to a certain misconception sometimes met in the literature: for the nearest-neighbors algorithms there is no fixed hypothesis class of limited Vapnik–Chervonenkis dimension. In the paper a simple reformulation (not a modification) of the nearest-neighbors algorithm is shown where instead of a natural number k, a percentage α ∈ (0, 1) of nearest neighbors is used. Owing to this reformulation one can construct sets of approximating functions, which we prove to have finite VC dimension. In a special (but practical) case this dimension is equal to ⌊2/ α⌋. It is also then possible to form a sequence of sets of functions with increasing VC dimension, and to perform complexity selection via cross-validation or similarly to the structural risk minimization framework. Results of such experiments are also presented.
ISSN:0167-8655
1872-7344
DOI:10.1016/j.patrec.2011.07.012