Adaptive Learning-Based k -Nearest Neighbor Classifiers With Resilience to Class Imbalance - Mullick, Sankha Subhra; Datta, Shounak; Das, Swagatam (IEEE transaction on neural networks and learning systems, 2018)
The classification accuracy of a k -nearest neighbor ( k NN) classifier is largely dependent on the choice of the number of nearest neighbors denoted by k . However, given a data set, it is a tedious task to optimize the performance of k NN by tuning k . Moreover, the performance of k NN degrades in the presence of class imbalance, a situation characterized by disparate representation from different classes. We aim to address both the issues in this paper and propose a variant of k NN called the Adaptive k NN (Ada- k NN). The Ada- k NN classifier uses the density and distribution of the neighborhood of a test point and learns a suitable point-specific k for it with the help of artificial neural networks. We further improve our proposal by replacing the neural network with a heuristic learning method guided by an indicator of the local density of a test point and using information about its neighboring training points. The proposed heuristic learning algorithm preserves the simplicity of k NN without incurring serious computational burden. We call this method Ada- k NN2. Ada- k NN and Ada- k NN2 perform very competitive when compared with k NN, five of k NN's state-of-the-art variants, and other popular classifiers. Furthermore, we propose a class-based global weighting scheme (Global Imbalance Handling Scheme or GIHS) to compensate for the effect of class imbalance. We perform extensive experiments on a wide variety of data sets to establish the improvement shown by Ada- k NN and Ada- k NN2 using the proposed GIHS, when compared with k NN, and its 12 variants specifically tailored for imbalanced classification.