Critical Boundary Vector Concept in Nearest Neighbor Classifiers using k-Means Centers for Efficient Template Reduction

Dealing with large data sets, the computational cost and resource demands using the nearest neighbor (NN) classifier can be prohibitive. Aiming at efficient template condensation, this paper proposes a template reduction algorithm for NN classifier by introducing the concept of critical boundary vectors in conjunction with K-means centers. Initially K-means centers are used as substitution for the entire template set. Then, in order to enhance the classification performance, critical boundary vectors are selected according to a newly proposed training algorithm which completes with only single iteration. COIL-20 and COIL-100 databases were utilized for evaluating the performance of image categorization in which the bio-inspired directionaledge-based image feature representation (Suzuki and Shibata. 2004) was employed. UCI iris and UCI Landsat databases were also utilized to evaluate the system for other classification tasks using numerical-valued vectors. Experimental results show that by using the reduced template sets, the proposed algorithm shows a superior performance to NN classifier using all samples, and comparable to Support Vector Machines using Gaussian kernel which are computationally more expensive.

[1]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[2]  Amir F. Atiya,et al.  A Novel Template Reduction Approach for the $K$-Nearest Neighbor Method , 2009, IEEE Transactions on Neural Networks.

[3]  Shixiong Xia,et al.  An Improved KNN Text Classification Algorithm Based on Clustering , 2009, J. Comput..

[4]  Venu Govindaraju,et al.  Improved k-nearest neighbor classification , 2002, Pattern Recognit..

[5]  Tadashi Shibata,et al.  Multiple-clue face detection algorithm using edge-based feature vectors , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[7]  Joachim Denzler,et al.  A Comparison of Nearest Neighbor Search Algorithms for Generic Object Recognition , 2006, ACIVS.

[8]  Christoph F. Eick,et al.  Using representative-based clustering for nearest neighbor dataset editing , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[9]  Q. Henry Wu,et al.  A class boundary preserving algorithm for data condensation , 2011, Pattern Recognit..

[10]  Yitao Ma,et al.  A Binary-Tree Hierarchical Multiple-Chip Architecture for Real-Time Large-Scale Learning Processor Systems , 2010 .

[11]  Francesca Bovolo,et al.  A Novel Technique for Subpixel Image Classification Based on Support Vector Machine , 2010, IEEE Transactions on Image Processing.

[12]  T. Shibata,et al.  A Real-Time Learning Processor Based on K-means Algorithm with Automatic Seeds Generation , 2007, 2007 International Symposium on System-on-Chip.