DDC: distance-based decision classifier

This paper presents a new classification method utilizing distance-based decision surface with nearest neighbor projection approach, called DDC. Kernel type of DDC has been extended to take into account the effective nonlinear structure of the data. DDC has some properties: (1) does not need conventional learning procedure (as k-NN algorithm), (2) does not need searching time to locate the k-nearest neighbors, and (3) does not need optimization process unlike some classification methods such as Support Vector Machine (SVM). In DDC, we compute the weighted average of distances to all the training samples. Unclassified sample will be classified as belonging to a class that has the minimum obtained distance. As a result, by such a rule we can derive a formula that can be used as the decision surface. DDC is tested on both synthetic and real-world data sets from the UCI repository, and the results were compared with k-NN, RBF Network, and SVM. The experimental results indicate DDC outperforms k-NN in the most experiments and the results are comparable to or better than SVM with some data sets.

[1]  Ulrike von Luxburg,et al.  Distance-Based Classification with Lipschitz Functions , 2004, J. Mach. Learn. Res..

[2]  Robert P. W. Duin,et al.  Using two-class classifiers for multiclass classification , 2002, Object recognition supported by user interaction for service robots.

[3]  Juan J. Navarro,et al.  Exploiting computer resources for fast nearest neighbor classification , 2007, Pattern Analysis and Applications.

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  J. Mark.  Introduction to radial basis function networks , 1996 .

[6]  Michihiko Minoh,et al.  A fast algorithm for the minimum distance classifier and its application to Kanji character recognition , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[7]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[8]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[9]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[10]  Xinhua Zhuang,et al.  A geometric algorithm finding set of linear decision boundaries , 1994, IEEE Trans. Signal Process..

[11]  Pedro E. López-de-Teruel,et al.  Nonlinear kernel-based statistical pattern analysis , 2001, IEEE Trans. Neural Networks.

[12]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[13]  Pascal Vincent,et al.  K-Local Hyperplane and Convex Distance Nearest Neighbor Algorithms , 2001, NIPS.

[14]  Dimitrios Gunopulos,et al.  Locally Adaptive Metric Nearest-Neighbor Classification , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Jiankuan Ye,et al.  Experimental Study of Support Vector Machines Based on Linear and Quadratic Optimization Criteria 1 , 2009 .

[16]  Trevor Darrell,et al.  Nearest-Neighbor Methods in Learning and Vision: Theory and Practice (Neural Information Processing) , 2006 .

[17]  Wai Lam,et al.  Discovering Useful Concept Prototypes for Classification Based on Filtering and Abstraction , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  M. Madheswaran,et al.  Texture pattern analysis of kidney tissues for disorder identification and classification using dominant Gabor wavelet , 2010, Machine Vision and Applications.

[19]  José Francisco Martínez Trinidad,et al.  A new fast prototype selection method based on clustering , 2010, Pattern Analysis and Applications.

[20]  Jiang Wen-han.  Kernel Nearest Neighbor Convex Hull Classification Algorithm , 2007 .

[21]  David Zhang,et al.  On kernel difference-weighted k-nearest neighbor classification , 2007, Pattern Analysis and Applications.

[22]  Sing-Tze Bow,et al.  Pattern recognition and image preprocessing , 1992 .

[23]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[24]  Cor J. Veenman,et al.  The nearest subclass classifier: a compromise between the nearest mean and nearest neighbor classifier , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[26]  Belur V. Dasarathy,et al.  Nearest neighbor (NN) norms: NN pattern classification techniques , 1991 .

[27]  Sahibsingh A. Dudani.  The Distance-Weighted k-Nearest-Neighbor Rule , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[28]  D. A. Karras,et al.  Pattern classification using a generalised Hamming distance metric , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[29]  Juan Luis Castro,et al.  Local distance-based classification , 2008, Knowl. Based Syst..

[30]  Elzbieta Pekalska,et al.  Kernel Discriminant Analysis for Positive Definite and Indefinite Kernels , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  David G. Stork,et al.  Pattern Classification , 1973 .

[32]  Thierry Pun,et al.  Distance-based discriminant analysis method and its applications , 2008, Pattern Analysis and Applications.

[33]  Xuehua Li,et al.  Kernel-based nonlinear dimensionality reduction for electrocardiogram recognition , 2009, Neural Computing and Applications.