Translation Invariance in the Polynomial Kernel Space and Its Applications in kNN Classification

In this paper, a new technique is presented to measure dissimilarity in kernel space providing scaling and translation invariance. The motivation comes from signal/image processing, where classifiers are often required to ensure invariance against linear transforms, since in many cases linear transforms do not affect the content of a signal/image for a human observer. We examine the theoretical background of linear invariance in the polynomial kernel space, introduce the centered correlation and centered Euclidean dissimilarity in kernel space, deduce formulas to compute it efficiently and test the proposed dissimilarity measures with the kNN classifier. The experimental results show that the presented techniques are highly competitive in similarity or dissimilarity based classification methods.

[1]  Sanjeev R. Kulkarni,et al.  Learning Pattern Classification - A Survey , 1998, IEEE Trans. Inf. Theory.

[2]  Xuegong Zhang,et al.  Kernel Nearest-Neighbor Algorithm , 2002, Neural Processing Letters.

[3]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[4]  Bram van Ginneken,et al.  Automatic detection of red lesions in digital color fundus photographs , 2005, IEEE Transactions on Medical Imaging.

[5]  M. Evans.  Statistical Distributions , 2000 .

[6]  Yuan Yao,et al.  Mercer's Theorem, Feature Maps, and Smoothing , 2006, COLT.

[7]  Matti Pietikäinen,et al.  Local Binary Pattern Descriptors for Dynamic Texture Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[8]  Wen Wen,et al.  Kernel based gene expression pattern discovery and its application on cancer classification , 2010, Neurocomputing.

[9]  Leif E. Peterson,et al.  Logistic Ensembles for Random Spherical Linear Oracles , 2007, ICMLA 2007.

[10]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[11]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[12]  S. Smale,et al.  Reproducing kernel hilbert spaces in learning theory , 2006 .

[13]  Dattatray V. Jadhav,et al.  Multiresolution Feature Based Fractional Power Polynomial Kernel Fisher Discriminant Model for Face Recognition , 2008, J. Multim..

[14]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[15]  Yang Liu,et al.  Face Recognition Using Kernel PCA and Hierarchical RBF Network , 2007, 6th International Conference on Computer Information Systems and Industrial Management Applications (CISIM'07).

[16]  Francesco Camastra,et al.  Kernel Methods for Clustering , 2005, WIRN/NAIS.

[17]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[18]  Ralf Herbrich,et al.  Learning Kernel Classifiers: Theory and Algorithms , 2001 .

[19]  Christoph H. Lampert.  Kernel Methods in Computer Vision , 2009, Found. Trends Comput. Graph. Vis..

[20]  Anne M. Denton,et al.  A Kernel-Based Semi-Naïve Bayesian Classifier Using P-Trees , 2004, SDM.

[21]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[22]  Michalis E. Blazadonakis,et al.  Polynomial and RBF Kernels as Marker Selection Tools-A Breast Cancer Case Study , 2007, Sixth International Conference on Machine Learning and Applications (ICMLA 2007).

[24]  Qin Li,et al.  Retinopathy Online Challenge: Automatic Detection of Microaneurysms in Digital Color Fundus Photographs , 2010, IEEE Transactions on Medical Imaging.

[25]  Klaus Obermayer,et al.  Self-organizing maps: Generalizations and new optimization techniques , 1998, Neurocomputing.

[26]  Navnath S. Nehe,et al.  Isolated Word Recognition Using Low Dimensional Features and Kernel Based Classification , 2009, 2009 International Conference on Advances in Recent Technologies in Communication and Computing.

[27]  Peyman Milanfar,et al.  Nonlinear kernel backprojection for computed tomography , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  Elena Deza,et al.  Encyclopedia of Distances , 2014 .