Regularization networks with indefinite kernels

Learning with indefinite kernels attracted considerable attention in recent years due to their success in various learning scenarios. In this paper we study the asymptotic properties of the regularization kernel networks where the kernels are assumed to be indefinite, without the usual restrictions of symmetry and positive semi-definiteness as in the traditional study of kernel methods. The kernels are characterized in terms of the singular value decomposition of the corresponding kernel integrals. Two reproducing kernel Hilbert spaces are induced to characterize the approximation ability. Capacity independent error bounds are proved. Fast convergence rates are obtained both in reproducing kernel Hilbert spaces and in L^2 sense.

[1]  Qiang Wu,et al.  Least square regression with indefinite kernels and coefficient regularization , 2011 .

[2]  Tatsuya Akutsu,et al.  Protein homology detection using string alignment kernels , 2004, Bioinform..

[3]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[4]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[5]  Alexander J. Smola,et al.  Learning with non-positive kernels , 2004, ICML.

[6]  Bernard Haasdonk,et al.  Feature space interpretation of SVMs with indefinite kernels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Colin Campbell,et al.  Analysis of SVM with Indefinite Kernels , 2009, NIPS.

[8]  Klaus Obermayer,et al.  Classi cation on Pairwise Proximity , 2007 .

[9]  Qiang Wu,et al.  Classification and regularization in learning theory , 2005 .

[10]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[11]  Ding-Xuan Zhou,et al.  Concentration estimates for learning with ℓ1-regularizer and data dependent hypothesis spaces , 2011 .

[12]  Elzbieta Pekalska,et al.  Kernel Discriminant Analysis for Positive Definite and Indefinite Kernels , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Ding-Xuan Zhou,et al.  LEARNING BY NONSYMMETRIC KERNELS WITH DATA DEPENDENT SPACES AND , 2010 .

[14]  Chengjun Liu,et al.  Gabor-based kernel PCA with fractional power polynomial models for face recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[16]  Alexandre d'Aspremont,et al.  Support vector machine classification with indefinite kernels , 2007, Math. Program. Comput..

[17]  Robert P. W. Duin,et al.  A Generalized Kernel Approach to Dissimilarity-based Classification , 2002, J. Mach. Learn. Res..

[18]  Hsuan-Tien Lin.  A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .

[19]  S. Smale,et al.  Learning Theory Estimates via Integral Operators and Their Approximations , 2007 .

[20]  Ding-Xuan Zhou,et al.  Learning with sample dependent hypothesis spaces , 2008, Comput. Math. Appl..

[21]  Alexander J. Smola,et al.  Regularization with Dot-Product Kernels , 2000, NIPS.

[22]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[23]  Qiang Wu,et al.  A note on application of integral operator in learning theory , 2009 .