Spectral Analysis of Symmetric and Anti-Symmetric Pairwise Kernels

We consider the problem of learning regression functions from pairwise data when there exists prior knowledge that the relation to be learned is symmetric or anti-symmetric. Such prior knowledge is commonly enforced by symmetrizing or anti-symmetrizing pairwise kernel functions. Through spectral analysis, we show that these transformations reduce the kernel's effective dimension. Further, we provide an analysis of the approximation properties of the resulting kernels, and bound the regularization bias of the kernels in terms of the corresponding bias of the original kernel.

[1]  Tapio Salakoski,et al.  Learning intransitive reciprocal relations with kernel methods , 2010, Eur. J. Oper. Res..

[2]  A. Caponnetto,et al.  Optimal Rates for the Regularized Least-Squares Algorithm , 2007, Found. Comput. Math..

[3]  William Stafford Noble,et al.  A new pairwise kernel for biological network inference with support vector machines , 2007, BMC Bioinformatics.

[4]  E. Carlen.  TRACE INEQUALITIES AND QUANTUM ENTROPY: An introductory course , 2009 .

[5]  Yuan Li,et al.  Von Neumann entropy and majorization , 2013, 1304.7442.

[6]  William Stafford Noble,et al.  Kernel methods for predicting protein-protein interactions , 2005, ISMB.

[7]  Thore Graepel,et al.  Large Margin Rank Boundaries for Ordinal Regression , 2000 .

[8]  Tong Zhang,et al.  Learning Bounds for Kernel Regression Using Effective Data Dimensionality , 2005, Neural Computation.

[9]  B. Berndt,et al.  Illinois Journal of Mathematics , 1957, Nature.

[10]  Tapio Salakoski,et al.  A Kernel-Based Framework for Learning Graded Relations From Data , 2011, IEEE Transactions on Fuzzy Systems.

[11]  Tong Zhang,et al.  Effective Dimension and Generalization of Kernel Learning , 2002, NIPS.

[12]  Yoshihiro Yamanishi,et al.  On Pairwise Kernels: An Efficient Alternative and Generalization Analysis , 2009, PAKDD.

[13]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[14]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[15]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[16]  Man-Duen Choi.  A schwarz inequality for positive linear maps on $C^{\ast}$-algebras , 1974 .

[17]  M. Fujii,et al.  Operator Inequalities Related To Cauchy-Schwarz And Hölder-McCarthy Inequalities , 1997 .

[18]  Shahar Mendelson,et al.  On the Performance of Kernel Classes , 2003, J. Mach. Learn. Res..

[19]  N. Aronszajn.  Rayleigh-Ritz and A. Weinstein Methods for Approximation of Eigenvalues: I. Operations in a Hilbert Space. , 1948, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Ingram Olkin,et al.  Matrix versions of the Cauchy and Kantorovich inequalities , 1990 .

[21]  A. Holevo,et al.  Quantum state majorization at the output of bosonic Gaussian channels , 2013, Nature Communications.

[22]  Andreas Fischer,et al.  Pairwise support vector machines and their application to large scale problems , 2012, J. Mach. Learn. Res..

[23]  Chun Yuan Deng,et al.  A generalization of the Sherman-Morrison-Woodbury formula , 2011, Appl. Math. Lett..