Unconstrained optimization in projection method for indefinite SVMs

Positive semi-definiteness is a critical property in Support Vector Machine (SVM) methods to ensure efficient solutions through convex quadratic programming. In this paper, we introduce a projection matrix on indefinite kernels to formulate a positive semi-definite one. The proposed model can be regarded as a generalized version of the spectrum method (denoising method and flipping method) by varying parameter λ. In particular, our suggested optimal λ under the Bregman matrix divergence theory can be obtained using unconstrained optimization. Experimental results on 4 real world data sets ranging from glycan classification to cancer prediction show that the proposed model can achieve better or competitive performance when compared to the related indefinite kernel methods. This may suggest a new way in motif extractions or cancer predictions.

[1]  Shigeki Sagayama,et al.  Dynamic Time-Alignment Kernel in Support Vector Machine , 2001, NIPS.

[2]  Kiyoko F. Aoki-Kinoshita,et al.  KEGG as a glycome informatics resource. , 2006, Glycobiology.

[3]  Inderjit S. Dhillon,et al.  Learning low-rank kernel matrices , 2006, ICML.

[4]  Alexander J. Smola,et al.  Regularization with Dot-Product Kernels , 2000, NIPS.

[5]  Dale Schuurmans,et al.  A Reformulation of Support Vector Machines for General Confidence Functions , 2009, ACML.

[6]  Nozha Boujemaa,et al.  Generalized histogram intersection kernel for image recognition , 2005, IEEE International Conference on Image Processing 2005.

[7]  Alexander J. Smola,et al.  Learning with non-positive kernels , 2004, ICML.

[8]  Domenico Conforti,et al.  Kernel based support vector machine via semidefinite programming: Application to medical diagnosis , 2010, Comput. Oper. Res..

[9]  Joachim M. Buhmann,et al.  Optimal Cluster Preserving Embedding of Nonmetric Proximity Data , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Bernard Haasdonk,et al.  Feature space interpretation of SVMs with indefinite kernels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Robert P. W. Duin,et al.  A Generalized Kernel Approach to Dissimilarity-based Classification , 2002, J. Mach. Learn. Res..

[12]  Emilio Carrizosa,et al.  Supervised classification and mathematical optimization , 2013, Comput. Oper. Res..

[13]  Hsuan-Tien Lin.  A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .

[14]  Tatsuya Akutsu,et al.  Protein homology detection using string alignment kernels , 2004, Bioinform..

[15]  P. Albersheim,et al.  Letter to the Glyco-Forum CarbBank , 1992 .

[16]  Bernhard Schölkopf,et al.  Kernel Methods in Computational Biology , 2005 .

[17]  W. Ching,et al.  PROJECTION METHOD FOR SUPPORT VECTOR MACHINES WITH INDEFINITE KERNELS , 2015 .

[18]  Alexandre d'Aspremont,et al.  Support vector machine classification with indefinite kernels , 2007, Math. Program. Comput..

[19]  Zheng Zhang,et al.  An Analysis of Transformation on Non - Positive Semidefinite Similarity Matrix for Kernel Machines , 2005, ICML 2005.

[20]  Klaus Obermayer,et al.  Classi cation on Pairwise Proximity , 2007 .

[21]  Yuhong Guo,et al.  Learning SVM Classifiers with Indefinite Kernels , 2012, AAAI.

[22]  Cristian Sminchisescu,et al.  Kernel Learning by Unconstrained Optimization , 2009, AISTATS.

[23]  E. Myers,et al.  Basic local alignment search tool. , 1990, Journal of molecular biology.

[24]  Alexander J. Smola,et al.  Learning with kernels , 1998 .