L n -norm Multiple Kernel Learning and Least Squares Support Vector Machines

In the era of information overflow, data mining and machine learning are indispensable tools to retrieve information and knowledge from data. The idea of incorporating several data sources in analysis may be beneficial by reducing the noise, as well as by improving statistical significance and leveraging the interactions and correlations between data sources to obtain more refined and higher-level information [50], which is known as data fusion. In bioinformatics, considerable effort has been devoted to genomic data fusion, which is an emerging topic pertaining to a lot of applications. At present, terabytes of data are generated by high-throughput techniques at an increasing rate. In data fusion, these terabytes are further multiplied by the number of data sources or the number of species. A statistical model describing this data is therefore not an easy matter. To tackle this challenge, it is rather effective to consider the data as being generated by a complex and unknown black box with the goal of finding a function or an algorithm that operates on an input to predict the output. About 15 years ago, Boser [8] and Vapnik [51] introduced the support vector method which makes use of kernel functions. This method has offered plenty of opportunities to solve complicated problems but also brought lots of interdisciplinary challenges in statistics, optimization theory, and the applications therein [40].

[1]  Gunnar Rätsch,et al.  Large Scale Multiple Kernel Learning , 2006, J. Mach. Learn. Res..

[2]  J. Khan,et al.  Database of mRNA gene expression profiles of multiple human organs. , 2005, Genome research.

[3]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[4]  Tijl De Bie,et al.  Kernel-based data fusion for gene prioritization , 2007, ISMB/ECCB.

[5]  Xiaoyun Yang,et al.  Reduction of False Positives in Polyp Detection Using Weighted Support Vector Machines , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[6]  A. Orth,et al.  Large-scale analysis of the human and mouse transcriptomes , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Olivier Gevaert,et al.  Mathematical decision trees versus clinician based algorithms in the diagnosis of endometrial disease , 2007 .

[8]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[9]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[10]  Yoshihiro Yamanishi,et al.  KEGG for linking genomes to life and the environment , 2007, Nucleic Acids Res..

[11]  Liva Ralaivola,et al.  Multiple indefinite kernel learning with mixed norm regularization , 2009, ICML '09.

[12]  Johan A. K. Suykens,et al.  Optimized Data Fusion for Kernel k-Means Clustering , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Bart De Moor,et al.  Development of a kernel function for clinical data , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[14]  Bart De Moor,et al.  Comparison of vocabularies, representations and ranking algorithms for gene prioritization by text mining , 2008, ECCB.

[15]  R. Reemtsen.  Some outer approximation methods for semi-infinite optimization problems , 1994 .

[16]  Stephen P. Boyd,et al.  Optimal kernel selection in Kernel Fisher discriminant analysis , 2006, ICML.

[17]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .

[18]  Johan A. K. Suykens,et al.  Multiclass least squares support vector machines , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[19]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[20]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[21]  Alexander E. Kel,et al.  TRANSFAC®: transcriptional regulation, from patterns to profiles , 2003, Nucleic Acids Res..

[22]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[23]  T. Bourne,et al.  Functional linear discriminant analysis: a new longitudinal approach to the assessment of embryonic growth. , 2009, Human reproduction.

[24]  Bart De Moor,et al.  Gene prioritization and clustering by multi-view text mining , 2010, BMC Bioinformatics.

[25]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[26]  Federico Girosi,et al.  Support Vector Machines: Training and Applications , 1997 .

[27]  Bart De Moor,et al.  Predicting the prognosis of breast cancer by integrating clinical and microarray data with Bayesian networks , 2006, ISMB.

[28]  Knud D. Andersen,et al.  The Mosek Interior Point Optimizer for Linear Programming: An Implementation of the Homogeneous Algorithm , 2000 .

[29]  Ali Amin Al Olama,et al.  Multiple newly identified loci associated with prostate cancer susceptibility , 2008, Nature Genetics.

[30]  Jieping Ye,et al.  Multi-class Discriminant Kernel Learning via Convex Programming , 2008, J. Mach. Learn. Res..

[31]  A. Ng.  Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[32]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[33]  Xuan Li,et al.  Association of tissue lineage and gene expression: conservatively and differentially expressed genes define common and special functions of tissues , 2010, BMC Bioinformatics.

[34]  Jian Ye,et al.  BLAST: improvements for better sequence analysis , 2006, Nucleic Acids Res..

[35]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[36]  Kenneth O. Kortanek,et al.  Semi-Infinite Programming: Theory, Methods, and Applications , 1993, SIAM Rev..

[37]  Nello Cristianini,et al.  Controlling the Sensitivity of Support Vector Machines , 1999 .

[38]  Bassem A. Hassan,et al.  Gene prioritization through genomic data fusion , 2006, Nature Biotechnology.

[39]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[40]  Anna Tramontano,et al.  Critical assessment of methods of protein structure prediction—Round VII , 2007, Proteins.

[41]  J. Suykens,et al.  A kernel-based integration of genome-wide data for clinical decision support , 2009, Genome Medicine.

[42]  Klaus-Robert Müller,et al.  Efficient and Accurate Lp-Norm Multiple Kernel Learning , 2009, NIPS.

[43]  Nello Cristianini,et al.  A statistical framework for genomic data fusion , 2004, Bioinform..

[44]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[45]  W. Willett,et al.  Multiple loci identified in a genome-wide association study of prostate cancer , 2008, Nature Genetics.

[46]  Eleazar Eskin,et al.  The Spectrum Kernel: A String Kernel for SVM Protein Classification , 2001, Pacific Symposium on Biocomputing.

[47]  Kevin M. Bradley,et al.  Common sequence variants on 2p15 and Xp11.22 confer susceptibility to prostate cancer , 2008, Nature Genetics.

[48]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[49]  Bart De Moor,et al.  TOUCAN 2: the all-inclusive open source workbench for regulatory sequence analysis , 2005, Nucleic Acids Res..

[50]  D. Timmerman,et al.  OC147: Mathematical decision trees vs. clinician‐based algorithms in the diagnosis of endometrial disease , 2007 .

[51]  Robert D. Finn,et al.  New developments in the InterPro database , 2007, Nucleic Acids Res..

[52]  S. Van Huffel,et al.  The use of a new logistic regression model for predicting the outcome of pregnancies of unknown location , 2004 .

[53]  Gavin C. Cawley,et al.  Leave-One-Out Cross-Validation Based Model Selection Criteria for Weighted LS-SVMs , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[54]  T. Terlaky,et al.  Logarithmic barrier decomposition methods for semi-infinite programming , 1997 .

[55]  M. Ashburner,et al.  Gene Ontology: tool for the unification of biology , 2000, Nature Genetics.

[56]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.