Kernel based simple regularized multiple criteria linear program for binary classification and regression

Handling data classification and regression problems through linear hyperplane is a naive and simple idea. In this paper, inspired by the idea of multiple criteria linear programs (MCLP) and multiple criteria quadratic programs (MCQP), we proposed a novel method for binary classification and regression problem. There are two main advantages for the proposed approach. One is that both of these two models guarantee the existence of feasible solutions when the model parameters were chosen properly. The other is that nonlinear patterns could be handled and captured by introducing kernel function into MCLP framework with a more natural way than previous work. Various classical approaches and datasets were evaluated in our experiments, and the result on both toy and real world data demonstrate the correctness and effectiveness of our proposed methods.

[1]  William Nick Street,et al.  Breast Cancer Diagnosis and Prognosis Via Linear Programming , 1995, Oper. Res..

[2]  Yong Shi,et al.  A Simple Regularized Multiple Criteria Linear Programs for Binary Classification , 2013, ICCS.

[3]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[4]  Yanchun Zhang,et al.  Domain-Driven Classification Based on Multiple Criteria and Multiple Constraint-Level Programming for Intelligent Credit Scoring , 2010, IEEE Transactions on Knowledge and Data Engineering.

[5]  J. B. Rosen.  Pattern separation by convex programming , 1965 .

[6]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  Yong Shi,et al.  Robust twin support vector machine for pattern classification , 2013, Pattern Recognit..

[9]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Abraham Charnes,et al.  Management Models and Industrial Applications of Linear Programming , 1961 .

[11]  D. Mackay,et al.  Introduction to Gaussian processes , 1998 .

[12]  O. Mangasarian.  Linear and Nonlinear Separation of Patterns by Linear Programming , 1965 .

[13]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[14]  Geoffrey I. Webb.  Decision Tree Grafting From the All Tests But One Partition , 1999, IJCAI.

[15]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[16]  Yong Shi.  China's national personal credit scoring system: a real-life intelligent knowledge application , 2012, KDD.

[17]  Olvi L. Mangasarian.  Mathematical Programming in Machine Learning , 1996 .

[18]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[19]  F. Glover,et al.  Simple but powerful goal programming models for discriminant problems , 1981 .

[20]  Yong Shi,et al.  Laplacian twin support vector machine for semi-supervised classification , 2012, Neural Networks.

[21]  P. Yu.  A Class of Solutions for Group Decision Problems , 1973 .

[22]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[23]  Jing-Yu Yang,et al.  A novel method for Fisher discriminant analysis , 2004, Pattern Recognit..

[24]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[25]  Yong Shi,et al.  Multiple criteria optimization-based data mining methods and applications: a systematic survey , 2010, Knowledge and Information Systems.

[26]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[27]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.