Optimizing ψ-learning via mixed integer programming

As a new margin-based classier, -learning shows great potential for high accuracy. However, the optimization of -learning involves non-convex min- imization and is very challenging to implement. In this article, we convert the optimization of -learning into a mixed integer programming (MIP) problem. This enables us to utilize the state-of-art algorithm of MIP to solve -learning. More- over, the new algorithm can solve -learning with a general piecewise linear loss and does not require continuity of the loss function. We also examine the variable selection property of 1-norm -learning and make comparisons with the SVM.

[1]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[2]  Yufeng Liu,et al.  Multicategory ψ-Learning , 2006 .

[3]  Yufeng Liu,et al.  Multicategory ψ-Learning and Support Vector Machine: Computational Tools , 2005 .

[4]  Olvi L. Mangasarian,et al.  Nuclear feature extraction for breast tumor diagnosis , 1993, Electronic Imaging.

[5]  W. Wong,et al.  On ψ-Learning , 2003 .

[6]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[7]  Charles E. Blair,et al.  Integer and Combinatorial Optimization (George L. Nemhauser and Laurence A. Wolsey) , 1990, SIAM Rev..

[8]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988 .

[9]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[10]  G. Nemhauser,et al.  Integer Programming , 2020 .

[11]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[12]  Xiaotong Shen,et al.  Computational Developments of ψ-learning , 2005, SDM.

[13]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[14]  Takashi Takenouchi,et al.  Statistical Learning Theory by Boosting Method , 2004 .

[15]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[16]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[17]  G. Wahba.  Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV , 1999 .

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .