The Entire Regularization Path for the Support Vector Machine

In this paper we argue that the choice of the SVM cost parameter can be critical. We then derive an algorithm that can fit the entire path of SVM solutions for every value of the cost parameter, with essentially the same computational cost as fitting one SVM model.

[1]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[2]  G. Wahba.  Spline models for observational data , 1990 .

[3]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[4]  Eugene L. Allgower,et al.  Continuation and path following , 1993, Acta Numerica.

[5]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[6]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[7]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[8]  Massimiliano Pontil,et al.  Properties of Support Vector Machines , 1998, Neural Computation.

[9]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[10]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[11]  Bernhard Schölkopf,et al.  GACV for Support Vector Machines , 2000 .

[12]  Kiri Wagstaff,et al.  Alpha seeding for support vector machines , 2000, KDD '00.

[13]  Bernhard Schölkopf,et al.  Regularization Networks and Support Vector Machines , 2000 .

[14]  Christopher K. I. Williams,et al.  The Effect of the Input Density Distribution on Kernel-based Classifiers , 2000, ICML.

[15]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[16]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[17]  Hansong Zhang,et al.  Gacv for support vector machines , 2000 .

[18]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[19]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[20]  T. Poggio,et al.  Multiclass cancer diagnosis using tumor gene expression signatures , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Katya Scheinberg,et al.  Incas: An incremental active set method for svm , 2002 .

[22]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[23]  Gert Cauwenberghs,et al.  SVM incremental learning, adaptation and optimization , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[24]  R. Tibshirani,et al.  Expression Arrays and the p n Problem , 2003 .

[25]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[26]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[27]  Ji Zhu,et al.  Margin Maximizing Loss Functions , 2003, NIPS.

[28]  T. Hastie,et al.  Classification of gene microarrays by penalized logistic regression. , 2004, Biostatistics.

[29]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[30]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[31]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[32]  Saharon Rosset,et al.  Tracking Curved Regularized Optimization Solution Paths , 2004, NIPS 2004.

[33]  Hans C. van Houwelingen,et al.  The Elements of Statistical Learning, Data Mining, Inference, and Prediction. Trevor Hastie, Robert Tibshirani and Jerome Friedman, Springer, New York, 2001. No. of pages: xvi+533. ISBN 0‐387‐95284‐5 , 2004 .

[34]  R. Tibshirani,et al.  Efficient quadratic regularization for expression arrays. , 2004, Biostatistics.