A Novel Procedure for Training L1-L2 Support Vector Machine Classifiers

In this work we propose a novel algorithm for training L1-L2 Support Vector Machine (SVM) classifiers. L1-L2 SVMs allow to combine the effectiveness of L2 models and the feature selection characteristics of L1 solutions. The proposed training approach for L1-L2 SVM requires a minimal effort for its implementation, relying on the exploitation of well-known and widespread tools already developed for conventional L2 SVMs. Moreover, the proposed method is flexible, as it allows to train L1, L1-L2 and L2 SVMs, as well as to fine tune the trade-off between dimensionality reduction and classification accuracy. This scope is of clear importance in applications on resource-limited devices, such as smartphones, like the one we consider to verify the main advantages of the proposed approach: the UCI Human Activity Recognition real-world dataset.

[1]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[2]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[3]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[4]  Ramón Hervás,et al.  Ambient Assisted Living and Home Care , 2012, Lecture Notes in Computer Science.

[5]  Kam D. Dahlquist,et al.  Regression Approaches for Microarray Data Analysis , 2002, J. Comput. Biol..

[6]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[7]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[8]  Davide Anguita,et al.  In-Sample and Out-of-Sample Model Selection and Error Estimation for Support Vector Machines , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Davide Anguita,et al.  A Public Domain Dataset for Human Activity Recognition using Smartphones , 2013, ESANN.

[10]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[11]  Shiliang Sun,et al.  A review of optimization methodologies in support vector machines , 2011, Neurocomputing.

[12]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[13]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[14]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[15]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[16]  J. Platt Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[17]  William H. Press,et al.  Numerical recipes in C , 2002 .