Robust spherical separation

Abstract We propose a robust spherical separation technique aimed at separating two finite sets of points and . Robustness concerns the possibility to admit uncertainties and perturbations in the data-set, which may occur when the data are corrupted by noise or are influenced by measurement errors. In particular, starting from the standard spherical separation under the assumption of spherical uncertainty, we propose a model characterized by a non-convex non-differentiable objective function, which we minimize by means of a bundle-type algorithm. Quite promising numerical results are provided on small and large data-sets drawn from well-established test beds in literature.

[1]  Chiranjib Bhattacharyya,et al.  Chance constrained uncertain classification via robust optimization , 2011, Math. Program..

[2]  Antonie Stam,et al.  Second order mathematical programming formulations for discriminant analysis , 1994 .

[3]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[4]  Alexander J. Smola,et al.  A Second Order Cone programming Formulation for Classifying Missing Data , 2004, NIPS.

[5]  S. Sathiya Keerthi,et al.  An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models , 2006, NIPS.

[6]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[7]  Adil M. Bagirov,et al.  Max–min separability , 2005, Optim. Methods Softw..

[8]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[9]  Le Thi Hoai An,et al.  Binary classification via spherical separator by DC programming and DCA , 2012, Journal of Global Optimization.

[10]  C. Gerth,et al.  Nonconvex separation theorems and some applications in vector optimization , 1990 .

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[13]  Dirk C. Mattfeld,et al.  Synergies of Operations Research and Data Mining , 2010, Eur. J. Oper. Res..

[14]  F. Giannessi.  Variational Analysis and Generalized Differentiation , 2006 .

[15]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[16]  Laura Schweitzer,et al.  Advances In Kernel Methods Support Vector Learning , 2016 .

[17]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[18]  Annabella Astorino,et al.  Nonsmooth Optimization Techniques for Semisupervised Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Alexander J. Smola,et al.  Second Order Cone Programming Approaches for Handling Missing and Uncertain Data , 2006, J. Mach. Learn. Res..

[20]  Frank Plastria,et al.  Multi-instance classification through spherical separation and VNS , 2014, Comput. Oper. Res..

[21]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[22]  D. Pallaschke,et al.  Separation of convex sets by Clarke subdifferential , 2010 .

[23]  S. Odewahn,et al.  Automated star/galaxy discrimination with neural networks , 1992 .

[24]  R. Mifflin.  Semismooth and Semiconvex Functions in Constrained Optimization , 1977 .

[25]  Wenjie Hu,et al.  Robust support vector machine with bullet hole image classification , 2002 .

[26]  Marcello Sanguineti,et al.  A theoretical framework for supervised learning from regions , 2014, Neurocomputing.

[27]  B. Mordukhovich.  Variational analysis and generalized differentiation , 2006 .

[28]  Emilio Carrizosa,et al.  Supervised classification and mathematical optimization , 2013, Comput. Oper. Res..

[29]  Shie Mannor,et al.  Robustness and Regularization of Support Vector Machines , 2008, J. Mach. Learn. Res..

[30]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[31]  Constantin Zalinescu,et al.  Set-valued Optimization - An Introduction with Applications , 2014, Vector Optimization.

[32]  Annabella Astorino,et al.  Margin maximization in spherical separation , 2012, Computational Optimization and Applications.

[33]  Theodore B. Trafalis,et al.  Robust support vector machines for classification and computational issues , 2007, Optim. Methods Softw..

[34]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[35]  Michael I. Jordan,et al.  Robust Sparse Hyperplane Classifiers: Application to Uncertain Molecular Profiling Data , 2004, J. Comput. Biol..

[36]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[37]  B. Schölkopf,et al.  Advances in kernel methods: support vector learning , 1999 .

[38]  Michal Kočvara,et al.  Nonsmooth approach to optimization problems with equilibrium constraints : theory, applications, and numerical results , 1998 .

[39]  Chiranjib Bhattacharyya,et al.  Efficient methods for robust classification under uncertainty in kernel matrices , 2012, J. Mach. Learn. Res..

[40]  Laurent El Ghaoui,et al.  Robust Solutions to Uncertain Semidefinite Programs , 1998, SIAM J. Optim..

[41]  Annabella Astorino,et al.  Scaling Up Support Vector Machines Using Nearest Neighbor Condensation , 2010, IEEE Transactions on Neural Networks.

[42]  Jian Xun Peng,et al.  A sequential algorithm for sparse support vector classifiers , 2013, Pattern Recognit..

[43]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[44]  Annabella Astorino,et al.  Edge detection by spherical separation , 2013, Computational Management Science.

[45]  Allen L. Soyster,et al.  Technical Note - Convex Programming with Set-Inclusive Constraints and Applications to Inexact Linear Programming , 1973, Oper. Res..

[46]  Ralf Herbrich,et al.  Adaptive margin support vector machines for classification , 1999 .

[47]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[48]  Annabella Astorino,et al.  DC models for spherical separation , 2010, J. Glob. Optim..

[49]  Chiranjib Bhattacharyya,et al.  Interval Data Classification under Partial Information: A Chance-Constraint Approach , 2009, PAKDD.

[50]  Theodore B. Trafalis,et al.  Robust classification and regression using support vector machines , 2006, Eur. J. Oper. Res..

[51]  Arkadi Nemirovski,et al.  Robust solutions of Linear Programming problems contaminated with uncertain data , 2000, Math. Program..

[52]  Annabella Astorino,et al.  Non-smoothness in classification problems , 2008, Optim. Methods Softw..

[53]  Kristin P. Bennett,et al.  Fast Bundle Algorithm for Multiple-Instance Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Michael I. Jordan,et al.  A Robust Minimax Approach to Classification , 2003, J. Mach. Learn. Res..

[55]  Rafail N. Gasimov,et al.  Separation via polyhedral conic functions , 2006, Optim. Methods Softw..

[56]  Annabella Astorino,et al.  A fixed-center spherical separation algorithm with kernel transformations for classification problems , 2009, Comput. Manag. Sci..

[57]  Kristin P. Bennett,et al.  Model selection for primal SVM , 2011, Machine Learning.

[58]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[59]  Antonio Fuduli,et al.  Minimizing Nonconvex Nonsmooth Functions via Cutting Planes and Proximity Control , 2003, SIAM J. Optim..

[60]  Emilio Carrizosa,et al.  Two-group classification via a biobjective margin maximization model , 2006, Eur. J. Oper. Res..