Geometric Algorithms to Large Margin Classifier Based on Affine Hulls

The geometric framework for binary data classification problems provides an intuitive foundation for the comprehension and application of geometric optimization algorithms, leading to practical solutions of real-world classification problems. In this paper, some theoretical results on the candidate extreme points of the notion of reduced affine hull (RAH) are introduced. These results allow the existing nearest point algorithms to be directly applied to solve both separable and inseparable classification problems based on RAHs successfully and efficiently. As the practical applications of the new theoretical results, the popular Gilbert-Schlesinger-Kozinec and Mitchell-Dem'yanov-Malozemov algorithms are presented to solve binary classification problems in the context of the RAH framework. The theoretical analysis and some experiments show that the proposed methods successfully achieve significant performance.

[1]  Jue Wang,et al.  A general soft method for learning SVM classifiers with L1-norm penalty , 2008, Pattern Recognit..

[2]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[3]  Yong Shi,et al.  A New Kernel-Based Classification Algorithm , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[4]  David J. Crisp,et al.  A Geometric Interpretation of ?-SVM Classifiers , 1999, NIPS 2000.

[5]  Hakan Cevikalp,et al.  Large margin classifiers based on convex class models , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[6]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[8]  Zhong Chen,et al.  A generalized Gilbert's algorithm for approximating general SVM classifiers , 2009, Neurocomputing.

[9]  V. N. Malozemov,et al.  Finding the Point of a Polyhedron Closest to the Origin , 1974 .

[10]  Shi Yong,et al.  Affine Subspace Nearest Points Classification Algorithm for Wavelet Face Recognition , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[11]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[12]  Yujian Li,et al.  Multiconlitron: A General Piecewise Linear Classifier , 2011, IEEE Transactions on Neural Networks.

[13]  Hakan Cevikalp,et al.  Large margin classifiers based on affine hulls , 2010, Neurocomputing.

[14]  Stevan M. Berber,et al.  Blind Multiuser Detector for Chaos-Based CDMA Using Support Vector Machine , 2010, IEEE Transactions on Neural Networks.

[15]  Yatong Zhou,et al.  Analysis of the Distance Between Two Classes for Tuning SVM Hyperparameters , 2010, IEEE Transactions on Neural Networks.

[16]  Sergios Theodoridis,et al.  A geometric approach to Support Vector Machine (SVM) classification , 2006, IEEE Transactions on Neural Networks.

[17]  Peter Williams,et al.  A Geometrical Method to Improve Performance of the Support Vector Machine , 2007, IEEE Transactions on Neural Networks.

[18]  Nikolas P. Galatsanos,et al.  A support vector machine approach for detection of microcalcifications , 2002, IEEE Transactions on Medical Imaging.

[19]  Guoyou Wang,et al.  A Novel Geometric Approach to Binary Classification Based on Scaled Convex Hulls , 2009, IEEE Transactions on Neural Networks.

[20]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[21]  Sergios Theodoridis,et al.  A Geometric Nearest Point Algorithm for the Efficient Solution of the SVM Classification Task , 2007, IEEE Transactions on Neural Networks.

[22]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[24]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[25]  José R. Dorronsoro,et al.  On the Equivalence of the SMO and MDM Algorithms for SVM Training , 2008, ECML/PKDD.

[26]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[27]  E. Gilbert.  An Iterative Procedure for Computing the Minimum of a Quadratic Form on a Convex Set , 1966 .

[28]  Jinbo Bi,et al.  A geometric approach to support vector regression , 2003, Neurocomputing.

[29]  José R. Dorronsoro,et al.  A Common Framework for the Convergence of the GSK, MDM and SMO Algorithms , 2010, ICANN.

[30]  Hakan Cevikalp,et al.  Nearest hyperdisk methods for high-dimensional classification , 2008, ICML '08.

[31]  Li Yujian,et al.  Multiconlitron: a general piecewise linear classifier. , 2011, IEEE transactions on neural networks.

[32]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[33]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[34]  Václav Hlavác,et al.  Ten Lectures on Statistical and Structural Pattern Recognition , 2002, Computational Imaging and Vision.

[35]  José R. Dorronsoro,et al.  Clipping algorithms for solving the nearest point problem over reduced convex hulls , 2011, Pattern Recognit..

[36]  Yang Jing-yu.  Kernel Affine Subspace Nearest Points Classification Algorithm , 2008 .

[37]  Václav Hlavác,et al.  An iterative algorithm learning the maximal margin classifier , 2003, Pattern Recognit..

[38]  Yifei Wang,et al.  The robust and efficient adaptive normal direction support vector regression , 2011, Expert Syst. Appl..

[39]  Jue Wang,et al.  A generalized S-K algorithm for learning v-SVM classifiers , 2004, Pattern Recognit. Lett..

[40]  Eduardo Bayro-Corrochano,et al.  Clifford Support Vector Machines for Classification, Regression, and Recurrence , 2010, IEEE Transactions on Neural Networks.

[41]  Kristin P. Bennett,et al.  Duality and Geometry in SVM Classifiers , 2000, ICML.

[42]  Yifei Wang,et al.  CCH-based geometric algorithms for SVM and applications , 2009 .

[43]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .