A Novel Geometric Approach to Binary Classification Based on Scaled Convex Hulls

Geometric methods are very intuitive and provide a theoretical foundation to many optimization problems in the fields of pattern recognition and machine learning. In this brief, the notion of scaled convex hull (SCH) is defined and a set of theoretical results are exploited to support it. These results allow the existing nearest point algorithms to be directly applied to solve both the separable and nonseparable classification problems successfully and efficiently. Then, the popular S-K algorithm has been presented to solve the nonseparable problems in the context of the SCH framework. The theoretical analysis and some experiments show that the proposed method may achieve better performance than the state-of-the-art methods in terms of the number of kernel evaluations and the execution time.

[1]  Swaroop Darbha,et al.  Dynamic surface control for a class of nonlinear systems , 2000, IEEE Trans. Autom. Control..

[2]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[3]  Jin Bae Park,et al.  Indirect adaptive control of nonlinear dynamic systems using self recurrent wavelet neural networks via adaptive learning rates , 2007, Inf. Sci..

[4]  Junmin Li,et al.  Adaptive neural control for a class of nonlinearly parametric time-delay systems , 2005, IEEE Transactions on Neural Networks.

[5]  Sergios Theodoridis,et al.  A novel SVM Geometric Algorithm based on Reduced Convex Hulls , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[6]  P. P. Yip,et al.  Adaptive dynamic surface control : a simplified algorithm for adaptive backstepping control of nonlinear systems , 1998 .

[7]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[8]  S. Ge,et al.  Adaptive neural control of nonlinear time-delay systems with unknown virtual control coefficients , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[9]  David J. Crisp,et al.  A Geometric Interpretation of v-SVM Classifiers , 1999, NIPS.

[10]  David J. Crisp,et al.  A Geometric Interpretation of ?-SVM Classifiers , 1999, NIPS 2000.

[12]  Keng Peng Tee,et al.  Approximation-based control of nonlinear MIMO time-delay systems , 2007, Autom..

[13]  Václav Hlavác,et al.  An iterative algorithm learning the maximal margin classifier , 2003, Pattern Recognit..

[14]  Sergios Theodoridis,et al.  A Geometric Nearest Point Algorithm for the Efficient Solution of the SVM Classification Task , 2007, IEEE Transactions on Neural Networks.

[15]  Jun Zhang,et al.  Wavelet neural networks for function learning , 1995, IEEE Trans. Signal Process..

[16]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[17]  I. Kanellakopoulos,et al.  Systematic Design of Adaptive Controllers for Feedback Linearizable Systems , 1991, 1991 American Control Conference.

[18]  M. Polycarpou,et al.  Stable adaptive tracking of uncertain systems using nonlinearly parametrized on-line approximators , 1998 .

[19]  Kristin P. Bennett,et al.  Duality and Geometry in SVM Classifiers , 2000, ICML.

[20]  Shuzhi Sam Ge,et al.  Practical adaptive neural control of nonlinear systems with unknown time delays , 2005, Proceedings of the 2004 American Control Conference.

[21]  Gérard Dreyfus,et al.  Training wavelet networks for nonlinear dynamic input-output modeling , 1998, Neurocomputing.

[22]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[23]  Jue Wang,et al.  A general soft method for learning SVM classifiers with L1-norm penalty , 2008, Pattern Recognit..

[24]  Shuzhi Sam Ge,et al.  Adaptive neural network control of nonlinear systems with unknown time delays , 2003, IEEE Trans. Autom. Control..

[25]  Wei Lin,et al.  Adaptive control of nonlinearly parameterized systems: the smooth feedback case , 2002, IEEE Trans. Autom. Control..

[26]  Sergios Theodoridis,et al.  A geometric approach to Support Vector Machine (SVM) classification , 2006, IEEE Transactions on Neural Networks.

[27]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[28]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .