Fast and convergence-guaranteed algorithm for linear separation

Efficient linear separation algorithms are important for pattern classification applications. In this paper, an algorithm is developed to solve linear separation problems in n-dimensional space. Its convergence feature is proved. The proposed algorithm is proved to converge to a correct solution whenever the two sets are separable. The complexity of the proposed algorithm is analyzed, and experiments on both randomly generated examples and real application problems were carried out. While analysis shows that its time complexity is lower than SVM that needs computations for quadratic programming optimization, experiment results show that the developed algorithm is more efficient than the least-mean-square (LMS), and the Perceptron.

[1]  Li Wei.  A mathematic-physical approach to the satisfiability problem , 1995 .

[2]  Sun-Yuan Kung,et al.  Digital neural networks , 1993, Prentice Hall Information and System Sciences Series.

[3]  David Zhang,et al.  The Negative Effects of Whitening Transformation in Face Recognition , 2009, 2009 Second International Symposium on Computational Intelligence and Design.

[4]  Bernard Widrow,et al.  On the statistical efficiency of the LMS algorithm with nonstationary inputs , 1984, IEEE Trans. Inf. Theory.

[5]  David G. Stork,et al.  Pattern Classification , 1973 .

[6]  David Zhang,et al.  Wrist pulse signal diagnosis using modified Gaussian models and Fuzzy C-Means classification. , 2009, Medical engineering & physics.

[7]  Sung-Kwon Park,et al.  The geometrical learning of binary neural networks , 1995, IEEE Trans. Neural Networks.

[8]  John Zukowski,et al.  Parallel VLSI neural system design , 1998 .

[9]  David Zhang,et al.  Face Recognition Based on Nonlinear DCT Discriminant Feature Extraction Using Improved Kernel DCV , 2009, IEICE Trans. Inf. Syst..

[11]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[12]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[13]  David Zhang,et al.  Automated tongue segmentation in hyperspectral images for medicine. , 2007, Applied optics.

[14]  David Zhang,et al.  A Novel Subspace-Based Facial Discriminant Feature Extraction Method , 2009, 2009 Chinese Conference on Pattern Recognition.

[15]  Zhenhua Guo,et al.  Palmprint verification using consistent orientation coding , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[16]  Fumio Harashima,et al.  IEEE International Conference on Systems, Man, and Cybernetics , 2000 .

[17]  Ivor W. Tsang,et al.  Core Vector Machines: Fast SVM Training on Very Large Data Sets , 2005, J. Mach. Learn. Res..

[18]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[19]  David Zhang,et al.  Principal line based ICP alignment for palmprint verification , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[20]  Wenqi Huang,et al.  A Quasi-mechanical Method for Solving the Rectangle Covering Problem An Approach to Tackling NP Hard Problems , 1994, CVGIP Graph. Model. Image Process..