Comparison of Support Vector Machine and Softmax Classifiers in Computer Vision

Support Vector Machine and Softmax are two widely used linear classifiers in computer vision. Especially in the field of deep learning algorithms, the application of these two classifiers is more frequent. In addition, in the field of statistics, speech recognition, character recognition and other aspects that Support Vector Machine and Softmax also have been used. However, there are still some controversies of these two classifiers in the specific implementations. And the understanding of them comes to a significance place in deep learning process. This paper will make a comparison and analysis of the two classifiers from a holistic perspective to help the reader have a more comprehensive understanding of the two classifiers.

[1]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[2]  Claus Nebauer,et al.  Evaluation of convolutional neural networks for visual recognition , 1998, IEEE Trans. Neural Networks.

[3]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[5]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Claude E. Shannon,et al.  The Mathematical Theory of Communication , 1950 .

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[9]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Thomas Bahr,et al.  APPLICATION OF SOFTMAX REGRESSION AND ITS VALIDATION FOR SPECTRAL-BASED LAND COVER MAPPING , 2017 .

[12]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[13]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[14]  Harris Drucker,et al.  Learning algorithms for classification: A comparison on handwritten digit recognition , 1995 .

[15]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[16]  Pascal Vincent,et al.  Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , 2011, ICML.

[17]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.