Statistical Learning Theory and Support Vector Machines

It has been more than 30 years that statistical learning theory (SLT) has been introduced in the field of machine learning. Its objective is to provide a framework for studying the problem of inference that is of gaining knowledge, making predictions, making decisions or constructing models from a set of data. Support Vector Machine, a method based on SLT, then emerged and becoming a widely accepted method for solving real-world problems. This paper overviews the pattern recognition techniques and describes the state of art in SVM in the field of pattern recognition.

[1]  Ching Y. Suen,et al.  A trainable feature extractor for handwritten digit recognition , 2007, Pattern Recognit..

[2]  Leonid I. Perlovsky,et al.  Conundrum of Combinatorial Complexity , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  David Haussler,et al.  Exploiting Generative Models in Discriminative Classifiers , 1998, NIPS.

[4]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[5]  Gábor Lugosi,et al.  Introduction to Statistical Learning Theory , 2004, Advanced Lectures on Machine Learning.

[6]  D. C. Shubhangi.  Noisy english character recognition by combining SVM classifier , 2007 .

[7]  William Stafford Noble,et al.  Support vector machine , 2013 .

[8]  Pedro J. Moreno,et al.  On the use of support vector machines for phonetic classification , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[9]  Hervé Bourlard,et al.  Connectionist Speech Recognition: A Hybrid Approach , 1993 .

[10]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[11]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[12]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[13]  Yixin Zhong,et al.  Statistical learning theory and state of the art in SVM , 2003, The Second IEEE International Conference on Cognitive Informatics, 2003. Proceedings..

[14]  Muhammad Sarfraz.  Computer-Aided Intelligent Recognition Techniques and Applications: Sarfraz/Computer-Aided Intelligent Recognition Techniques and Applications , 2005 .

[15]  Bernhard Schölkopf,et al.  On a Kernel-Based Method for Pattern Recognition, Regression, Approximation, and Operator Inversion , 1998, Algorithmica.

[16]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[17]  Shantanu Chakrabartty,et al.  Forward-decoding kernel-based phone sequence recognition , 2002, NIPS 2002.

[18]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[19]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[20]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[21]  Marimuthu Palaniswami,et al.  Machine learning using support vector machines , 2000 .

[22]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .

[23]  Dusan Cakmakov,et al.  Handwritten digit recognition by combining support vector machines using rule-based reasoning , 2001, Proceedings of the 23rd International Conference on Information Technology Interfaces, 2001. ITI 2001..

[24]  Gert Cauwenberghs,et al.  Charge-mode parallel architecture for matrix-vector multiplication , 2000, Proceedings of the 43rd IEEE Midwest Symposium on Circuits and Systems (Cat.No.CH37144).

[25]  Bernhard Schölkopf,et al.  Improving the accuracy and speed of support vector learning machines , 1997, NIPS 1997.

[26]  Patrick Gallinari,et al.  An MLP-SVM combination architecture for offline handwritten digit recognition , 2003, Document Analysis and Recognition.

[27]  Kristin P. Bennett,et al.  Support vector machines: hype or hallelujah? , 2000, SKDD.

[28]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[29]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[30]  David Haussler,et al.  Probabilistic kernel regression models , 1999, AISTATS.

[31]  Enrique Alba,et al.  Training Neural Networks with GA Hybrid Algorithms , 2004, GECCO.

[32]  V. Vapnik.  Pattern recognition using generalized portrait method , 1963 .

[33]  G. Cauwenberghs,et al.  A low-power CMOS analog vector quantizer , 1997 .

[34]  Gert Cauwenberghs,et al.  Forward Decoding Kernel Machines: A Hybrid HMM/SVM Approach to Sequence Recognition , 2002, SVM.

[35]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[36]  John Shawe-Taylor,et al.  Generalization Performance of Support Vector Machines and Other Pattern Classifiers , 1999 .

[37]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[38]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[39]  D. Mackay,et al.  Introduction to Gaussian processes , 1998 .

[40]  M. Sarfraz.  Computer-Aided Intelligent Recognition Techniques and Applications , 2005 .

[41]  Mark A. Girolami,et al.  Mercer kernel-based clustering in feature space , 2002, IEEE Trans. Neural Networks.

[42]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[43]  Ping Li,et al.  Performance analysis and comparison of neural networks and support vector machines classifier , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[44]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  M. Khalid,et al.  Kernel methods and support vector machines for handwriting recognition , 2002, Student Conference on Research and Development.

[46]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1993 .

[47]  Jin-Kao Hao,et al.  A Hybrid GA/SVM Approach for Gene Selection and Classification of Microarray Data , 2006, EvoWorkshops.

[48]  P. Bartlett,et al.  Gaussian Processes and SVM: Mean Field and Leave-One-Out , 2000 .

[49]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[50]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.