Design of a modified one-against-all SVM classifier

Support Vector Machine (SVM) is one of the state of-the-art tools for linear and nonlinear pattern classification. One of the design issues in SVM classifier is reducing the number of support vectors without compromising the classification accuracy. In this paper, a novel technique which requires only a subset of the support vectors is proposed. The subset is obtained by including only those support vectors for which Lagrange multiplier is greater than a threshold. In order to find the subset which yields the highest classification accuracy with the least number of support vectors in the subset, the recognition performance corresponding to subsets with different threshold values are to be evaluated and compared. The proposed technique is applied for SVM based isolated digit recognition system and is studied using speaker dependent as well as multispeaker dependent TI46 database of isolated digits. Two feature extraction techniques, one using LPC and another using MFCC are applied to the speech from the above database and the features are mapped using SOFM. This in turn is used by the SVM classifier to evaluate the recognition accuracy. The proposed technique is applied to One-Against-All (OAA) scheme and is denoted as Modified One-Against-All (M-OAA) approach in this paper. Based on this study, it is found that for MFCC feature input, the proposed M-OAA based SVM classifier approach results in reduction of support vectors by a factor of 1.86 to 18.3 with no compromise in recognition accuracy. For LPC feature input, the M-OAA based SVM classifier results in reduction of support vectors by a factor of 1.59 to 2.52 without any compromise in recognition accuracy for some cases and with a maximum of 1% degradation in recognition accuracy for some cases. The proposed approach is also applicable for other schemes such as Half-Against-Half (HAH) and Directed Acyclic Graphs (DAG) based SVM classifiers as well as for any other classification problem such as face recognition, fingerprint recognition, target recognition, speaker recognition and speaker verification.

[1]  Anthony Kuh,et al.  A combined self-organizing feature map and multilayer perceptron for isolated word recognition , 1992, IEEE Trans. Signal Process..

[2]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[3]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Xindong Wu,et al.  Support vector machines based on K-means clustering for real-time business intelligence systems , 2005, Int. J. Bus. Intell. Data Min..

[5]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[6]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[7]  D. Howard,et al.  Speech and audio signal processing: processing and perception of speech and music [Book Review] , 2000 .

[8]  José Luis Rojo-Álvarez,et al.  Support vector machines for robust channel estimation in OFDM , 2006, IEEE Signal Processing Letters.

[9]  Yan Liu,et al.  A novel two-step SVM classifier for voiced/unvoiced/silence classification of speech , 2004, 2004 International Symposium on Chinese Spoken Language Processing.

[10]  Kurt Hornik,et al.  The support vector machine under test , 2003, Neurocomputing.

[11]  Martin T. Hagan,et al.  Neural network design , 1995 .

[12]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[13]  C.G. Christodoulou,et al.  Beamforming using support vector machines , 2005, IEEE Antennas and Wireless Propagation Letters.

[14]  Li Zhang,et al.  Wavelet support vector machine , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[16]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[17]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[18]  Qun Zhao,et al.  Support vector machines for SAR automatic target recognition , 2001 .

[19]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[20]  Xu Zhu,et al.  Support Vector Machines for DS-UWB Channel Equalisation , 2007, 2007 International Conference on Wireless Communications, Networking and Mobile Computing.

[21]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[22]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[23]  Q. Henry Wu,et al.  Online training of support vector classifier , 2003, Pattern Recognit..