Signal theory for SVM kernel design with applications to parameter estimation and sequence kernels

Fourier-based regularisation is considered for the support vector machine (SVM) classification problem over absolutely integrable loss functions. By considering the problem in a signal theory setting, we show that a principled and finite kernel hyperparameter search space can be discerned a priori by using the sinc kernel. The training and validation phase required to optimise the SVM can thus be limited to this hyperparameter search space. The method is adapted to a recently proposed max sequence kernel such that positive semi-definiteness, and so convergence, is guaranteed.

[1]  Gérard Chollet,et al.  Swiss French PolyPhone and PolyVar: telephone speech databases to model inter- and intra-speaker variability , 1996 .

[2]  David A. Landgrebe,et al.  Hyperspectral image data analysis , 2002, IEEE Signal Process. Mag..

[3]  Hakan Melin,et al.  DATABASES FOR SPEAKER RECOGNITION: ACTIVITIES IN COST250 WORKING GROUP 2 , 1999 .

[4]  Peng Du,et al.  SELF ADAPTIVE SUPPORT VECTOR MACHINES AND AUTOMATIC FEATURE SELECTION , 2004 .

[5]  B. Guo,et al.  Hyperspectral image fusion using spectrally weighted kernels , 2005, 2005 7th International Conference on Information Fusion.

[6]  Douglas A. Reynolds,et al.  Robust text-independent speaker identification using Gaussian mixture speaker models , 1995, IEEE Trans. Speech Audio Process..

[7]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[8]  Robert I. Damper,et al.  Improving speaker identification in noise by subband processing and decision fusion , 2003, Pattern Recognit. Lett..

[9]  Rameswar Debnath,et al.  Analyzing the Behavior of Distribution of Data in the Feature Space of SVM with Gaussian Kernel , 2004 .

[10]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[11]  Samy Bengio,et al.  A Max Kernel For Text-Independent Speaker Verification Systems , 2006 .

[12]  Bernhard Schölkopf,et al.  The connection between regularization operators and support vector kernels , 1998, Neural Networks.

[13]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[14]  Saldju Tadjudin,et al.  CLASSIFICATION OF HIGH DIMENSIONAL DATA WITH LIMITED TRAINING SAMPLES , 1998 .

[15]  Tatsuya Akutsu,et al.  Protein homology detection using string alignment kernels , 2004, Bioinform..

[16]  N. Higham.  COMPUTING A NEAREST SYMMETRIC POSITIVE SEMIDEFINITE MATRIX , 1988 .

[17]  J. Mercer.  Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[18]  James D. B. Nelson,et al.  The construction of some Riesz basis families and their application to coe-cient quantization, sampling theory, and wavelet analysis. , 2001 .

[19]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[20]  Robert I. Damper,et al.  Band Selection for Hyperspectral Image Classification Using Mutual Information , 2006, IEEE Geoscience and Remote Sensing Letters.

[21]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[22]  William M. Campbell,et al.  Support vector machines for speaker and language recognition , 2006, Comput. Speech Lang..

[23]  J. Anthony Gualtieri,et al.  Support vector machines for hyperspectral remote sensing classification , 1999, Other Conferences.