Wavelet kernel learning

This paper addresses the problem of optimal feature extraction from a wavelet representation. Our work aims at building features by selecting wavelet coefficients resulting from signal or image decomposition on an adapted wavelet basis. For this purpose, we jointly learn in a kernelized large-margin context the wavelet shape as well as the appropriate scale and translation of the wavelets, hence the name ''wavelet kernel learning''. This problem is posed as a multiple kernel learning problem, where the number of kernels can be very large. For solving such a problem, we introduce a novel multiple kernel learning algorithm based on active constraints methods. We furthermore propose some variants of this algorithm that can produce approximate solutions more efficiently. Empirical analysis show that our active constraint MKL algorithm achieves state-of-the art efficiency. When used for wavelet kernel learning, our experimental results show that the approaches we propose are competitive with respect to the state-of-the-art on brain-computer interface and Brodatz texture datasets.

[1]  Tae Jin Kang,et al.  Texture classification and segmentation using wavelet packet frame and Gaussian mixture model , 2007, Pattern Recognit..

[2]  Alain Rakotomamonjy,et al.  BCI Competition III: Dataset II- Ensemble of SVMs for BCI P300 Speller , 2008, IEEE Transactions on Biomedical Engineering.

[3]  Gabriele Steidl,et al.  Feature extraction by shape-adapted local discriminant bases , 2003, Signal Process..

[4]  Mehryar Mohri,et al.  L2 Regularization for Learning Kernels , 2009, UAI.

[5]  H TewfikAhmed,et al.  Extraction subject-specific motor imagery time-frequency patterns for single trial EEG classification , 2007 .

[6]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[7]  Adrian Smith,et al.  Bayesian Assessment of Network Reliability , 1998, SIAM Rev..

[8]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[9]  David L. Donoho,et al.  WaveLab and Reproducible Research , 1995 .

[10]  Barry G. Sherlock,et al.  On the space of orthonormal wavelets , 1998, IEEE Trans. Signal Process..

[11]  Francis R. Bach,et al.  Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning , 2008, NIPS.

[12]  Georg Regensburger,et al.  Parametrizing compactly supported orthonormal wavelets by discrete moments , 2007, Applicable Algebra in Engineering, Communication and Computing.

[13]  David L. Donoho,et al.  Improved linear discrimination using time-frequency dictionaries , 1995, Optics + Photonics.

[14]  Sadik Kara,et al.  Atrial fibrillation classification with artificial neural networks , 2007, Pattern Recognit..

[15]  O. Chapelle.  Second order optimization of kernel parameters , 2008 .

[16]  Wageeh Boles,et al.  Texture Classification Using Multiple Wavelet Analysis , 2002 .

[17]  Stphane Mallat,et al.  A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way , 2008 .

[18]  Klaus-Robert Müller,et al.  Efficient and Accurate Lp-Norm Multiple Kernel Learning , 2009, NIPS.

[19]  Ahmed H. Tewfik,et al.  Extraction subject-specific motor imagery time-frequency patterns for single trial EEG classification , 2007, Comput. Biol. Medicine.

[20]  Robert Bregovic,et al.  Multirate Systems and Filter Banks , 2002 .

[21]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[22]  William Stafford Noble,et al.  Support vector machine learning from heterogeneous data: an empirical analysis using protein sequence and structure , 2006, Bioinform..

[23]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[24]  Denis Vautrin,et al.  A Novel Criterion of Wavelet Packet Best Basis Selection for Signal Classification With Application to Brain–Computer Interfaces , 2009, IEEE Transactions on Biomedical Engineering.

[25]  Charles A. Micchelli,et al.  A DC-programming algorithm for kernel selection , 2006, ICML.

[26]  Zenglin Xu,et al.  An Extended Level Method for Efficient Multiple Kernel Learning , 2008, NIPS.

[27]  Yves Grandvalet,et al.  Composite kernel learning , 2008, ICML '08.

[28]  S. Sathiya Keerthi,et al.  A simple and efficient algorithm for gene selection using sparse logistic regression , 2003, Bioinform..

[29]  Alexander Shapiro,et al.  Optimization Problems with Perturbations: A Guided Tour , 1998, SIAM Rev..

[30]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[31]  Marie-Françoise Lucas,et al.  Multi-channel surface EMG classification using support vector machines and signal-based wavelet optimization , 2008, Biomed. Signal Process. Control..

[32]  G. Steidl,et al.  Hybrid wavelet-support vector classification of waveforms , 2002 .

[33]  Mohammad Hassan Moradi,et al.  Quantitative evaluation of a wavelet-based method in ventricular late potential detection , 2006, Pattern Recognit..

[34]  Ronald R. Coifman,et al.  Local discriminant bases and their applications , 1995, Journal of Mathematical Imaging and Vision.

[35]  Kim Dremstrup,et al.  Auditory and spatial navigation imagery in Brain–Computer Interface using optimized wavelets , 2008, Journal of Neuroscience Methods.

[36]  Sebastian Nowozin,et al.  Infinite Kernel Learning , 2008, NIPS 2008.

[37]  Elif Derya Übeyli,et al.  ECG beat classifier designed by combined neural network model , 2005, Pattern Recognit..

[38]  Gabriele Steidl,et al.  Efficient wavelet adaptation for hybrid wavelet-large margin classifiers , 2005, Pattern Recognit..

[39]  Zenglin Xu,et al.  Simple and Efficient Multiple Kernel Learning by Group Lasso , 2010, ICML.

[40]  Yaonan Wang,et al.  Texture classification using the support vector machines , 2003, Pattern Recognit..

[41]  Marie-Françoise Lucas,et al.  Optimization of wavelets for classification of movement-related cortical potentials generated by variation of force-related parameters , 2007, Journal of Neuroscience Methods.

[42]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..