Wavelet Kernel Principal Component Analysis in Noisy Multiscale Data Classification

We introduce multiscale wavelet kernels to kernel principal component analysis (KPCA) to narrow down the search of parameters required in the calculation of a kernel matrix. This new methodology incorporates multiscale methods into KPCA for transforming multiscale data. In order to illustrate application of our proposed method and to investigate the robustness of the wavelet kernel in KPCA under different levels of the signal to noise ratio and different types of wavelet kernel, we study a set of two-class clustered simulation data. We show that WKPCA is an effective feature extraction method for transforming a variety of multidimensional clustered data into data with a higher level of linearity among the data attributes. That brings an improvement in the accuracy of simple linear classifiers. Based on the analysis of the simulation data sets, we observe that multiscale translation invariant wavelet kernels for KPCA has an enhanced performance in feature extraction. The application of the proposed method to real data is also addressed.

[1]  Yinliang Zhao,et al.  Least Squares Support Vector Machine on Gaussian Wavelet Kernel Function Set , 2006, ISNN.

[2]  Lasse Holmström,et al.  Bayesian Multiscale Smoothing for Making Inferences About Features in Scatterplots , 2005 .

[3]  Stéphane Canu,et al.  Frames, Reproducing Kernels, Regularization and Learning , 2005, J. Mach. Learn. Res..

[4]  Kurt Hornik,et al.  kernlab - An S4 Package for Kernel Methods in R , 2004 .

[5]  Xiao-Feng Liao,et al.  A Multiresolution Wavelet Kernel for Support Vector Regression , 2006, ISNN.

[6]  Hong Yan,et al.  Framelet Kernels With Applications to Support Vector Regression and Regularization Networks , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[8]  Roland Opfer,et al.  Multiscale kernels , 2006, Adv. Comput. Math..

[9]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[10]  Shubha Kadambe,et al.  Adaptive wavelets for signal classification and compression , 2006 .

[11]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[12]  M. Palaniswami,et al.  Multi-Scale Kernel Methods for Classification , 2005, 2005 IEEE Workshop on Machine Learning for Signal Processing.

[13]  Jun Wang,et al.  Multi-scale Wavelet Support Vector Regression for Soft Sensor Modeling , 2005, 2005 International Conference on Neural Networks and Brain.

[14]  Jian-Huang Lai,et al.  Wavelet Kernel Construction for Kernel Discriminant Analysis on Face Recognition , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[15]  Dirk P. Kroese,et al.  Statistical Learning , 2001, Science.

[16]  Hsuen Chyun Shyu,et al.  Underwater acoustic signal analysis by multiscaling and multitranslation wavelets , 1998, Defense, Security, and Sensing.

[17]  Mu Zhu,et al.  Kernels and Ensembles : Perspectives on Statistical Learning , 2008 .

[18]  Tetsuya Takiguchi,et al.  Robust Feature Extraction using Kernel PCA , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[19]  Andrzej Cichocki,et al.  Kernel PCA for Feature Extraction and De-Noising in Nonlinear Regression , 2001, Neural Computing & Applications.

[20]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[21]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[22]  Li Zhang,et al.  Wavelet support vector machine , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[24]  F. Leisch.  FlexMix: A general framework for finite mixture models and latent class regression in R , 2004 .

[25]  Yu Fang,et al.  Multiscale Wavelet Support Vector Regression for Traffic Flow Prediction , 2009, 2009 Third International Symposium on Intelligent Information Technology Application.

[26]  Hui Cheng,et al.  Super-resolution image reconstruction based on MWSVR estimation , 2008, 2008 7th World Congress on Intelligent Control and Automation.

[27]  Boonserm Kijsirikul,et al.  Evolutionary strategies for multi-scale radial basis function kernels in support vector machines , 2005, GECCO '05.

[28]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[29]  R. Tibshirani,et al.  Flexible Discriminant Analysis by Optimal Scoring , 1994 .