KIMEL: A kernel incremental metalearning algorithm

The Kernel method is a powerful tool for extending an algorithm from linear to nonlinear case. Metalearning algorithm learns the base learning algorithm, thus to improve performance of the learning system. Usually, metalearning algorithms exhibit faster convergence rate and lower Mean-Square Error (MSE) than the corresponding base learning algorithms. In this paper, we present a kernelized metalearning algorithm, named KIMEL, which is a metalearning algorithm in the Reproducing Kernel Hilbert Space (RKHS). The convergence analyses of the KIMEL algorithm are performed in detail. To demonstrate the effectiveness and advantage of the proposed algorithm, we firstly apply the algorithm to a simple example of nonlinear channel equalization. Then we focus on a more practical application in blind Image Quality Assessment (IQA). Experimental results show that the KIMEL algorithm has superior performance.

[1]  Xiaojun Wu,et al.  Blind Image Quality Assessment Using a General Regression Neural Network , 2011, IEEE Transactions on Neural Networks.

[2]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Ali H. Sayed,et al.  Variable step-size NLMS and affine projection algorithms , 2004, IEEE Signal Processing Letters.

[4]  Robyn A. Owens,et al.  Feature detection from local energy , 1987, Pattern Recognit. Lett..

[5]  Ingrid Heynderickx,et al.  Objective quality assessment of displayed images by using neural networks , 2005, Signal Process. Image Commun..

[6]  Christophe Charrier,et al.  A DCT Statistics-Based Blind Image Quality Index , 2010, IEEE Signal Processing Letters.

[7]  Zhou Wang,et al.  No-reference perceptual quality assessment of JPEG compressed images , 2002, Proceedings. International Conference on Image Processing.

[8]  B. Widrow,et al.  Adaptive antenna systems , 1967 .

[9]  Truong Q. Nguyen,et al.  No-reference image quality assessment using structural activity , 2011, Signal Process..

[10]  Zhigang Luo,et al.  Non-Negative Patch Alignment Framework , 2011, IEEE Transactions on Neural Networks.

[11]  Xuelong Li,et al.  A natural image quality evaluation metric , 2009, Signal Process..

[12]  T. Aboulnasr,et al.  A robust variable step size LMS-type algorithm: analysis and simulations , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[13]  Weifeng Liu,et al.  The Kernel Least-Mean-Square Algorithm , 2008, IEEE Transactions on Signal Processing.

[14]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[15]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[16]  Mohamed M. Fouad,et al.  Image Compression Technique for Low Bit Rate Transmission , 2011, Int. J. Comput. Vis. Image Process..

[17]  M. Arbib,et al.  A model of cerebellar metaplasticity. , 1998, Learning & memory.

[18]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[19]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[20]  R. SheikhH.,et al.  No-reference quality assessment using natural scene statistics , 2005 .

[21]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[22]  W. Y. Chen,et al.  A variable step size LMS algorithm , 1990, Proceedings of the 33rd Midwest Symposium on Circuits and Systems.

[23]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[24]  Xuelong Li,et al.  Reduced-Reference IQA in Contourlet Domain , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Peter Kovesi,et al.  Image Features from Phase Congruency , 1995 .

[26]  Weifeng Liu,et al.  Kernel Adaptive Filtering: A Comprehensive Introduction , 2010 .

[27]  Tiago Rosa Maria Paula Queluz,et al.  No-reference image quality assessment based on DCT domain statistics , 2008, Signal Process..

[28]  John A. Saghri,et al.  Image Quality Measure Based On A Human Visual System Model , 1989 .

[29]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[30]  Bernard Widrow,et al.  The least mean fourth (LMF) adaptive algorithm and its family , 1984, IEEE Trans. Inf. Theory.

[31]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[32]  Jeng-Kuang Hwang,et al.  Variable Step-Size LMS Algorithm With a Gradient-Based Weighted Average , 2009, IEEE Signal Processing Letters.

[33]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[34]  Xuelong Li,et al.  Image Quality Assessment Based on Multiscale Geometric Analysis , 2009, IEEE Transactions on Image Processing.

[35]  Xuelong Li,et al.  Direct kernel biased discriminant analysis: a new content-based image retrieval relevance feedback algorithm , 2006, IEEE Transactions on Multimedia.

[36]  Zhigang Luo,et al.  Online Nonnegative Matrix Factorization With Robust Stochastic Approximation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[37]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[38]  Richard S. Sutton,et al.  Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta , 1992, AAAI.

[39]  Zhou Wang,et al.  Reduced-reference image quality assessment using a wavelet-domain natural image statistic model , 2005, IS&T/SPIE Electronic Imaging.

[40]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[41]  B. Widrow,et al.  Stationary and nonstationary learning characteristics of the LMS adaptive filter , 1976, Proceedings of the IEEE.

[42]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[43]  Alan C. Bovik,et al.  No-reference quality assessment using natural scene statistics: JPEG2000 , 2005, IEEE Transactions on Image Processing.

[44]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .