Convergence of a Fixed-Point Minimum Error Entropy Algorithm

The minimum error entropy (MEE) criterion is an important learning criterion in information theoretical learning (ITL). However, the MEE solution cannot be obtained in closed form even for a simple linear regression problem, and one has to search it, usually, in an iterative manner. The fixed-point iteration is an efficient way to solve the MEE solution. In this work, we study a fixed-point MEE algorithm for linear regression, and our focus is mainly on the convergence issue. We provide a sufficient condition (although a little loose) that guarantees the convergence of the fixed-point MEE algorithm. An illustrative example is also presented.

[1]  Jose C. Principe,et al.  Information Theoretic Learning - Renyi's Entropy and Kernel Perspectives , 2010, Information Theoretic Learning.

[2]  Luís A. Alexandre,et al.  Minimum Error Entropy Classification , 2013, Studies in Computational Intelligence.

[3]  Jie Chen,et al.  Nonnegative Least-Mean-Square Algorithm , 2011, IEEE Transactions on Signal Processing.

[4]  Badong Chen,et al.  On the Smoothed Minimum Error Entropy Criterion , 2012, Entropy.

[5]  Deniz Erdogmus,et al.  Convergence properties and data efficiency of the minimum error entropy criterion in ADALINE training , 2003, IEEE Trans. Signal Process..

[6]  Jie Chen,et al.  Non-negative least-mean-square algorithm , 2011 .

[7]  Badong Chen,et al.  Mean-Square Convergence Analysis of ADALINE Training With Minimum Error Entropy Criterion , 2010, IEEE Transactions on Neural Networks.

[8]  Badong Chen,et al.  Some Further Results on the Minimum Error Entropy Estimation , 2012, Entropy.

[9]  Simone G. O. Fiori,et al.  Fast fixed-point neural blind-deconvolution algorithm , 2004, IEEE Transactions on Neural Networks.

[10]  Jie Chen,et al.  Variants of Non-Negative Least-Mean-Square Algorithm and Convergence Analysis , 2014, IEEE Transactions on Signal Processing.

[11]  Deniz Erdogmus,et al.  An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems , 2002, IEEE Trans. Signal Process..

[12]  Seungju Han,et al.  A Fixed-Point Minimum Error Entropy Algorithm , 2006, 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing.

[13]  Y. Cho,et al.  Fixed Point Theory and Applications , 2000 .

[14]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[15]  Phillip A. Regalia,et al.  Monotonic convergence of fixed-point algorithms for ICA , 2003, IEEE Trans. Neural Networks.

[16]  Deniz Erdogmus,et al.  Generalized information potential criterion for adaptive system training , 2002, IEEE Trans. Neural Networks.

[17]  Nanning Zheng,et al.  Convergence of a Fixed-Point Algorithm under Maximum Correntropy Criterion , 2015, IEEE Signal Processing Letters.

[18]  Badong Chen,et al.  System Parameter Identification: Information Criteria and Algorithms , 2013 .

[19]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .