AN ON-LINE ADAPTATION ALGORITHM FOR ADAPTIVE SYSTEM TRAINING WITH MINIMUM ERROR ENTROPY: STOCHASTIC INFORMATION GRADIENT

We have recently reported on the use of minimum error entropy criterion as an alternative to minimum square error (MSE) in supervised adaptive system training. A nonparametric estimator for Renyi’s entropy was formulated by employing Parzen windowing. This formulation revealed interesting insights about the process of information theoretical learning, namely information potential and information forces. Variants of this criterion were applied to the training of linear and nonlinear adaptive topologies in blind source separation, channel equalization, and chaotic time-series prediction with superior results. In this paper, we propose an on-line version of the error entropy minimization algorithm, which can be used to train linear or nonlinear topologies in a supervised fashion. The algorithms used for blind source separation and deconvolution can be modified in a similar fashion. For the sake of simplicity, we present preliminary experimental results for FIR filter adaptation using this online algorithm and compare the performance with LMS.