Stochastic Information Gradient Algorithm with Generalized Gaussian Distribution Model

This paper presents a parameterized version of the stochastic information gradient (SIG) algorithm, in which the error distribution is modeled by generalized Gaussian density (GGD), with location, shape, and dispersion parameters. Compared with the kernel-based SIG (SIG-Kernel) algorithm, the GGD-based SIG (SIG-GGD) algorithm does not involve kernel width selection. If the error is zero-mean, the SIG-GGD algorithm will become the least mean p-power (LMP) algorithm with adaptive order and variable step-size. Due to its well matched density estimation and automatic switching capability, the proposed algorithm is favorably in line with existing algorithms.

[1]  K. M. M. Prabhu,et al.  Radix-2 decimation-in-frequency algorithm for the computation of the real-valued FFT , 1999, IEEE Trans. Signal Process..

[2]  信太 克規,et al.  Adaptive Algorithm Based on Least Mean p-Power Error Criterion for Fourier Analysis in Additive Noise , 1998 .

[3]  José Carlos Príncipe,et al.  Information theoretic learning with adaptive kernels , 2011, Signal Process..

[4]  B. Silverman Density estimation for statistics and data analysis , 1986 .

[5]  Andrzej Cichocki,et al.  Flexible Independent Component Analysis , 2000, J. VLSI Signal Process..

[6]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Asoke K. Nandi,et al.  Exponent parameter estimation for generalized Gaussian probability density functions with application to speech modeling , 2005, Signal Process..

[8]  E. Jaynes Information Theory and Statistical Mechanics , 1957 .

[9]  Alberto Leon-Garcia,et al.  Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[10]  V. J. Mathews,et al.  Improved convergence analysis of stochastic gradient adaptive filters using the sign algorithm , 1987, IEEE Trans. Acoust. Speech Signal Process..

[11]  M. Varanasi,et al.  Parametric generalized Gaussian density estimation , 1989 .

[12]  Chien-Cheng Tseng,et al.  Least mean p-power error criterion for adaptive FIR filter , 1994, IEEE J. Sel. Areas Commun..

[13]  Teresa H. Y. Meng,et al.  Stochastic gradient adaptation under general error criteria , 1994, IEEE Trans. Signal Process..

[14]  Deniz Erdoğmuş,et al.  Online entropy manipulation: stochastic information gradient , 2003, IEEE Signal Processing Letters.

[15]  Bernard Widrow,et al.  The least mean fourth (LMF) adaptive algorithm and its family , 1984, IEEE Trans. Inf. Theory.

[16]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[17]  Deniz Erdogmus,et al.  Convergence properties and data efficiency of the minimum error entropy criterion in ADALINE training , 2003, IEEE Trans. Signal Process..

[18]  Deniz Erdoğmuş,et al.  COMPARISON OF ENTROPY AND MEAN SQUARE ERROR CRITERIA IN ADAPTIVE SYSTEM TRAINING USING HIGHER ORDER STATISTICS , 2004 .

[19]  Thomas R. Fischer,et al.  Comparison of generalized Gaussian and Laplacian modeling in DCT image coding , 1995, IEEE Signal Processing Letters.

[20]  Zengqi Sun,et al.  Stochastic Gradient Algorithm Under (h,φ)-Entropy Criterion , 2007 .

[21]  Deniz Erdogmus,et al.  Generalized information potential criterion for adaptive system training , 2002, IEEE Trans. Neural Networks.