Adaptive blind deconvolution of linear channels using Renyi's entropy with Parzen window estimation

Blind deconvolution of linear channels is a fundamental signal processing problem that has immediate extensions to multiple-channel applications. In this paper, we investigate the suitability of a class of Parzen-window-based entropy estimates, namely Renyi's entropy, as a criterion for blind deconvolution of linear channels. Comparisons between maximum and minimum entropy approaches, as well as the effect of entropy order, equalizer length, sample size, and measurement noise on performance, will be investigated through Monte Carlo simulations. The results indicate that this nonparametric entropy estimation approach outperforms the standard Bell-Sejnowski and normalized kurtosis algorithms in blind deconvolution. In addition, the solutions using Shannon's entropy were not optimal either for super- or sub-Gaussian source densities.

[1]  Santamaria,et al.  A fast algorithm for adaptive blind equalization using order-/spl alpha/ Renyi's entropy , 2002 .

[2]  J. Cadzow.  Blind deconvolution via cumulant extrema , 1996, IEEE Signal Process. Mag..

[3]  Inbar Fijalkow,et al.  A globally convergent approach for blind MIMO adaptive deconvolution , 2001, IEEE Trans. Signal Process..

[4]  Jitendra K. Tugnait.  On blind separation of convolutive mixtures of independent linear signals in unknown additive noise , 1998, IEEE Trans. Signal Process..

[5]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[6]  Deniz Erdogmus,et al.  Generalized information potential criterion for adaptive system training , 2002, IEEE Trans. Neural Networks.

[7]  Deniz Erdoğmuş,et al.  Blind source separation using Renyi's mutual information , 2001, IEEE Signal Processing Letters.

[8]  Yujiro Inouye,et al.  Iterative algorithms based on multistage criteria for multichannel blind deconvolution , 1999, IEEE Trans. Signal Process..

[9]  C. Diks,et al.  Detecting differences between delay vector distributions. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[10]  D. Donoho.  ON MINIMUM ENTROPY DECONVOLUTION , 1981 .

[11]  C. Tsallis.  Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[12]  Athina P. Petropulu,et al.  Frequency domain blind MIMO system identification based on second and higher order statistics , 2001, IEEE Trans. Signal Process..

[13]  Chong-Yung Chi,et al.  Cumulant-based inverse filter criteria for MIMO blind deconvolution: properties, algorithms, and application to DS/CDMA systems in multipath , 2001, IEEE Trans. Signal Process..

[14]  Julian J. Bussgang,et al.  Crosscorrelation functions of amplitude-distorted gaussian signals , 1952 .

[15]  Andrzej Cichocki,et al.  Relationships between instantaneous blind source separation and multichannel blind deconvolution , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[16]  Athina P. Petropulu,et al.  Blind two-input-two-output FIR channel identification based on frequency domain second-order statistics , 2000, IEEE Trans. Signal Process..

[17]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[18]  Jitendra K. Tugnait,et al.  Comments on 'New criteria for blind deconvolution of nonminimum phase systems (channels)' , 1992, IEEE Trans. Inf. Theory.

[19]  Jean-François Bercher,et al.  Estimating the entropy of a signal with applications , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[20]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[21]  Jean-François Bercher,et al.  A Renyi entropy convolution inequality with application , 2002, 2002 11th European Signal Processing Conference.

[22]  Inder Jeet Taneja,et al.  Entropy of type (α, β) and other generalized measures in information theory , 1975 .

[23]  Deniz Erdoğmuş.  INFORMATION THEORETIC LEARNING: RENYI'S ENTROPY AND ITS APPLICATIONS TO ADAPTIVE SYSTEM TRAINING , 2002 .

[24]  Pierre Comon,et al.  Analytical blind channel identification , 2002, IEEE Trans. Signal Process..

[25]  José Carlos Príncipe,et al.  Fast algorithm for adaptive blind equalization using order-α Renyi's entropy , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[26]  A. Benveniste,et al.  Robust identification of a nonminimum phase system: Blind adjustment of a linear equalizer in data communications , 1980 .

[27]  C. L. Nikias,et al.  Higher-order spectra analysis : a nonlinear signal processing framework , 1993 .

[28]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[29]  Deniz Erdogmus,et al.  Blind source separation using Renyi's -marginal entropies , 2002, Neurocomputing.

[30]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[31]  Jan Havrda,et al.  Quantification method of classification processes. Concept of structural a-entropy , 1967, Kybernetika.

[32]  Yujiro Inouye,et al.  Cumulant-based blind identification of linear multi-input-multi-output systems driven by colored inputs , 1997, IEEE Trans. Signal Process..

[33]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[34]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[35]  H. K. Kesavan,et al.  The generalized maximum entropy principle , 1989, IEEE Trans. Syst. Man Cybern..

[36]  Paul A. Viola,et al.  Empirical Entropy Manipulation for Real-World Problems , 1995, NIPS.

[37]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.