Sample-Spacings-Based Density and Entropy Estimators for Spherically Invariant Multidimensional Data

While the sample-spacings-based density estimation method is simple and efficient, its applicability has been restricted to one-dimensional data. In this letter, the method is generalized such that it can be extended to multiple dimensions in certain circumstances. As a consequence, a multidimensional entropy estimator of spherically invariant continuous random variables is derived. Partial bias of the estimator is analyzed, and the estimator is further used to derive a nonparametric objective function for frequency-domain independent component analysis. The robustness and the effectiveness of the objective function are demonstrated with simulation results.

[1]  Te-Won Lee,et al.  Nonparametric Independent Component Analysis for Circular Complex Variables , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[2]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[3]  Ibrahim A. Ahmad,et al.  A nonparametric estimation of the entropy for absolutely continuous distributions (Corresp.) , 1976, IEEE Trans. Inf. Theory.

[4]  Robert Wieczorkowski,et al.  Entropy estimators‐improvements and comparisons , 1999 .

[5]  G. Lugosi,et al.  Consistency of Data-driven Histogram Methods for Density Estimation and Classification , 1996 .

[6]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis of Complex Valued Signals , 2000, Int. J. Neural Syst..

[7]  Erik G. Miller.  A new class of entropy estimators for multi-dimensional densities , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[8]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[9]  Christian Jutten,et al.  Space or time adaptive signal processing by neural network models , 1987 .

[10]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Sub-Gaussian and Super-Gaussian Sources , 1999, Neural Comput..

[11]  David Correa Martins,et al.  Intrinsically Multivariate Predictive Genes , 2008, IEEE Journal of Selected Topics in Signal Processing.

[12]  John W. Fisher,et al.  ICA Using Spacings Estimates of Entropy , 2003, J. Mach. Learn. Res..

[13]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[14]  Dinh-Tuan Pham,et al.  Blind separation of instantaneous mixture of sources based on order statistics , 2000, IEEE Trans. Signal Process..

[15]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[16]  Alfred O. Hero,et al.  Geodesic entropic graphs for dimension and entropy estimation in manifold learning , 2004, IEEE Transactions on Signal Processing.

[17]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[18]  R. Moddemeijer.  On estimation of entropy and mutual information of continuous distributions , 1989 .

[19]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[20]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[21]  P. D. Thouin,et al.  Survey and comparative analysis of entropy and relative entropy thresholding techniques , 2006 .

[22]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[23]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[24]  Harshinder Singh,et al.  Nearest Neighbor Estimates of Entropy , 2003 .

[25]  Oldrich A Vasicek,et al.  A Test for Normality Based on Sample Entropy , 1976 .

[26]  L. Györfi,et al.  Nonparametric entropy estimation. An overview , 1997 .

[27]  Vwani P. Roychowdhury,et al.  Independent component analysis based on nonparametric density estimation , 2004, IEEE Transactions on Neural Networks.

[28]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .