ESTIMATION OF MULTIVARIATE SHANNON ENTROPY USING MOMENTS

Summary Three new entropy estimators of multivariate distributions are introduced. The two cases considered here concern when the distribution is supported by a unit sphere and by a unit cube. In the former case, the consistency and the upper bound of the absolute error for the proposed entropy estimator are established. In the latter one, under the assumption that only the moments of the underlying distribution are available, a non-traditional estimator of the entropy is suggested. We also study the practical performances of the constructed estimators through simulation studies and compare the estimators based on the moment-recovered approaches with their counterparts derived by using the histogram and kth nearest neighbour constructions. In addition, one worked example is briefly discussed.

[1]  Robert M. Mnatsakanov,et al.  Moment-recovered approximations of multivariate distributions: The Laplace transform inversion , 2011 .

[2]  Neeraj Misra,et al.  Kn-nearest neighbor estimators of entropy , 2008 .

[3]  E. Khmaladze.  The statistical analysis of a large number of rare events , 1988 .

[4]  A. Tsybakov,et al.  Root-N consistent estimators of entropy for densities with unbounded support , 1994, Proceedings of 1994 Workshop on Information Theory and Statistics.

[5]  Michael E. Andrew,et al.  k-Nearest Neighbor Based Consistent Entropy Estimation for Hyperspherical Distributions , 2011, Entropy.

[6]  Robert M. Mnatsakanov,et al.  Hausdorff moment problem: Reconstruction of probability density functions , 2008 .

[7]  P. Hall,et al.  On the estimation of entropy , 1993 .

[8]  Harshinder Singh,et al.  Probabilistic model for two dependent circular variables , 2002 .

[9]  R. Mnatsakanov.  Hausdorff moment problem: Reconstruction of distributions , 2008 .

[10]  L. Györfi,et al.  Density-free convergence properties of various estimators of entropy , 1987 .

[11]  Harshinder Singh,et al.  Nearest Neighbor Estimates of Entropy for Multivariate Circular Distributions , 2010, Entropy.

[12]  F. Ruymgaart.  Strong uniform convergence of density estimators on spheres , 1989 .

[13]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[14]  H. Joe.  Estimation of entropy and other functionals of a multivariate density , 1989 .

[15]  Aldo Tagliani,et al.  Hausdorff moment problem via fractional moments , 2002, Appl. Math. Comput..

[16]  S. Li.  Concise Formulas for the Area and Volume of a Hyperspherical Cap , 2011 .

[17]  L. Györfi,et al.  Nonparametric entropy estimation. An overview , 1997 .

[18]  Harshinder Singh,et al.  Nearest Neighbor Estimates of Entropy , 2003 .

[19]  J. Shohat,et al.  The problem of moments , 1943 .