A new class of random vector entropy estimators and its applications in testing statistical hypotheses

This paper proposes a new class of estimators of an unknown entropy of random vector. Its asymptotic unbiasedness and consistency are proved. Further, this class of estimators is used to build both goodness-of-fit and independence tests based on sample entropy. A simulation study indicates that the test involving the proposed entropy estimate has higher power than other well-known competitors under heavy tailed alternatives which are frequently used in many financial applications.

[1]  H. Lebesgue.  Sur l'intégration des fonctions discontinues , 1910 .

[2]  R. L. Dobrushin.  A Simplified Method of Experimentally Evaluating the Entropy of a Stationary Sequence , 1958 .

[3]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[4]  G. Basharin.  On a Statistical Estimate for the Entropy of a Sequence of Independent Random Variables , 1959 .

[5]  S. Kullback.  Probability Densities with Given Marginals , 1968 .

[6]  S. Shapiro,et al.  A Comparative Study of Various Tests for Normality , 1968 .

[7]  L. R. Shenton,et al.  Some moments of an estimate of shannon's measure of information , 1974 .

[8]  J. Ord,et al.  Characterization Problems in Mathematical Statistics , 1975 .

[9]  Oldrich A Vasicek,et al.  A Test for Normality Based on Sample Entropy , 1976 .

[10]  Ibrahim A. Ahmad,et al.  A nonparametric estimation of the entropy for absolutely continuous distributions (Corresp.) , 1976, IEEE Trans. Inf. Theory.

[11]  Philip Prescott,et al.  On a Test for Normality Based on Sample Entropy , 1976 .

[12]  Edward C. van der Meulen,et al.  Entropy-Based Tests of Uniformity , 1981 .

[13]  C. Stein.  Lectures on the theory of estimation of many parameters , 1986 .

[14]  H. Joe.  Majorization, randomness and dependence for multivariate distributions , 1987 .

[15]  H. Joe.  Estimation of entropy and other functionals of a multivariate density , 1989 .

[16]  J. N. Kapur.  Maximum-entropy models in science and engineering , 1992 .

[17]  H. Joe.  Relative Entropy Measures of Multivariate Dependence , 1989 .

[18]  P. Hall,et al.  On the estimation of entropy , 1993 .

[19]  A. Tagliani.  On the application of maximum entropy to the moments problem , 1993 .

[20]  A. Tsybakov,et al.  Root-N consistent estimators of entropy for densities with unbounded support , 1994, Proceedings of 1994 Workshop on Information Theory and Statistics.

[21]  Aldo Tagliani,et al.  Entropy-convergence in Stieltjes and Hamburger moment problem , 1997 .

[22]  L. Györfi,et al.  Nonparametric entropy estimation. An overview , 1997 .

[23]  Dipak K. Dey,et al.  A nonparametric approach to k-sample inference based on entropy-super-* , 1997 .

[24]  Przeniyslaw Crzcgorzewski,et al.  Entropy-based goodness-of-fit test for exponentiality , 1999 .

[25]  Paul P. B. Eggermont,et al.  Best Asymptotic Normality of the Kernel Density Entropy Estimator for Smooth Densities , 1999, IEEE Trans. Inf. Theory.

[26]  Samuel Kotz,et al.  The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance , 2001 .

[27]  T. Kozubowski,et al.  An asymmetric generalization of Gaussian and Laplace laws , .