k-Nearest Neighbor Based Consistent Entropy Estimation for Hyperspherical Distributions

A consistent entropy estimator for hyperspherical data is proposed based on the k-nearest neighbor (knn) approach. The asymptotic unbiasedness and consistency of the estimator are proved. Moreover, cross entropy and Kullback-Leibler (KL) divergence estimators are also discussed. Simulation studies are conducted to assess the performance of the estimators for models including uniform and von Mises-Fisher distributions. The proposed knn entropy estimator is compared with the moment based counterpart via simulations. The results show that these two methods are comparable.

[1]  S. R. Jammalamadaka,et al.  The generalized von Mises distribution , 2007 .

[2]  Paul P. B. Eggermont,et al.  Best Asymptotic Normality of the Kernel Density Entropy Estimator for Smooth Densities , 1999, IEEE Trans. Inf. Theory.

[3]  Qing Wang,et al.  Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor Distances , 2009, IEEE Transactions on Information Theory.

[4]  Paul M. Thompson,et al.  A Study of Information Gain in High Angular Resolution Diffusion Imaging (HARDI) , 2008 .

[5]  B. Vemuri,et al.  Generalized scalar measures for diffusion MRI using trace, variance, and entropy , 2005, Magnetic resonance in medicine.

[6]  S. Li.  Concise Formulas for the Area and Volume of a Hyperspherical Cap , 2011 .

[7]  Harshinder Singh,et al.  Nearest Neighbor Estimates of Entropy for Multivariate Circular Distributions , 2010, Entropy.

[8]  Neeraj Misra,et al.  Kn-nearest neighbor estimators of entropy , 2008 .

[9]  H. Knutsson.  Producing a Continuous and Distance Preserving 5-D Vector Representation of 3-D Orientation , 1985 .

[10]  Carl-Fredrik Westin,et al.  Hyperspherical von Mises-Fisher Mixture (HvMF) Modelling of High Angular Resolution Diffusion MRI , 2007, MICCAI.

[11]  綿森 葉子,et al.  Statistical inference of Langevin distribution for directional data , 1994 .

[12]  L. Pronzato,et al.  A class of Rényi information estimators for multidimensional densities , 2008, 0810.5302.

[13]  Baba C. Vemuri,et al.  Segmentation of High Angular Resolution Diffusion MRI Modeled as a Field of von Mises-Fisher Mixtures , 2006, ECCV.

[14]  J. Yukich,et al.  Laws of Large Numbers and Nearest Neighbor Distances , 2009, 0911.0331.

[15]  E. A. Yfantis,et al.  An extension of the von mises distribution , 1982 .

[16]  Ashis SenGupta,et al.  Advances in Directional and Linear Statistics , 2011 .

[17]  Harshinder Singh,et al.  Nearest Neighbor Estimates of Entropy , 2003 .

[18]  Shengqiao Li,et al.  ESTIMATION OF MULTIVARIATE SHANNON ENTROPY USING MOMENTS , 2011 .

[19]  S. R. Jammalamadaka,et al.  Directional Statistics, I , 2011 .

[20]  M. Rosenblatt,et al.  Multivariate k-nearest neighbor density estimates , 1979 .

[21]  Michel Loève,et al.  Probability Theory I , 1977 .

[22]  Lucas J. van Vliet,et al.  Representing Orientation in n-Dimensional Spaces , 2003, CAIP.