Non-parametric Entropy Estimation Toolbox (NPEET)

This document describes a package of Python code for implementing various non-parametric continuous entropy estimators (and some discrete ones for convenience). After describing installation, Sec. 4 provides a wide-ranging discussion of technical, theoretical, and numerical issues surrounding entropy estimation. Sec. 5 provides references to the relevant literature for each estimator implemented. If you use these estimators in your research, please cite the appropriate authors. Sec. 6 describes the functionality and options in details.

[1]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[2]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Jonathan D Victor,et al.  Approaches to Information-Theoretic Analysis of Neural Activity , 2006, Biological theory.

[4]  S. Saigal,et al.  Relative performance of mutual information estimation methods for quantifying the dependence among short and noisy data. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  M. Paluš,et al.  Inferring the directionality of coupling with conditional mutual information. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  Qing Wang,et al.  Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor Distances , 2009, IEEE Transactions on Information Theory.

[7]  Michel Verleysen,et al.  Mutual information based feature selection for mixed data , 2011, ESANN.

[8]  Barnabás Póczos,et al.  Nonparametric Estimation of Conditional Information and Divergences , 2012, AISTATS.

[9]  Aram Galstyan,et al.  Information transfer in social media , 2011, WWW.

[10]  Aram Galstyan,et al.  Information-theoretic measures of influence based on content dynamics , 2012, WSDM.

[11]  Sara Klingenstein,et al.  Bootstrap Methods for the Empirical Study of Decision-Making and Information Flows in Social Systems , 2013, Entropy.

[12]  Fei Sha,et al.  Demystifying Information-Theoretic Clustering , 2013, ICML.

[13]  Aram Galstyan,et al.  Discovering Structure in High-Dimensional Data Through Correlation Explanation , 2014, NIPS.

[14]  Aram Galstyan,et al.  Maximally Informative Hierarchical Representations of High-Dimensional Data , 2014, AISTATS.