Information theoretical estimators toolbox

We present ITE (information theoretical estimators) a free and open source, multi-platform, Matlab/Octave toolbox that is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross quantities, and kernels on distributions. Thanks to its highly modular design, ITE supports additionally (i) the combinations of the estimation techniques, (ii) the easy construction and embedding of novel information theoretical estimators, and (iii) their immediate application in information theoretical optimization problems. ITE also includes a prototype application in a central problem class of signal processing, independent subspace analysis and its extensions.

[1]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[2]  L. Györfi,et al.  Nonparametric entropy estimation. An overview , 1997 .

[3]  Jean-François Cardoso,et al.  Multidimensional independent component analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[4]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[5]  Aapo Hyvärinen,et al.  Blind signal separation and independent component analysis , 2002, Neurocomputing.

[6]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[7]  Elmar Wolfgang Lang,et al.  Blind source separation and independent component analysis , 2006, Neurocomputing.

[8]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[9]  Barnabás Póczos,et al.  Undercomplete Blind Subspace Deconvolution , 2007, J. Mach. Learn. Res..

[10]  Ling Huang,et al.  Fast approximate spectral clustering , 2009, KDD.

[11]  Qing Wang,et al.  Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor Distances , 2009, IEEE Transactions on Information Theory.

[12]  T. Villmann,et al.  Mathematical Aspects of Divergence Based Vector Quantization Using Fr´ echet-Derivatives , 2009 .

[13]  Barnabás Póczos,et al.  Separation theorem for independent subspace analysis and its consequences , 2012, Pattern Recognit..

[14]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[15]  Gert R. G. Lanckriet,et al.  On the empirical estimation of integral probability metrics , 2012 .

[16]  Barnabás Póczos,et al.  Copula-based Kernel Dependency Measures , 2012, ICML.

[17]  Michèle Basseville,et al.  Divergence measures for statistical data processing - An annotated bibliography , 2013, Signal Process..