Asymptotically Unbiased Estimation of A Nonsymmetric Dependence Measure Applied to Sensor Data Analytics and Financial Time Series

A fundamental concept frequently applied to statistical machine learning is the detection of dependencies between unknown random variables found from data samples. In previous work, we have introduced a nonparametric unilateral dependence measure based on Onicescu’s information energy and a kNN method for estimating this measure from an available sample set of discrete or continuous variables. This paper provides the formal proofs which show that the estimator is asymptotically unbiased and has asymptotic zero variance when the sample size increases. It implies that the estimator has good statistical qualities. We investigate the performance of the estimator for data analysis applications in sensor data analysis and financial time series.

[1]  Liam Paninski,et al.  Estimation of Entropy and Mutual Information , 2003, Neural Computation.

[2]  Robert D. Tortora,et al.  Sampling: Design and Analysis , 2000 .

[3]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[5]  Angel Cataron,et al.  An informational energy LVQ approach for feature ranking , 2004, ESANN.

[6]  Silviu Guiaşu,et al.  Information theory with applications , 1977 .

[7]  R. Andonie,et al.  How to infer the informational energy from small datasets , 2012, 2012 13th International Conference on Optimization of Electrical and Electronic Equipment (OPTIM).

[8]  G. Battail Théorie de l'information , 1982, Électronique.

[9]  Olga Kosheleva,et al.  Selecting the most representative sample is NP-hard: Need for expert (fuzzy) knowledge , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[10]  Angel Cataron,et al.  Mortality rate modeling of joint lives and survivor insurance contracts tested by a novel unilateral dependence measure , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[11]  Jacob Goldberger,et al.  ICA based on a Smooth Estimation of the Differential Entropy , 2008, NIPS.

[12]  Sanjeev R. Kulkarni,et al.  A Nearest-Neighbor Approach to Estimating Divergence between Continuous Random Vectors , 2006, 2006 IEEE International Symposium on Information Theory.

[13]  B. Schweizer,et al.  On Nonparametric Measures of Dependence for Random Variables , 1981 .

[14]  Angel Caţaron,et al.  Asymptotically Unbiased Estimator of the Informational Energy with kNN , 2013, Int. J. Comput. Commun. Control.

[15]  Yan Li,et al.  Estimation of Mutual Information: A Survey , 2009, RSKT.

[16]  Angel Cataron,et al.  Financial data analysis using the informational energy unilateral dependency measure , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[17]  Angel Cataron,et al.  kNN estimation of the unilateral dependency measure between random variables , 2014, 2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM).

[18]  Haye Hinrichsen,et al.  Entropy estimates of small data sets , 2008, 0804.4561.