Big Data Viewpoint On Channel Information Measures Based on ACE Algorithm

In this paper, we focus on the mutual information, which can characterize the transmission ability because it shows correlation between channel input and channel output. Shannon entropy and mutual information are the cornerstones of information theory. In addition, Chernoff information is another fundamental channel information measure, and it describe the maximum achievable exponent of the error probability in hypothesis testing. Uased on alternating conditional expectation (ACE) algorithm, we decompose these two mutual information. In fact, their decomposition results are similar in big data prespective. In this sense, these two kinds of mutual information are just different measures of the same information quantity. This paper also deduces that the channel performance only depends on channel parameters and the decomposition results of a new proposed mutual information should agree with the impact of the parameters.

[1]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[2]  Deniz Erdogmus,et al.  An analysis of entropy estimators for blind source separation , 2006, Signal Process..

[3]  A. Rényi On Measures of Entropy and Information , 1961 .

[4]  Sergio Verdú,et al.  Fifty Years of Shannon Theory , 1998, IEEE Trans. Inf. Theory.

[5]  V. Vapnik Estimation of Dependences Based on Empirical Data , 2006 .

[6]  Peter Harremoës,et al.  Rényi Divergence and Kullback-Leibler Divergence , 2012, IEEE Transactions on Information Theory.

[7]  Duolao Wang,et al.  Estimating Optimal Transformations for Multiple Regression Using the ACE Algorithm , 2004, Journal of Data Science.

[8]  Deniz Erdogmus,et al.  Information Theoretic Learning , 2005, Encyclopedia of Artificial Intelligence.

[9]  H. Krim,et al.  Renyi entropy based divergence measures for ICA , 2004, IEEE Workshop on Statistical Signal Processing, 2003.

[10]  J. Friedman,et al.  Estimating Optimal Transformations for Multiple Regression and Correlation. , 1985 .

[11]  Deniz Erdoğmuş,et al.  Blind source separation using Renyi's mutual information , 2001, IEEE Signal Processing Letters.