Mutual Information for Stochastic Signals and LÉvy Processes

In this paper, some relations between estimation and mutual information are given by expressing two mutual information calculations in terms of two distinct estimation errors. Specifically the mutual information between a stochastic signal and a pure jump Levy process whose rate function depends on the signal is expressed in terms of a filtering error and the rate of change of this mutual information with respect to a parameter multiplying the rate function of the Levy process is expressed in terms of a smoothing error. These results generalize the analogous mutual information results for some Gaussian noise processes with additive stochastic signals.

[1]  T. Duncan.  ON THE CALCULATION OF MUTUAL INFORMATION , 1970 .

[2]  P. Brémaud.  Point Processes and Queues , 1981 .

[3]  F. Utzet,et al.  Canonical Lévy process and Malliavin calculus , 2007 .

[4]  Shlomo Shamai,et al.  Proof of Entropy Power Inequalities Via MMSE , 2006, 2006 IEEE International Symposium on Information Theory.

[5]  Moshe Zakai,et al.  On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel , 2004, IEEE Transactions on Information Theory.

[6]  P. Varaiya,et al.  Martingales on Jump Processes. II: Applications , 1975 .

[7]  Shlomo Shamai,et al.  Mutual information and conditional mean estimation in Poisson channels , 2004, ITW.

[8]  Ken-iti Sato.  Lévy Processes and Infinitely Divisible Distributions , 1999 .

[9]  Shlomo Shamai,et al.  Mutual Information and Conditional Mean Estimation in Poisson Channels , 2004, IEEE Transactions on Information Theory.

[10]  P. Protter.  Stochastic integration and differential equations , 1990 .

[11]  Tyrone E. Duncan,et al.  Evaluation of Likelihood Functions , 1968, Inf. Control..

[12]  Shlomo Shamai,et al.  Mutual information and minimum mean-square error in Gaussian channels , 2004, IEEE Transactions on Information Theory.

[13]  M. Aschwanden.  Statistics of Random Processes , 2021, Biomedical Measurement Systems and Data Science.

[14]  Moshe Zakai,et al.  Some relations between mutual information and estimation error in Wiener space , 2006 .

[15]  D. Nualart,et al.  Anticipative calculus for the Poisson process based on the Fock space , 1990 .

[16]  Mokshay M. Madiman,et al.  Fisher Information, Compound Poisson Approximation, and the Poisson Channel , 2007, 2007 IEEE International Symposium on Information Theory.

[17]  Kiyosi Itô,et al.  SPECTRAL TYPE OF THE SHIFT TRANSFORMATION OF DIFFERENTIAL PROCESSES WITH STATIONARY INCREMENTS( , 1956 .

[18]  Andrea Montanari,et al.  Modern Coding Theory: The Statistical Mechanics and Computer Science Point of View , 2007, ArXiv.

[19]  Tyrone E. Duncan.  Mutual Information for Stochastic Signals and Fractional Brownian Motion , 2008, IEEE Transactions on Information Theory.