Entropy rate of continuous-state hidden Markov chains

We prove that under mild positivity assumptions, the entropy rate of a continuous-state hidden Markov chain, observed when passing a finite-state Markov chain through a discrete-time continuous-output channel, is analytic as a function of the transition probabilities of the underlying Markov chain. We further prove that the entropy rate of a continuous-state hidden Markov chain, observed when passing a mixing finite-type constrained Markov chain through a discrete-time Gaussian channel, is smooth as a function of the transition probabilities of the underlying Markov chain.

[1]  Shunsuke Ihara,et al.  Information theory - for continuous systems , 1993 .

[2]  Loic Dubois,et al.  Projective metrics and contraction principles for complex cones , 2009 .

[3]  G. Constantine,et al.  A Multivariate Faa di Bruno Formula with Applications , 1996 .

[4]  H. H. Rugh,et al.  Cones and gauges in complex spaces: Spectral gaps and complex Perron-Frobenius theory , 2006, math/0610354.

[5]  I. T. Jolliffe,et al.  Springer series in statistics , 1986 .

[6]  Yuval Peres,et al.  A note on a complex Hilbert metric with application to domain of analyticity for entropy rate of hidden Markov processes , 2009, ArXiv.

[7]  Philippe Jacquet,et al.  On the entropy of a hidden Markov process , 2008, Theor. Comput. Sci..

[8]  E. Seneta.  Non-negative Matrices and Markov Chains , 2008 .

[9]  Brian H. Marcus,et al.  Asymptotics of Input-Constrained Binary Symmetric Channel Capacity , 2008, ArXiv.

[10]  Eytan Domany,et al.  From Finite-System Entropy to Entropy Rate for a Hidden Markov Process , 2006, IEEE Signal Processing Letters.

[11]  B. Marcus.  Constrained Systems and Coding for Recording Channels, in Handbook of Coding Theory, v. Finite-state Modulation Codes for Data Storage, Ieee , 2000 .

[12]  Douglas Lind,et al.  An Introduction to Symbolic Dynamics and Coding , 1995 .

[13]  Valerie Isham,et al.  Non‐Negative Matrices and Markov Chains , 1983 .

[14]  Brian H. Marcus,et al.  Analyticity of Entropy Rate of Hidden Markov Chains , 2005, IEEE Transactions on Information Theory.