The Entropy of a Binary Hidden Markov Process

The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter ε. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion coefficients up to second order in ε. Using a conjecture we extend the calculation to 11th order and discuss the convergence of the resulting series

[1]  Michael I. Jordan,et al.  Boltzmann Chains and Hidden Markov Models , 1994, NIPS.

[2]  Philippe Jacquet,et al.  On the entropy of a hidden Markov process , 2004, Data Compression Conference, 2004. Proceedings. DCC 2004.

[3]  P Le Doussal,et al.  Nonequilibrium dynamics of random field Ising spin chains: exact results via real space renormalization group. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[5]  Neri Merhav,et al.  Hidden Markov processes , 2002, IEEE Trans. Inf. Theory.

[6]  B. M. Fulk.  MATH , 1992 .

[7]  T. Nattermann.  Theory of the Random Field Ising Model , 1997 .

[8]  J. Luck,et al.  Exactly soluble random field Ising models in one dimension , 1986 .

[9]  Alexander Schliep,et al.  Using hidden Markov models to analyze gene expression time course data , 2003, ISMB.

[10]  Tsachy Weissman,et al.  New bounds on the entropy rate of hidden Markov processes , 2004, Information Theory Workshop.

[11]  Eytan Domany,et al.  Asymptotics of the entropy rate for a hidden Markov process , 2005, Data Compression Conference.

[12]  A. Young,et al.  Spin glasses and random fields , 1997 .

[13]  David J. C. MacKay,et al.  Equivalence of Linear Boltzmann Chains and Hidden Markov Models , 1996, Neural Computation.

[14]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[15]  G. Grinstein,et al.  Exact solution of a one-dimensional Ising model in a random magnetic field , 1983 .

[16]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[17]  B Derrida,et al.  Singular behaviour of certain infinite products of random 2 × 2 matrices , 1983 .

[18]  Ido Kanter,et al.  Is a multiple excitation of a single atom equivalent to a single excitation of an ensemble of atoms , 2005 .

[19]  B. Derrida,et al.  Exactly solvable one-dimensional inhomogeneous models , 1986 .

[20]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[21]  Ido Kanter,et al.  Utilizing hidden Markov processes as a tool for experimental physics , 2005 .