A Deterministic Algorithm for the Capacity of Finite-State Channels

We propose a modified version of the classical gradient descent method to compute the capacity of finite-state channels with Markovian input. Under some concavity assumptions, our algorithm proves to achieve polynomial accuracy in polynomial time for general finite-state channels. Moreover, for some special families of finite-state channels, our algorithm can achieve exponential accuracy in polynomial time.

[1]  Dicode.  The Capacity of Finite State Channels , 2003 .

[2]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[3]  Aleksandar Kavcic.  On the capacity of Markov sources over noisy channels , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[4]  John G. Proakis,et al.  Digital Communications , 1983 .

[5]  B. Marcus.  Constrained Systems and Coding for Recording Channels, in Handbook of Coding Theory, v. Finite-state Modulation Codes for Data Storage, Ieee , 2000 .

[6]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[7]  Paul H. Siegel,et al.  Markov Processes Asymptotically Achieve the Capacity of Finite-State Intersymbol Interference Channels , 2004, IEEE Transactions on Information Theory.

[8]  Brian H. Marcus,et al.  Analyticity of Entropy Rate of Hidden Markov Chains , 2005, IEEE Transactions on Information Theory.

[9]  Hans-Andrea Loeliger,et al.  A Generalization of the Blahut–Arimoto Algorithm to Finite-State Channels , 2008, IEEE Transactions on Information Theory.

[10]  Guangyue Han,et al.  Concavity of mutual information rate of finite-state channels , 2013, 2013 IEEE International Symposium on Information Theory.

[11]  Douglas Lind,et al.  An Introduction to Symbolic Dynamics and Coding , 1995 .

[12]  Robert E. Mahony,et al.  Convergence of the Iterates of Descent Methods for Analytic Cost Functions , 2005, SIAM J. Optim..

[13]  R. Gray.  Entropy and Information Theory , 1990, Springer New York.

[14]  Guangyue Han.  Limit Theorems in Hidden Markov Models , 2013, IEEE Transactions on Information Theory.

[15]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[16]  Guangyue Han,et al.  Asymptotics of Input-Constrained Erasure Channel Capacity , 2016, IEEE Transactions on Information Theory.

[17]  Brian H. Marcus,et al.  Concavity of the Mutual Information Rate for Input-Restricted Memoryless Channels at High SNR , 2012, IEEE Transactions on Information Theory.

[18]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[19]  H. Thapar,et al.  A class of partial response systems for increasing storage density in magnetic recording , 1987 .

[20]  G. David Forney,et al.  Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference , 1972, IEEE Trans. Inf. Theory.

[21]  Shlomo Shamai,et al.  On the capacity of binary and Gaussian channels with run-length-limited inputs , 1990, IEEE Trans. Commun..

[22]  Paul H. Siegel,et al.  Markov processes asymptotically achieve the capacity of finite-state intersymbol interference channels , 2004, ISIT.

[23]  Shirley Dex,et al.  JR 旅客販売総合システム（マルス）における運用及び管理について , 1991 .

[24]  Israel Bar-David,et al.  Capacity and coding for the Gilbert-Elliot channels , 1989, IEEE Trans. Inf. Theory.

[25]  Guangyue Han,et al.  A Randomized Algorithm for the Capacity of Finite-State Channels , 2015, IEEE Transactions on Information Theory.