Ieee Transactions on Communications, Accepted for Publication Generalized Belief Propagation for the Noiseless Capacity and Information Rates of Run-length Limited Constraints

The performance of the generalized belief propagation algorithm to compute the noiseless capacity and mutual information rates of finite-size two-dimensional and three-dimensional run-length limited constraints is investigated. In both cases, the problem is reduced to estimating the partition function of graphical models with cycles. The partition function is then estimated using the region-based free energy approximation technique. For each constraint, a method is proposed to choose the basic regions and to construct the region graph which provides the graphical framework to run the generalized belief propagation algorithm. Simulation results for the noiseless capacity of different constraints as a function of the size of the channel are reported. In the cases that tight lower and upper bounds on the Shannon capacity exist, convergence to the Shannon capacity is discussed. For noisy constrained channels, simulation results are reported for mutual information rates as a function of signal-to-noise ratio.

[1]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[2]  H.-A. Loeliger,et al.  An introduction to factor graphs , 2004, IEEE Signal Process. Mag..

[3]  Mehdi Molkaraie,et al.  Generalized belief propagation algorithm for the capacity of multi-dimensional run-length limited constraints , 2010, 2010 IEEE International Symposium on Information Theory.

[4]  Paul H. Siegel,et al.  On the achievable information rates of finite state ISI channels , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[5]  Ido Tal,et al.  Concave programming upper bounds on the capacity of 2-D constraints , 2009, 2009 IEEE International Symposium on Information Theory.

[6]  Payam Pakzad,et al.  Kikuchi approximation method for joint decoding of LDPC codes and partial-response channels , 2006, IEEE Transactions on Communications.

[7]  Kenneth Zeger,et al.  Capacity bounds for the three-dimensional (0, 1) run length limited channel , 2000, IEEE Trans. Inf. Theory.

[8]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[9]  Søren Forchhammer,et al.  Entropy of Bit-Stuffing-Induced Measures for Two-Dimensional Checkerboard Constraints , 2007, IEEE Transactions on Information Theory.

[10]  Richard E. Blahut,et al.  The Capacity and Coding Gain of Certain Checkerboard Codes , 1998, IEEE Trans. Inf. Theory.

[11]  Shlomo Shamai,et al.  Discrete-Input Two-Dimensional Gaussian Channels With Memory: Estimation and Information Rates Via Graphical Models and Statistical Mechanics , 2008, IEEE Transactions on Information Theory.

[12]  Wei Zeng,et al.  Simulation-Based Computation of Information Rates for Channels With Memory , 2006, IEEE Transactions on Information Theory.

[13]  Herbert S. Wilf,et al.  The Number of Independent Sets in a Grid Graph , 1998, SIAM J. Discret. Math..

[14]  P.H. Siegel Information-Theoretic Limits of Two-Dimensional Optical Recording Channels , 2006, 2006 Optical Data Storage Topical Meeting.

[15]  Schouhamer Immink,et al.  Codes for mass data storage systems , 2004 .

[16]  Claude E. Shannon,et al.  A Mathematical Theory of Communications , 1948 .

[17]  Hans-Andrea Loeliger,et al.  Monte Carlo Algorithms for the Partition Function and Information Rates of Two-Dimensional Channels , 2011, IEEE Transactions on Information Theory.

[18]  Hans-Andrea Loeliger,et al.  Estimating the partition function of 2-D fields and the capacity of constrained noiseless 2-D channels using tree-based Gibbs sampling , 2009, 2009 IEEE Information Theory Workshop.

[19]  Hans-Andrea Loeliger,et al.  On the information rate of binary-input channels with memory , 2001, ICC 2001. IEEE International Conference on Communications. Conference Record (Cat. No.01CH37240).

[20]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[21]  Ron M. Roth,et al.  Approximate enumerative coding for 2-D constraints through ratios of maprix Products , 2009, 2009 IEEE International Symposium on Information Theory.

[22]  Hisashi Ito,et al.  Zero capacity region of multidimensional run length constraints , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[23]  Kenneth Zeger,et al.  On the capacity of two-dimensional run-length constrained channels , 1999, IEEE Trans. Inf. Theory.

[24]  Hans-Andrea Loeliger,et al.  Estimating the information rate of noisy two-dimensional constrained channels , 2010, 2010 IEEE International Symposium on Information Theory.

[25]  Max Welling,et al.  On the Choice of Regions for Generalized Belief Propagation , 2004, UAI.