Iterative Algorithms for Maximum Likelihood Sequence Detection

The success of iterative decoding algorithms for turbo codes, for low density parity check codes, and for related codes has motivated studying connections to other potentially related iterative algorithms in information theory and in estimation theory. Important iterative algorithms in information theory include the Blahut-Arimoto algorithm for computing the distribution that achieves channel capacity and Blahut’s algorithm for computing the rate-distortion function. In estimation theory, the expectation-maximization algorithm is widely used, especially in high-dimensional inference problems such as in image estimation. A study of these algorithms as alternating minimization algorithms lays the foundation for deriving new alternating minimization algorithms for a variety of problems. One family of such algorithms includes iterative algorithms for maximum-likelihood sequence detection. These algorithms and some of their properties are presented.

[1]  H. Malcolm Hudson,et al.  Accelerated image reconstruction using ordered subsets of projection data , 1994, IEEE Trans. Medical Imaging.

[2]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  Brendan J. Frey,et al.  Graphical Models for Machine Learning and Digital Communication , 1998 .

[5]  Alvaro R. De Pierro,et al.  A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography , 1995, IEEE Trans. Medical Imaging.

[6]  T. Aaron Gulliver,et al.  Cross-Entropy and Iterative Decoding , 1998, IEEE Trans. Inf. Theory.

[7]  John Cocke,et al.  Optimal decoding of linear codes for minimizing symbol error rate (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[8]  Yair Weiss,et al.  Correctness of Local Probability Propagation in Graphical Models with Loops , 2000, Neural Computation.

[9]  James A. Bucklew,et al.  Large Deviation Techniques in Decision, Simulation, and Estimation , 1990 .

[10]  J. O’Sullivan.  Alternating Minimization Algorithms: From Blahut-Arimoto to Expectation-Maximization , 1998 .

[11]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[12]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[13]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[14]  M.I. Miller,et al.  The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process intensities and toeplitz constrained covariances , 1987, Proceedings of the IEEE.

[15]  Robert J. McEliece,et al.  The generalized distributive law , 2000, IEEE Trans. Inf. Theory.

[16]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[17]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[18]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[19]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[20]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[21]  Jeffrey A. Fessler,et al.  Ieee Transactions on Image Processing: to Appear Globally Convergent Algorithms for Maximum a Posteriori Transmission Tomography , 2022 .

[22]  Joseph A. O'Sullivan,et al.  Roughness penalties on finite domains , 1995, IEEE Trans. Image Process..

[23]  Alexander Vardy.  Codes, Curves, and Signals: Common Threads in Communications , 1998 .

[24]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[25]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[26]  A. R. De Pierro,et al.  On the relation between the ISRA and the EM algorithm for positron emission tomography , 1993, IEEE Trans. Medical Imaging.