Bounds on the Performance of Maximum-Likelihood Decoded Binary Block Codes in AWGN Interference

The error probability of Maximum-Likelihood (ML) soft-decision decoded binary block codes rarely accepts nice closed forms. In addition, for long codes ML decoding becomes prohibitively complex. Nevertheless, bounds on the performance of ML decoded systems provide insight into the effect of system parameters on the overall system performance as well as a measure of efficiency of the suboptimum decoding methods used in practice. In this dissertation, using the so-called Gallager’s first bounding technique (involving a so-called Gallager region) and within the framework of Tangential Sphere Bound (TSB) of Poltyrev, a general bound referred to as the Generalized Tangential Sphere Bound (GTSB) is developed. The Gallager region is chosen to be a general Hyper-Surface of Revolution (HSR) which is optimized to tighten the bound. The search for the optimal Gallager region is a classical problem dating back to Gallager’s thesis in the early 1960’s. For the random coding case, Gallager provided the optimal solution in a closed form while for the non-random case the problem has been an active area of research in information theory for many years. GTSB is applied to two problems. First, to the word error probability of BPSKmodulated binary block codes with ML decoding, and second, to the nonuniform signaling of binary block codes with Maximum-A-Posteriori (MAP) decoding. For both, the optimum HSR is obtained and it is noted that in the former case the resulting bound is the TSB of Poltyrev which has been the tightest bound developed for binary block codes to date. This terminates the search for a better Gallager region in the groundwork of the GTSB for the underlying application. In the development of TSB, union bound which is the simplest inequality from the large class of so-called Bonferroni-type inequalities in probability theory is uti-

[1]  E.R. Berlekamp,et al.  The technology of error-correcting codes , 1980, Proceedings of the IEEE.

[2]  Donald B. Owen,et al.  Survey of Properties and Applications of the Noncentral t-Distribution , 1968 .

[3]  Amir K. Khandani,et al.  A new upper bound on the ML decoding error probability of linear binary block codes in AWGN interference , 2004, IEEE Transactions on Information Theory.

[4]  R. Prim Shortest connection networks and some generalizations , 1957 .

[5]  R. Connelly In Handbook of Convex Geometry , 1993 .

[6]  Dariush Divsalar,et al.  Some new twists to problems involving the Gaussian probability integral , 1998, IEEE Trans. Commun..

[7]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[8]  Alexander Vardy,et al.  Which codes have cycle-free Tanner graphs? , 1999, IEEE Trans. Inf. Theory.

[9]  W. Fischer,et al.  Sphere Packings, Lattices and Groups , 1990 .

[10]  Fady Alajaji,et al.  Tight error bounds for nonuniform signaling over AWGN channels , 2000, IEEE Trans. Inf. Theory.

[11]  Shlomo Shamai,et al.  On improved bounds on the decoding error probability of block codes over interleaved fading channels, with applications to turbo-like codes , 2001, IEEE Trans. Inf. Theory.

[12]  David J. Hunter An upper bound for the probability of a union , 1976, Journal of Applied Probability.

[13]  Shlomo Shamai,et al.  On information rates for mismatched decoders , 1994, IEEE Trans. Inf. Theory.

[14]  Erik Agrell,et al.  Voronoi regions for binary linear block codes , 1996, IEEE Trans. Inf. Theory.

[15]  Gregory Poltyrev,et al.  The error probability of M-ary PSK block coded modulation schemes , 1996, IEEE Trans. Commun..

[16]  Eric W. Weisstein,et al.  Eric Weisstein''s World of Mathematics , 1999, WWW 1999.

[17]  M. Newman,et al.  Topics in Algebra , 1978 .

[18]  J. Galambos,et al.  Bonferroni-type inequalities with applications , 1996 .

[19]  Masoud Salehi,et al.  New performance bounds for turbo codes , 1998, IEEE Trans. Commun..

[20]  Shlomo Shamai,et al.  Improved upper bounds on the ML performance of turbo codes for interleaved Rician fading channels, with comparison to iterative decoding , 2000, 2000 IEEE International Conference on Communications. ICC 2000. Global Convergence Through Communications. Conference Record.

[21]  A. M. Viterbi,et al.  Improved union bound on linear codes for the input-binary AWGN channel, with applications to turbo codes , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[22]  Edward C. Posner Properties of error-correcting codes at low signal-to-noise ratios. , 1967 .

[23]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[24]  I. Blake,et al.  Group Codes for the Gaussian Channel , 1975 .

[25]  S. Brendle,et al.  Calculus of Variations , 1927, Nature.

[26]  T.E. Fuja,et al.  Channel codes that exploit the residual redundancy in CELP-encoded speech , 1996, IEEE Trans. Speech Audio Process..

[27]  Peter F. Swaszek A lower bound on the error probability for signals in white Gaussian noise , 1995, IEEE Trans. Inf. Theory.

[28]  G. David Forney,et al.  Geometrically uniform codes , 1991, IEEE Trans. Inf. Theory.

[29]  Richard M. Wilson,et al.  A course in combinatorics , 1992 .

[30]  D. Slepian A class of binary signaling alphabets , 1956 .

[31]  Dariush Divsalar,et al.  Introduction to Trellis-Coded Modulation With Applications , 1991 .

[32]  Godfried T. Toussaint,et al.  Relative neighborhood graphs and their relatives , 1992, Proc. IEEE.

[33]  Sergio Benedetto,et al.  Unveiling turbo codes: some results on parallel concatenated coding schemes , 1996, IEEE Trans. Inf. Theory.

[34]  Alain Glavieux,et al.  Reflections on the Prize Paper : "Near optimum error-correcting coding and decoding: turbo codes" , 1998 .

[35]  R. Herzog,et al.  Improved tight performance bounds on concatenated codes , 1999, Seamless Interconnection for Universal Services. Global Telecommunications Conference. GLOBECOM'99. (Cat. No.99CH37042).

[36]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[37]  C. Shannon Probability of error for optimal codes in a Gaussian channel , 1959 .

[38]  E. Kounias Bounds for the Probability of a Union, with Applications , 1968 .

[39]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[40]  L. Comtet,et al.  Advanced Combinatorics: The Art of Finite and Infinite Expansions , 1974 .

[41]  James E. pLebensohn Geometry and the Imagination , 1952 .

[42]  Amir K. Khandani,et al.  Generalized tangential sphere bound on the ML decoding error probability of linear binary block codes in AWGN interference , 2004, IEEE Transactions on Information Theory.

[43]  Masoud Salehi,et al.  Turbo codes and turbo coded modulation systems: analysis and performance bounds , 1998 .

[44]  Kamil Sh. Zigangirov,et al.  Tighter bounds on the error probability of fixed convolutional codes , 2001, IEEE Trans. Inf. Theory.

[45]  Marc P. C. Fossorier,et al.  Sphere-packing bounds revisited for moderate block lengths , 2004, IEEE Transactions on Information Theory.

[46]  N. J. A. Sloane,et al.  New trellis codes based on lattices and cosets , 1987, IEEE Trans. Inf. Theory.

[47]  Johannes Zangl,et al.  Improved tangential sphere bound on the bit-error probability of concatenated codes , 2001, IEEE J. Sel. Areas Commun..

[48]  Shlomo Shamai,et al.  Variations on the Gallager bounds, connections, and applications , 2002, IEEE Trans. Inf. Theory.

[49]  G. C. Shephard,et al.  Convex Polytopes and the Upper Bound Conjecture , 1971 .

[50]  Radford M. Neal,et al.  Near Shannon limit performance of low density parity check codes , 1996 .

[51]  Masoud Salehi,et al.  Performance bounds for turbo-coded modulation systems , 1999, IEEE Trans. Commun..

[52]  Atsuyuki Okabe,et al.  Spatial Tessellations: Concepts and Applications of Voronoi Diagrams , 1992, Wiley Series in Probability and Mathematical Statistics.

[53]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[54]  J. Kruskal On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[55]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.

[56]  Hans-Andrea Loeliger,et al.  Signal sets matched to groups , 1991, IEEE Trans. Inf. Theory.

[57]  Reinhard Diestel,et al.  Graph Theory , 1997 .

[58]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[59]  Masoud Salehi,et al.  The union bound for turbo-coded modulation systems over fading channels , 1999, IEEE Trans. Commun..

[60]  Robert G. Gallager,et al.  A simple derivation of the coding theorem and some applications , 1965, IEEE Trans. Inf. Theory.

[61]  Dominique de Caen,et al.  A lower bound on the probability of a union , 1997, Discret. Math..

[62]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[63]  Gregory Poltyrev,et al.  Bounds on the decoding error probability of binary linear codes via their spectra , 1994, IEEE Trans. Inf. Theory.

[64]  Shlomo Shamai,et al.  Improved upper bounds on the ML decoding error probability of parallel and serial concatenated turbo codes via their ensemble distance spectrum , 2000, IEEE Trans. Inf. Theory.

[65]  Fady Alajaji,et al.  Detection of binary Markov sources over channels with additive Markov noise , 1996, IEEE Trans. Inf. Theory.

[66]  Brian L. Hughes,et al.  On the error probability of signals in additive white Gaussian noise , 1991, IEEE Trans. Inf. Theory.

[67]  David Burshtein,et al.  Bounds on the maximum-likelihood decoding error probability of low-density parity-check codes , 2000, IEEE Trans. Inf. Theory.

[68]  G. A. Barnard,et al.  Transmission of Information: A Statistical Theory of Communications. , 1961 .

[69]  G. Poscetti An upper bound for probability of error related to a given decision region in N-dimensional signal set (Corresp.) , 1971, IEEE Trans. Inf. Theory.

[70]  Dariush Divsalar,et al.  Coding theorems for 'turbo-like' codes , 1998 .

[71]  P Erd,et al.  On the application of the borel-cantelli lemma , 1952 .

[72]  Neil J. A. Sloane,et al.  Techniques of Bounding the Probability of Decoding Error for Block Coded Modulation Structures , 1994 .

[73]  Fady Alajaji,et al.  A lower bound on the probability of a finite union of events , 2000, Discret. Math..

[74]  Fady Alajaji,et al.  An error resilient scheme for image transmission over noisy channels with memory , 1998, IEEE Trans. Image Process..

[75]  Meir Feder,et al.  Random coding techniques for nonrandom codes , 1999, IEEE Trans. Inf. Theory.

[76]  J. Craig A new, simple and exact result for calculating the probability of error for two-dimensional signal constellations , 1991, MILCOM 91 - Conference record.

[77]  Marco Ajmone Marsan,et al.  Combined coding and modulation: Theory and applications , 1988, IEEE Trans. Inf. Theory.

[79]  D. Divsalar A Simple Tight Bound on Error Probability of Block Codes with Application to Turbo Codes , 1999 .

[80]  Robert J. McEliece,et al.  Performance of binary block codes at low signal-to-noise ratios , 1992, IEEE Trans. Inf. Theory.

[81]  Emre Telatar,et al.  Mismatched decoding revisited: General alphabets, channels with memory, and the wide-band limit , 2000, IEEE Trans. Inf. Theory.