Mismatched Decoding: Error Exponents, Second-Order Rates and Saddlepoint Approximations

This paper considers the problem of channel coding with a given (possibly suboptimal) maximum-metric decoding rule. A cost-constrained random-coding ensemble with multiple auxiliary costs is introduced, and is shown to achieve error exponents and second-order coding rates matching those of constant-composition random coding, while being directly applicable to channels with infinite or continuous alphabets. The number of auxiliary costs required to match the error exponents and second-order rates of constant-composition coding is studied, and is shown to be at most two. For independent identically distributed random coding, asymptotic estimates of two well-known non-asymptotic bounds are given using saddlepoint approximations. Each expression is shown to characterize the asymptotic behavior of the corresponding random-coding bound at both fixed and varying rates, thus unifying the regimes characterized by error exponents, second-order rates, and moderate deviations. For fixed rates, novel exact asymptotics expressions are obtained to within a multiplicative 1+o(1) term. Using numerical examples, it is shown that the saddlepoint approximations are highly accurate even at short block lengths.

[1]  S. Silverman,et al.  A Local Limit Theorem and Recurrence Conditions for Sums of Independent Non-Lattice Random Variables , 1970 .

[2]  Albert Guillén i Fàbregas,et al.  Cost-constrained random coding and applications , 2013, 2013 Information Theory and Applications Workshop (ITA).

[3]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[4]  V. V. Petrov.  Sums of Independent Random Variables , 1975 .

[5]  Anelia Somekh-Baruch.  A General Formula for the Mismatch Capacity , 2015, IEEE Transactions on Information Theory.

[6]  Emre Telatar,et al.  Mismatched decoding revisited: General alphabets, channels with memory, and the wide-band limit , 2000, IEEE Trans. Inf. Theory.

[7]  R. Gallager.  Information Theory and Reliable Communication , 1968 .

[8]  Robert G. Gallager,et al.  The random coding bound is tight for the average code (Corresp.) , 1973, IEEE Trans. Inf. Theory.

[9]  H. Vincent Poor,et al.  Channel coding: non-asymptotic fundamental limits , 2010 .

[10]  Neri Merhav,et al.  Expurgated Random-Coding Ensembles , 2014 .

[11]  N. N. Amosova,et al.  Probabilities of moderate deviations , 1982 .

[12]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[13]  Imre Csiszár.  Generalized cutoff rates and Renyi's information measures , 1995, IEEE Trans. Inf. Theory.

[14]  K Fan,et al.  Minimax Theorems. , 1953, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Hiroki Koga,et al.  Information-Spectrum Methods in Information Theory , 2002 .

[16]  S. Verdú,et al.  Channel dispersion and moderate deviations limits for memoryless channels , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[17]  Masahito Hayashi,et al.  Information Spectrum Approach to Second-Order Coding Rate in Channel Coding , 2008, IEEE Transactions on Information Theory.

[18]  Albert Guillén i Fàbregas,et al.  Multiuser Random Coding Techniques for Mismatched Decoding , 2013, IEEE Transactions on Information Theory.

[19]  Nadav Shulman,et al.  Communication over an unknown channel via common broadcasting , 2003 .

[20]  H. Vincent Poor,et al.  Channel Coding Rate in the Finite Blocklength Regime , 2010, IEEE Transactions on Information Theory.

[21]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[22]  R. L. Dobrushin,et al.  Asymptotic Estimates of the Probability of Error for Transmission of Messages over a Discrete Memoryless Communication Channel with a Symmetric Transition Probability Matrix , 1962 .

[23]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[24]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[25]  Aaron B. Wagner,et al.  Moderate Deviations in Channel Coding , 2012, IEEE Transactions on Information Theory.

[26]  Juan Carlos Abril,et al.  Saddlepoint Approximations , 2011, International Encyclopedia of Statistical Science.

[27]  Philip S. Griffin,et al.  Matrix Normalized Sums of Independent Identically Distributed Random Vectors , 1986 .

[28]  Anelia Somekh-Baruch.  A General Formula for the Mismatch Capacity , 2015, IEEE Trans. Inf. Theory.

[29]  Vladimir B. Balakirsky.  A converse coding theorem for mismatched decoding at the output of binary-input memoryless channels , 1995, IEEE Trans. Inf. Theory.

[30]  Albert Guillén i Fàbregas,et al.  A derivation of the asymptotic random-coding prefactor , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[31]  S. Shamai,et al.  Information rates and error exponents of compound channels with application to antipodal signaling in a fading environment , 1993 .

[32]  Dominique de Caen,et al.  A lower bound on the probability of a union , 1997, Discret. Math..

[33]  Imre Csiszár,et al.  Channel capacity for a given decoding metric , 1995, IEEE Trans. Inf. Theory.

[34]  R. R. Bahadur,et al.  On Deviations of the Sample Mean , 1960 .

[35]  V. V. Petrov.  On Probabilities of Moderate Deviations , 2002 .

[36]  Imre Csiszár,et al.  Graph decomposition: A new key to coding theorems , 1981, IEEE Trans. Inf. Theory.

[37]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[38]  Anelia Somekh-Baruch.  On Achievable Rates for Channels with Mismatched Decoding , 2013, ArXiv.

[39]  Albert Guillén i Fàbregas,et al.  Random-coding bounds for threshold decoders: Error exponent and saddlepoint approximation , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[40]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[41]  Joseph Y. N. Hui,et al.  Fundamental issues of multiple accessing , 1983 .

[42]  Shlomo Shamai,et al.  On information rates for mismatched decoders , 1994, IEEE Trans. Inf. Theory.

[43]  Johan Efberg,et al.  YALMIP : A toolbox for modeling and optimization in MATLAB , 2004 .

[44]  Albert Guillén i Fàbregas,et al.  Saddlepoint approximation of random-coding bounds , 2011, 2011 Information Theory and Applications Workshop.

[46]  Shlomo Shamai,et al.  Variations on the Gallager bounds, connections, and applications , 2002, IEEE Trans. Inf. Theory.

[47]  Amos Lapidoth,et al.  Mismatched decoding and the multiple-access channel , 1994, IEEE Trans. Inf. Theory.

[48]  Charles Stone,et al.  On local and ratio limit theorems , 1967 .

[49]  Aaron B. Wagner,et al.  Refinement of the Sphere-Packing Bound: Asymmetric Channels , 2012, IEEE Transactions on Information Theory.