Joint Source–Channel Coding Error Exponent for Discrete Communication Systems With Markovian Memory

We study the error exponent, E<sub>J</sub>, for reliably transmitting a discrete stationary ergodic Markov (SEM) source Q over a discrete channel W with additive SEM noise via a joint source-channel (JSC) code. We first establish an upper bound for E<sub>J</sub> in terms of the Renyi entropy rates of the source and noise processes. We next investigate the analytical computation of E<sub>J</sub> by comparing our bound with Gallager's lower bound (1968) when the latter one is specialized to the SEM source-channel system. We also note that both bounds can be represented in Csiszar's form (1980), as the minimum of the sum of the source and channel error exponents. Our results provide us with the tools to systematically compare E<sub>J</sub> with the tandem (separate) coding exponent E<sub>J</sub>. We show that as in the case of memoryless source-channel pairs E<sub>J</sub> les 2E<sub>r</sub> and we provide explicit conditions for which E<sub>J</sub> > E<sub>T</sub>. Numerical results indicate that E<sub>J</sub> ap 2E<sub>T</sub> for many SEM source-channel pairs, hence illustrating a substantial advantage of JSC coding over tandem coding for systems with Markovian memory.

[1]  Fady Alajaji,et al.  A Model for Correlated Rician Fading Channels Based on a Finite Queue , 2008, IEEE Transactions on Vehicular Technology.

[2]  Giuseppe Longo,et al.  The error exponent for the noiseless encoding of finite ergodic Markov sources , 1981, IEEE Trans. Inf. Theory.

[3]  Y. Zhong.  Joint Source-Channel Coding Reliability Function for Single and Multi-Terminal Communication Systems , 2008 .

[4]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[5]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[6]  E. Seneta.  Non-negative Matrices and Markov Chains , 2008 .

[7]  Sergio Verdú,et al.  A general formula for channel capacity , 1994, IEEE Trans. Inf. Theory.

[8]  Karol Vasek.  On the error exponent for ergodic Markov source , 1980, Kybernetika.

[9]  Fady Alajaji,et al.  Rényi's divergence and entropy rates for finite alphabet Markov sources , 2001, IEEE Trans. Inf. Theory.

[10]  Richard E. Blahut,et al.  Hypothesis testing and information theory , 1974, IEEE Trans. Inf. Theory.

[11]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[12]  S. Natarajan,et al.  Large deviations, hypotheses testing, and source coding for finite Markov chains , 1985, IEEE Trans. Inf. Theory.

[13]  P. Billingsley,et al.  Ergodic theory and information , 1966 .

[14]  N. Sloane,et al.  Lower Bounds to Error Probability for Coding on Discrete Memoryless Channels. I , 1993 .

[15]  L. Finesso,et al.  The optimal Error Exponent for Markov Order Estimation , 1993, Proceedings. IEEE International Symposium on Information Theory.

[16]  Hiroki Koga,et al.  Information-Spectrum Methods in Information Theory , 2002 .

[17]  Fady Alajaji,et al.  On the joint source-channel coding error exponent for discrete memoryless systems , 2006, IEEE Transactions on Information Theory.

[18]  Katalin Marton,et al.  Error exponent for source coding with a fidelity criterion , 1974, IEEE Trans. Inf. Theory.

[19]  Imre Csiszár.  On the error exponent of source-channel transmission with a distortion threshold , 1982, IEEE Trans. Inf. Theory.

[20]  Bertrand M. Hochwald,et al.  Tradeoff between source and channel coding , 1997, Proceedings of IEEE International Symposium on Information Theory.

[21]  Valerie Isham,et al.  Non‐Negative Matrices and Markov Chains , 1983 .

[22]  Fady Alajaji,et al.  Csiszár's cutoff rates for arbitrary discrete sources , 2001, IEEE Trans. Inf. Theory.

[23]  Fady Alajaji,et al.  A Binary Communication Channel With Memory Based on a Finite Queue , 2007, IEEE Transactions on Information Theory.

[24]  Elwyn R. Berlekamp,et al.  Lower Bounds to Error Probability for Coding on Discrete Memoryless Channels. II , 1967, Inf. Control..

[25]  Cecilio Pimentel,et al.  Finite-state Markov modeling of correlated Rician-fading channels , 2004, IEEE Transactions on Vehicular Technology.