Joint Source-Channel Decoding of Variable-Length Codes with Soft Information: A Survey

Multimedia transmission over time-varying wireless channels presents a number of challenges beyond existing capabilities conceived so far for third-generation networks. Efficient quality-of-service (QoS) provisioning for multimedia on these channels may in particular require a loosening and a rethinking of the layer separation principle. In that context, joint source-channel decoding (JSCD) strategies have gained attention as viable alternatives to separate decoding of source and channel codes. A statistical framework based on hidden Markov models (HMM) capturing dependencies between the source and channel coding components sets the foundation for optimal design of techniques of joint decoding of source and channel codes. The problem has been largely addressed in the research community, by considering both fixed-length codes (FLC) and variable-length source codes (VLC) widely used in compression standards. Joint source-channel decoding of VLC raises specific difficulties due to the fact that the segmentation of the received bitstream into source symbols is random. This paper makes a survey of recent theoretical and practical advances in the area of JSCD with soft information of VLC-encoded sources. It first describes the main paths followed for designing efficient estimators for VLC-encoded sources, the key component of the JSCD iterative structure. It then presents the main issues involved in the application of the turbo principle to JSCD of VLC-encoded sources as well as the main approaches to source-controlled channel decoding. This survey terminates by performance illustrations with real image and video decoding systems.

[1]  Brendan J. Frey,et al.  A Revolution: Belief Propagation in Graphs with Cycles , 1997, NIPS.

[2]  Marion Jeanne,et al.  Source and joint source-channel decoding of variable length codes , 2002, 2002 IEEE International Conference on Communications. Conference Proceedings. ICC 2002 (Cat. No.02CH37333).

[3]  Andrew J. Viterbi,et al.  An Intuitive Justification and a Simplified Implementation of the MAP Decoder for Convolutional Codes , 1998, IEEE J. Sel. Areas Commun..

[4]  Victor Buttigieg,et al.  On variable-length error-correcting codes , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[5]  Lars-Åke Larzon,et al.  The UDP-Lite Protocol , 2001 .

[6]  Joachim Hagenauer,et al.  Symbol-by-Symbol MAP Decoding of Variable Length Codes , 2000 .

[7]  John D. Villasenor,et al.  Utilizing soft information in decoding of variable length codes , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[8]  Robert Mario Fano,et al.  A heuristic discussion of probabilistic decoding , 1963, IEEE Trans. Inf. Theory.

[9]  Khalid Sayood,et al.  Joint source/channel coding for variable length codes , 2000, IEEE Trans. Commun..

[10]  S. Pasupathy,et al.  Low-complexity parallel-structure symbol-by-symbol detection for ISI channels , 1989, Conference Proceeding IEEE Pacific Rim Conference on Communications, Computers and Signal Processing.

[11]  J. Vaisey,et al.  Joint source-channel decoding of entropy coded Markov sources over binary symmetric channels , 1999, 1999 IEEE International Conference on Communications (Cat. No. 99CH36311).

[12]  Daniel J. Costello,et al.  Turbo codes for image transmission-a joint channel and source decoding approach , 2000, IEEE Journal on Selected Areas in Communications.

[13]  Nariman Farvardin,et al.  A study of vector quantization for noisy channels , 1990, IEEE Trans. Inf. Theory.

[14]  Joachim Hagenauer,et al.  Iterative source/channel-decoding using reversible variable length codes , 2000, Proceedings DCC 2000. Data Compression Conference.

[15]  John Cocke,et al.  Optimal decoding of linear codes for minimizing symbol error rate (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[16]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[17]  S. Merigeault,et al.  Concepts for exchanging extra information between protocol layers transparently for the standard protocol stack , 2003, 10th International Conference on Telecommunications, 2003. ICT 2003..

[18]  George F. Elmasry.  Joint lossless-source and channel coding using automatic repeat request , 1999, IEEE Trans. Commun..

[19]  Joachim Hagenauer,et al.  The turbo principle-tutorial introduction and state of the art , 1997 .

[20]  Tamotsu Kasai,et al.  A Method for the Correction of Garbled Words Based on the Levenshtein Metric , 1976, IEEE Transactions on Computers.

[21]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[22]  Javier Garcia-Frías,et al.  Joint turbo decoding and estimation of hidden Markov sources , 2001, IEEE J. Sel. Areas Commun..

[23]  Patrick Robertson,et al.  A comparison of optimal and sub-optimal MAP decoding algorithms operating in the log domain , 1995, Proceedings IEEE International Conference on Communications ICC '95.

[24]  Khalid Sayood,et al.  Joint source channel coding using arithmetic codes and trellis coded modulation , 2001, Proceedings DCC 2001. Data Compression Conference.

[25]  Marc P. C. Fossorier,et al.  On the equivalence between SOVA and max-log-MAP decodings , 1998, IEEE Communications Letters.

[26]  A. R. Reibman,et al.  Self-synchronizing variable-length codes for image transmission , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[27]  Joachim Hagenauer,et al.  On variable length codes for iterative source/channel decoding , 2001, Proceedings DCC 2001. Data Compression Conference.

[28]  Enrico Magli,et al.  Error correcting arithmetic coding for robust video compression , 2003 .

[29]  G. F. Elmasry.  Embedding channel coding in arithmetic coding , 1999 .

[30]  Jeffrey Scott Vitter,et al.  Design and Analysis of Fast Text Compression Based on Quasi-Arithmetic Coding , 1994, Inf. Process. Manag..

[31]  David J. Miller,et al.  Joint source-channel decoding for variable-length encoded data by exact and approximate MAP sequence estimation , 2000, IEEE Trans. Commun..

[32]  A. Elbaz,et al.  Iterative decoding of product codes with a priori information over a Gaussian channel for still image transmission , 1999, Seamless Interconnection for Universal Services. Global Telecommunications Conference. GLOBECOM'99. (Cat. No.99CH37042).

[33]  Ragnar Thobaben,et al.  Parallel concatenated joint source-channel coding , 2003 .

[34]  John D. Villasenor,et al.  Soft-input soft-output decoding of variable length codes , 2002, IEEE Trans. Commun..

[35]  Peter Vary,et al.  Convergence behavior of iterative source-channel decoding , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[36]  Shu Lin,et al.  Error control coding : fundamentals and applications , 1983 .

[37]  James L. Massey.  Variable-length codes and the Fano metric , 1972, IEEE Trans. Inf. Theory.

[38]  Marion Jeanne,et al.  Joint source-channel decoding of variable-length codes for convolutional codes and turbo codes , 2005, IEEE Transactions on Communications.

[39]  Jun Tan,et al.  A MAP equivalent SOVA for non-binary turbo codes , 2000, 2000 IEEE International Conference on Communications. ICC 2000. Global Convergence Through Communications. Conference Record.

[40]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[41]  Thomas E. Fuja,et al.  Joint source-channel decoding of variable-length encoded sources , 1998, 1998 Information Theory Workshop (Cat. No.98EX131).

[42]  Pierre Siohan,et al.  Joint source-channel soft decoding of Huffman codes with turbo-codes , 2000, Proceedings DCC 2000. Data Compression Conference.

[43]  Marion Jeanne,et al.  Comparaison de deux approches du décodage conjoint source-canal , 2003 .

[44]  Touradj Ebrahimi,et al.  The JPEG2000 still image coding system: an overview , 2000, IEEE Trans. Consumer Electron..

[45]  N. Gortz.  On the iterative approximation of optimal joint source-channel decoding , 2001 .

[46]  Joachim Hagenauer,et al.  A Viterbi algorithm with soft-decision outputs and its applications , 1989, IEEE Global Telecommunications Conference, 1989, and Exhibition. 'Communications Technology for the 1990s and Beyond.

[47]  Stephan ten Brink,et al.  Convergence behavior of iteratively decoded parallel concatenated codes , 2001, IEEE Trans. Commun..

[48]  Javier Garcia-Frías,et al.  Combining hidden Markov source models and parallel concatenated codes , 1997, IEEE Communications Letters.

[49]  Christine Guillemot,et al.  Joint source-channel turbo decoding of entropy-coded sources , 2001, IEEE J. Sel. Areas Commun..

[50]  J. Vitter,et al.  Practical Implementations of Arithmetic Coding , 1991 .

[51]  R. Chang,et al.  On receiver structures for channels having memory , 1966, IEEE Trans. Inf. Theory.

[52]  Thomas Guionnet,et al.  Soft decoding and synchronization of arithmetic codes: application to image transmission over noisy channels , 2003, IEEE Trans. Image Process..

[53]  Nam C. Phamdo,et al.  Optimal detection of discrete Markov sources over discrete memoryless channels - applications to combined source-channel coding , 1994, IEEE Trans. Inf. Theory.

[54]  N. Phamdo,et al.  Optimal Detection of Discrete Markov Sources Over Discrete Memoryless Channels - Applications to Combined Source-Channel Coding , 1993, Proceedings. IEEE International Symposium on Information Theory.

[55]  David J. Miller,et al.  A sequence-based approximate MMSE decoder for source coding over noisy channels using discrete hidden Markov models , 1998, IEEE Trans. Commun..

[56]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[57]  Andrea Goldsmith.  Joint source/channel coding for wireless channels , 1995, 1995 IEEE 45th Vehicular Technology Conference. Countdown to the Wireless Twenty-First Century.

[58]  John G. Cleary,et al.  Integrating error detection into arithmetic coding , 1997, IEEE Trans. Commun..

[59]  Joachim Hagenauer,et al.  Rate-compatible punctured convolutional codes (RCPC codes) and their applications , 1988, IEEE Trans. Commun..

[60]  Khalid Sayood,et al.  Joint source/channel coding using arithmetic codes , 2001, IEEE Trans. Commun..

[61]  M. Wada,et al.  Reversible variable length codes , 1995, IEEE Trans. Commun..

[62]  John D. Villasenor,et al.  Combining variable length codes and turbo codes , 2002, Vehicular Technology Conference. IEEE 55th Vehicular Technology Conference. VTC Spring 2002 (Cat. No.02CH37367).

[63]  Marion Jeanne.  Etude de systèmes robustes de décodage conjoint source-canal pour la transmission sans fil de vidéo , 2003 .

[64]  Maja Bystrom,et al.  Sequential Decoding of MPEG-4 Coded Bitstreams for Error Resilience , 1999 .

[65]  Lisa Perros-Meilhac,et al.  LOW COMPLEXITY ITERATIVE DECODING OF VARIABLE-LENGTH CODES , 2003 .

[66]  T.E. Fuja,et al.  Channel codes that exploit the residual redundancy in CELP-encoded speech , 1996, IEEE Trans. Speech Audio Process..

[67]  David S. Taubman,et al.  High performance scalable image compression with EBCOT , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[68]  J. Hagenauer,et al.  Sequential decoding using a priori information , 1996 .

[69]  Iraj Sodagar,et al.  A new error resilience technique for image compression using arithmetic coding , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[70]  Brendan J. Frey,et al.  Iterative Decoding of Compound Codes by Probability Propagation in Graphical Models , 1998, IEEE J. Sel. Areas Commun..

[71]  Dariush Divsalar,et al.  Iterative decoding of serially concatenated codes with interleavers and comparison with turbo codes , 1997, GLOBECOM 97. IEEE Global Telecommunications Conference. Conference Record.

[72]  Thomas E. Fuja,et al.  Robust transmission of variable-length encoded sources , 1999, WCNC. 1999 IEEE Wireless Communications and Networking Conference (Cat. No.99TH8466).

[73]  Nariman Farvardin,et al.  On the performance and complexity of channel-optimized vector quantizers , 1991, IEEE Trans. Inf. Theory.

[74]  Detlev Marpe,et al.  Video compression using context-based adaptive arithmetic coding , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[75]  Thomas J. Ferguson,et al.  Self-synchronizing Huffman codes , 1984, IEEE Trans. Inf. Theory.

[76]  Joachim Hagenauer,et al.  Source-controlled channel decoding , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[77]  F. Jelinek.  Fast sequential decoding algorithm using a stack , 1969 .

[78]  Fady Alajaji,et al.  Turbo codes for nonuniform memoryless sources over noisy channels , 2002, IEEE Communications Letters.

[79]  Thomas Guionnet,et al.  Soft and Joint Source-Channel Decoding of Quasi-Arithmetic Codes , 2004, EURASIP J. Adv. Signal Process..

[80]  Khalid Sayood,et al.  Use of residual redundancy in the design of joint source/channel coders , 1991, IEEE Trans. Commun..

[81]  Alain Glavieux,et al.  Reflections on the Prize Paper : "Near optimum error-correcting coding and decoding: turbo codes" , 1998 .

[82]  Lisa Perros-Meilhac,et al.  Huffman tree based metric derivation for a low-complexity sequential soft VLC decoding , 2002, 2002 IEEE International Conference on Communications. Conference Proceedings. ICC 2002 (Cat. No.02CH37333).

[83]  Jerry D. Gibson,et al.  A constrained joint source/channel coder design , 1994, IEEE J. Sel. Areas Commun..