A Tutorial and Review on Inter-Layer FEC Coded Layered Video Streaming

Layered video coding creates multiple layers of unequal importance, which enables us to progressively refine the reconstructed video quality. When the base layer (BL) is corrupted or lost during transmission, the enhancement layers (ELs) must be dropped, regardless whether they are perfectly decoded or not, which implies that the transmission power assigned to the ELs is wasted. For the sake of combating this problem, the class of inter-layer forward error correction (IL-FEC) solutions, also referred to as layer-aware FEC (LA-FEC), has been proposed for layered video transmissions, which jointly encode the BL and the ELs, thereby protecting the BL using the ELs. This tutorial aims for inspiring further research on IL-FEC/LA-FEC techniques, with special emphasis on the family of soft-decoded bit-level IL-FEC schemes.

[1]  Gary J. Sullivan,et al.  Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard , 2011, Proceedings of the IEEE.

[2]  Guosen Yue,et al.  Message-Wise Unequal Error Protection Based on Low-Density Parity-Check Codes , 2011, IEEE Transactions on Communications.

[3]  Faouzi Kossentini,et al.  Rate-distortion optimized layered coding with unequal error protection for robust Internet video , 2001, IEEE Trans. Circuits Syst. Video Technol..

[4]  Ville Lappalainen,et al.  Comparison of video protection methods for wireless networks , 2003, Signal Process. Image Commun..

[5]  Lajos Hanzo,et al.  EXIT-Chart Optimized Short Block Codes for Iterative Joint Source and Channel Decoding in H.264 Video Telephony , 2009, IEEE Transactions on Vehicular Technology.

[6]  Alan F. Lippman,et al.  Video coding for streaming media delivery on the Internet , 2001, IEEE Trans. Circuits Syst. Video Technol..

[7]  Reuben A. Farrugia,et al.  Correlation Noise-Based Unequal Error Protected Rate-Adaptive Codes for Distributed Video Coding , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Lajos Hanzo,et al.  Voice Compression and Communications , 2001 .

[9]  Stephan Wenger,et al.  H.264/AVC over IP , 2003, IEEE Trans. Circuits Syst. Video Technol..

[10]  J. Ostermann,et al.  Congestion Control for Scalable Video Streaming Using the Scalability Extension of H.264/AVC , 2007, IEEE Journal of Selected Topics in Signal Processing.

[11]  Qionghai Dai,et al.  A Novel JSCC Framework With Diversity-Multiplexing-Coding Gain Tradeoff for Scalable Video Transmission Over Cooperative MIMO , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Vijay Sivaraman,et al.  PET Protection Optimization for Streaming Scalable Videos With Multiple Transmissions , 2013, IEEE Transactions on Image Processing.

[13]  Sorina Dumitrescu,et al.  Unequal Erasure Protection Technique for Scalable Multistreams , 2009, IEEE Transactions on Image Processing.

[14]  Lajos Hanzo,et al.  Turbo- and BCH-coded wide-band burst-by-burst adaptive H.263-assisted wireless video telephony , 2000, IEEE Trans. Circuits Syst. Video Technol..

[15]  Heiko Schwarz,et al.  Performance Analysis of SVC , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Zixiang Xiong,et al.  Low bit-rate scalable video coding with 3-D set partitioning in hierarchical trees (3-D SPIHT) , 2000, IEEE Trans. Circuits Syst. Video Technol..

[17]  Thomas Stockhammer,et al.  Raptor Forward Error Correction Scheme for Object Delivery , 2007, RFC.

[18]  Chang Wen Chen,et al.  Scalable H.264/AVC Video Transmission Over MIMO Wireless Systems With Adaptive Channel Selection Based on Partial Channel Information , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Kai-Kuang Ma,et al.  Reducing video-quality fluctuations for streaming scalable video using unequal error protection, retransmission, and interleaving , 2006, IEEE Trans. Image Process..

[20]  Narendra Ahuja,et al.  Wyner-Ziv coding of video: an error-resilient compression framework , 2004, IEEE Transactions on Multimedia.

[21]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, ANTS Conference.

[22]  Lajos Hanzo,et al.  Two-Dimensional Iterative Source-Channel Decoding for Distributed Video Coding , 2014, IEEE Communications Letters.

[23]  Sampath Rangarajan,et al.  Cross-layer optimization for streaming scalable video over fading wireless networks , 2010, IEEE Journal on Selected Areas in Communications.

[24]  Aggelos K. Katsaggelos,et al.  Cost-distortion optimized unequal error protection for object-based video communications , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Yixuan Zhang,et al.  A Joint Source-Channel Video Coding Scheme Based on Distributed Source Coding , 2008, IEEE Transactions on Multimedia.

[26]  Miska M. Hannuksela,et al.  H.264/AVC in wireless environments , 2003, IEEE Trans. Circuits Syst. Video Technol..

[27]  Olgica Milenkovic,et al.  On unequal error protection LDPC codes based on Plotkin-type constructions , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[28]  Lajos Hanzo,et al.  Orthogonal frequency-division multiplex transmission of H.263 encoded video over highly frequency-selective wireless networks , 1999, IEEE Trans. Circuits Syst. Video Technol..

[29]  Lajos Hanzo,et al.  Burst-by-burst adaptive decision feedback equalized TCM, TTCM, and BICM for H.263-assisted wireless video telephony , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Johannes Peltola,et al.  Cross-layer architecture for scalable video transmission in wireless network , 2007, Signal Process. Image Commun..

[31]  Ying Chen,et al.  Error Resilient Coding and Error Concealment in Scalable Video Coding , 2009 .

[32]  Si Wu,et al.  Unequal loss protection for robust transmission of motion compensated video over the internet , 2003, Signal Process. Image Commun..

[33]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[34]  Wolfgang Kellerer,et al.  SVEF: an open-source experimental evaluation framework for H.264 scalable video streaming , 2009, 2009 IEEE Symposium on Computers and Communications.

[35]  Mohammed Ghanbari,et al.  Layered H.264 video transmission with hierarchical QAM , 2006, J. Vis. Commun. Image Represent..

[36]  Cornelius Hellge,et al.  Prioritized transmission of layered media with layer-aware FEC , 2013 .

[37]  K. Ramchandran,et al.  Multiple description source coding using forward error correction codes , 1999, Conference Record of the Thirty-Third Asilomar Conference on Signals, Systems, and Computers (Cat. No.CH37020).

[38]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.

[39]  Bahram Javidi,et al.  Three-Dimensional Television, Video and Display Technology , 2002 .

[40]  M. Ghanbari,et al.  Combined turbo coding and hierarchical QAM for unequal error protection of H.264 coded video , 2006, Signal Process. Image Commun..

[41]  Lajos Hanzo,et al.  Inter-layer-decoding aided self-concatenated coded scalable video transmission , 2013, 2013 IEEE Wireless Communications and Networking Conference (WCNC).

[42]  Hayder Radha,et al.  Network Coding with Multi-Generation Mixing: A Generalized Framework for Practical Network Coding , 2011, IEEE Transactions on Wireless Communications.

[43]  Wanjiun Liao,et al.  Cooperative Multicasting for Wireless Scalable Video Transmissions , 2013, IEEE Transactions on Communications.

[44]  Kavé Salamatian,et al.  A new analytic approach to evaluation of packet error rate in wireless networks , 2005, 3rd Annual Communication Networks and Services Research Conference (CNSR'05).

[45]  Itu-T Video coding for low bitrate communication , 1996 .

[46]  Mathias Wien,et al.  Real-Time System for Adaptive Video Streaming Based on SVC , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[47]  Fernando Jaureguizar,et al.  Enhancement of Pro-MPEG COP3 codes and application to layer-aware FEC protection of two-layered video transmission , 2013, 2013 IEEE International Conference on Image Processing.

[48]  Salman Khan,et al.  A Link Adaptation Scheme for Efficient Transmission of H.264 Scalable Video Over Multirate WLANs , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  van Wj Wil Gils,et al.  On linear unequal error protection codes , 1982 .

[50]  Nazanin Rahnavard,et al.  Rateless Codes With Unequal Error Protection Property , 2007, IEEE Transactions on Information Theory.

[51]  Peter Vary,et al.  Unequal error protection by modulation with unequal power allocation , 2005, IEEE Communications Letters.

[52]  Stephan ten Brink,et al.  Convergence behavior of iteratively decoded parallel concatenated codes , 2001, IEEE Trans. Commun..

[53]  Zhenzhong Chen,et al.  Scalable Resource Allocation for SVC Video Streaming Over Multiuser MIMO-OFDM Networks , 2013, IEEE Transactions on Multimedia.

[54]  John W. Woods,et al.  Adaptive two-stage FEC scheme for scalable video transmission over wireless networks , 2009, Signal Process. Image Commun..

[55]  Ajay Luthra,et al.  MPEG-2 multiview profile and its application in 3D TV , 1997, Electronic Imaging.

[56]  Chee Kheong Siew,et al.  Forward Error Correction-Based 2-D Layered Multiple Description Coding for Error-Resilient H.264 SVC Video Transmission , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[57]  Lap-Pui Chau,et al.  GOP-based channel rate allocation using genetic algorithm for scalable video streaming over error-prone networks , 2006, IEEE Transactions on Image Processing.

[58]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[59]  Yücel Altunbasak,et al.  A multiple-substream unequal error-protection and error-concealment algorithm for SPIHT-coded video bitstreams , 2004, IEEE Transactions on Image Processing.

[60]  Yong Pei,et al.  An Adaptive Motion-Based Unequal Error Protection Approach for Real-Time Video Transport Over Wireless IP Networks , 2006, IEEE Transactions on Multimedia.

[61]  Michael Mitzenmacher,et al.  A digital fountain approach to asynchronous reliable multicast , 2002, IEEE J. Sel. Areas Commun..

[62]  Peng Li,et al.  A novel hierarchical QAM-based unequal error protection scheme for H.264/AVC video over frequency-selective fading channels , 2010, IEEE Transactions on Consumer Electronics.

[63]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[64]  Aggelos K. Katsaggelos,et al.  Unequal Error Protection for Robust Streaming of Scalable Video Over Packet Lossy Networks , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[65]  Lajos Hanzo,et al.  Near-Capacity H.264 Multimedia Communications Using Iterative Joint Source-Channel Decoding , 2012, IEEE Communications Surveys & Tutorials.

[66]  Lajos Hanzo,et al.  Spatio-Temporal Iterative Source–Channel Decoding Aided Video Transmission , 2013, IEEE Transactions on Vehicular Technology.

[67]  Carlos Miguel Tavares Calafate,et al.  Evaluation of the H.264 Scalable Video Coding in Error Prone IP Networks , 2008, IEEE Transactions on Broadcasting.

[68]  Thomas Stockhammer,et al.  RaptorQ Forward Error Correction Scheme for Object Delivery , 2011, RFC.

[69]  Lajos Hanzo,et al.  Video Compression and Communications: From Basics to H.261, H.263, H.264, MPEG4 for DVB and HSDPA-Style Adaptive Turbo-Transceivers , 2007 .

[70]  Sze Wei Lee,et al.  A fast forward error correction allocation algorithm for unequal error protection of video transmission over wireless channels , 2008, IEEE Transactions on Consumer Electronics.

[71]  Wen Gao,et al.  AVS standard - Audio Video Coding Standard Workgroup of China , 2005, 14th Annual International Conference on Wireless and Optical Communications, 2005. WOCC 2005.

[72]  Lajos Hanzo,et al.  Short block codes for guaranteed convergence in soft-bit assisted iterative joint source and channel decoding , 2008 .

[73]  Lajos Hanzo,et al.  Near-Capacity Multi-Functional MIMO Systems , 2009 .

[74]  Joachim Hagenauer,et al.  Iterative decoding of binary block and convolutional codes , 1996, IEEE Trans. Inf. Theory.

[75]  Shakeel Ahmad,et al.  Unequal Error Protection Using Fountain Codes With Applications to Video Communication , 2011, IEEE Transactions on Multimedia.

[76]  Aggelos K. Katsaggelos,et al.  Error resilient video coding techniques , 2000, IEEE Signal Process. Mag..

[77]  Lajos Hanzo,et al.  Near-Capacity Multi-Functional MIMO Systems: Sphere-Packing, Iterative Detection and Cooperation , 2009 .

[78]  Lajos Hanzo,et al.  Programmable H.263-based wireless video transceivers for interference-limited environments , 1998, IEEE Trans. Circuits Syst. Video Technol..

[79]  Ying Chen,et al.  The Emerging MVC Standard for 3D Video Services , 2008, EURASIP J. Adv. Signal Process..

[80]  Hua Yang,et al.  GOP-level transmission distortion modeling for mobile streaming video , 2008, Signal Process. Image Commun..

[81]  Dan Schonfeld,et al.  A new method for multidimensional optimization and its application in image and video processing , 2006, IEEE Signal Processing Letters.

[82]  Rolf Johannesson,et al.  Unequal error protection for convolutional codes , 2006, IEEE Transactions on Information Theory.

[83]  Ajay Dholakia,et al.  Reduced-complexity decoding of LDPC codes , 2005, IEEE Transactions on Communications.

[84]  Fan Yang,et al.  End-to-end TCP-friendly streaming protocol and bit allocation for scalable video over wireless Internet , 2004, IEEE Journal on Selected Areas in Communications.

[85]  B. Sklar,et al.  Rayleigh fading channels in mobile digital communication systems Part I: Characterization , 1997, IEEE Commun. Mag..

[86]  Nazanin Rahnavard,et al.  Unequal Error Protection Using Partially Regular LDPC Codes , 2007, IEEE Transactions on Communications.

[87]  Joumana Farah,et al.  A novel approach to achieve unequal error protection for video transmission over 3G wireless networks , 2004, Signal Process. Image Commun..

[88]  Sen-Ching S. Cheung,et al.  Video Streaming with Network Coding , 2010, J. Signal Process. Syst..

[89]  Yoong Choon Chang,et al.  A low complexity hierarchical QAM symbol bits allocation algorithm for unequal error protection of wireless video transmission , 2009, IEEE Transactions on Consumer Electronics.

[90]  Ya-Qin Zhang,et al.  Transporting real-time video over the Internet: challenges and approaches , 2000, Proceedings of the IEEE.

[91]  Lajos Hanzo,et al.  Motion-Aware Mesh-Structured Trellis for Correlation Modelling Aided Distributed Multi-View Video Coding , 2014, IEEE Transactions on Image Processing.

[92]  Chu-Sing Yang,et al.  Optimal FEC assignment for scalable video transmission over burst error channel with loss rate feedback , 2003, Signal Process. Image Commun..

[93]  Thomas Wiegand,et al.  Multidimensional Layered Forward Error Correction Using Rateless Codes , 2008, 2008 IEEE International Conference on Communications.

[94]  Thomas Stockhammer,et al.  Application Layer Forward Error Correction for Mobile Multimedia Broadcasting , 2008 .

[95]  Yao Zhao,et al.  Dynamic Sub-GOP Forward Error Correction Code for Real-Time Video Applications , 2012, IEEE Transactions on Multimedia.

[96]  Mahbub Hassan,et al.  Internet telephony: services, technical challenges, and products , 2000, IEEE Commun. Mag..

[97]  Wei Xiang,et al.  Unequal error protection scheme based hierarchical 16-QAM for 3-D video transmission , 2012, IEEE Transactions on Consumer Electronics.

[98]  Nazanin Rahnavard,et al.  New results on unequal error protection using LDPC codes , 2006, IEEE Communications Letters.

[99]  Sun Yong Kim,et al.  Motion-Compensated Scalable Video Transmission over MIMO Wireless Channels under Imperfect Channel Estimation , 2009, GLOBECOM.

[100]  Robert W. Heath,et al.  A Cross-Layer Design for Perceptual Optimization Of H.264/SVC with Unequal Error Protection , 2012, IEEE Journal on Selected Areas in Communications.

[101]  Mihaela van der Schaar,et al.  Adaptive cross-layer protection strategies for robust scalable video transmission over 802.11 WLANs , 2003, IEEE J. Sel. Areas Commun..

[102]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1992 .

[103]  M. Chakraborty,et al.  Unequally protected SPIHT video codec for low bit rate transmission over highly error-prone mobile channels , 2002, Signal Process. Image Commun..

[104]  Chung-Lin Huang,et al.  Unequal error protection for MPEG-2 video transmission over wireless channels , 2004, Signal Process. Image Commun..

[105]  Youshi Xu,et al.  Unequal packet loss protection for layered video transmission , 1999, IEEE Trans. Broadcast..

[106]  Velio Tralli,et al.  Distortion-Fair Cross-Layer Resource Allocation for Scalable Video Transmission in OFDMA Wireless Networks , 2014, IEEE Transactions on Multimedia.

[107]  Michael Tüchler,et al.  EXIT CHART ANALYSIS APPLIED TO ADAPTIVE TURBO EQUALIZATION , 2002 .

[108]  Michael Luby,et al.  LT codes , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[109]  Lajos Hanzo,et al.  Wireless Video: An Interlayer Error-Protection-Aided Multilayer Approach , 2014, IEEE Vehicular Technology Magazine.

[110]  Madhu Sudan,et al.  Priority encoding transmission , 1996, IEEE Trans. Inf. Theory.

[111]  Chien-Min Ou,et al.  Layered video transmission based on genetic programming for lossy channels , 2004, Neurocomputing.

[112]  Zixiang Xiong,et al.  Scalable Video Multicast Using Expanding Window Fountain Codes , 2009, IEEE Transactions on Multimedia.

[113]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[114]  John W. Woods,et al.  Scalable Video Streaming With Fine-Grain Adaptive Forward Error Correction , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[115]  Lajos Hanzo,et al.  Iterative source and channel decoding relying on correlation modelling for wireless video transmission , 2013, IET Commun..

[116]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[117]  Dejan Vukobratovic,et al.  Expanding Window Fountain Codes for Unequal Error Protection , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[118]  Thomas Wiegand,et al.  Mobile Video Transmission Using Scalable Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[119]  Hojin Ha,et al.  Layer-weighted unequal error protection for scalable video coding extension of H.264/AVC , 2008, IEEE Transactions on Consumer Electronics.

[120]  Stephan ten Brink,et al.  Design of low-density parity-check codes for modulation and detection , 2004, IEEE Transactions on Communications.

[121]  Lajos Hanzo,et al.  Inter-Layer FEC Aided Unequal Error Protection for Multilayer Video Transmission in Mobile TV , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[122]  Masoud Salehi,et al.  Turbo Coded Modulation for Unequal Error Protection , 2008, IEEE Transactions on Communications.

[123]  Lajos Hanzo,et al.  Extrinsic Information Transfer Analysis and Design of Block-Based Intermediate Codes , 2011, IEEE Transactions on Vehicular Technology.

[124]  Yoong Choon Chang,et al.  A low-complexity unequal error protection of H.264/AVC video using adaptive hierarchical QAM , 2006, IEEE Transactions on Consumer Electronics.

[125]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[126]  Thomas Wiegand,et al.  Layer-Aware Forward Error Correction for Mobile Broadcast of Layered Media , 2011, IEEE Transactions on Multimedia.

[127]  Robert G. Maunder,et al.  Layered Wireless Video Relying on Minimum-Distortion Inter-Layer FEC Coding , 2014, IEEE Transactions on Multimedia.

[128]  Lajos Hanzo,et al.  Layered steered space-time codes and their capacity , 2007 .

[129]  K. Rijkse,et al.  H.263: video coding for low-bit-rate communication , 1996, IEEE Commun. Mag..

[130]  Lajos Hanzo,et al.  Inter-layer FEC decoded multi-layer video streaming , 2012, 2012 IEEE Global Communications Conference (GLOBECOM).

[131]  Yücel Altunbasak,et al.  Error-resilient image and video transmission over the Internet using unequal error protection , 2003, IEEE Trans. Image Process..

[132]  Ali Aghagolzadeh,et al.  Robust transmission of scalable video stream using modified LT codes , 2011, Comput. Electr. Eng..

[133]  Qian Zhang,et al.  Channel-adaptive resource allocation for scalable video transmission over 3G wireless network , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[134]  Lionel Bouchard,et al.  Multimedia Software for Mobile Phones , 2010, IEEE Software.

[135]  B. L. Yeap,et al.  Turbo Coding, Turbo Equalisation and Space-Time Coding , 2002 .

[136]  Somnath Sengupta,et al.  Scene content driven FEC allocation for video streaming , 2014, Signal Process. Image Commun..

[137]  Yao Zhao,et al.  Real-Time Video Streaming Using Randomized Expanding Reed–Solomon Code , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[138]  K. R. Rao,et al.  High efficiency video coding , 2016, 2016 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA).

[139]  Itu-T and Iso Iec Jtc Advanced video coding for generic audiovisual services , 2010 .

[140]  Yiwei Thomas Hou,et al.  Scalable video coding and transport over broadband wireless networks , 2001, Proc. IEEE.

[141]  Zhihai He,et al.  Rate-distortion optimized unequal loss protection for video transmission over packet erasure channels , 2013, Signal Process. Image Commun..

[142]  Songyu Yu,et al.  Adaptive unequal loss protection for scalable video streaming over IP networks , 2005, IEEE Transactions on Consumer Electronics.

[143]  Hojin Ha,et al.  Packet loss resilience using unequal forward error correction assignment for video transmission over communication networks , 2007, Comput. Commun..

[144]  Lajos Hanzo,et al.  Turbo-detected unequal protection MPEG-4 wireless video telephony using multi-level coding, trellis coded modulation and space-time trellis coding , 2005 .